Build with Python 3.12 fails

[rhauser]

Hi,

we (ATLAS TDAQ) are seeing build failures of ipbus in experimental builds against Python 3.12. They seem to be mostly caused by the removal of distutils in this release (https://docs.python.org/3.12/whatsnew/3.12.html#distutils).

I worked around the build failures by adding an 'import setuptools' in various places before distutils is used (as setuptools contains an internal copy of distutils), see rhauser@7277980

The Python installation still fails, though, with the following warnings and error:

% make install prefix=/scratch/rhauser/ipbus/install/
OS Detected: el9
Using BUILD_HOME=/scratch/rhauser/ipbus/ipbus-software
mkdir -p /scratch/rhauser/ipbus/ipbus-software/uhal/python/rpm/RPMBUILD
cp /scratch/rhauser/ipbus/ipbus-software/uhal/config//setupTemplate.py /scratch/rhauser/ipbus/ipbus-software/uhal/python/rpm/RPMBUILD/cactuscore-uhal-python312.py
sed -i -e 's#__python_packages__#["uhal"]#' \
       -e 's#__packagename__#cactuscore-uhal-python312#' \
       -e 's#__version__#2.8.15#' \
       -e 's#__author__#Tom Williams#' \
       -e 's#__description__#Python bindings for the uHAL library#' \
       -e 's#__url__#https://ipbus.web.cern.ch/ipbus#' \
       -e 's#__project__#uhal#' \
       -e 's#__install_dir__#/opt/cactus#' \
       -e 's#__package_build_dir__#/scratch/rhauser/ipbus/ipbus-software/uhal/python/rpm/RPMBUILD#' \
       /scratch/rhauser/ipbus/ipbus-software/uhal/python/rpm/RPMBUILD/cactuscore-uhal-python312.py
# Change directory into pkg and copy everything into rpm/pkg
cd pkg && \
  find . -name "*" -exec install -D \{\} /scratch/rhauser/ipbus/ipbus-software/uhal/python/rpm/RPMBUILD/\{\} \;
install: omitting directory '.'
install: omitting directory './uhal'
Regex version mismatch, expected: 10.42 2022-12-11 actual: 10.40 2022-04-14
Regex version mismatch, expected: 10.42 2022-12-11 actual: 10.40 2022-04-14
Regex version mismatch, expected: 10.42 2022-12-11 actual: 10.40 2022-04-14
Regex version mismatch, expected: 10.42 2022-12-11 actual: 10.40 2022-04-14
Regex version mismatch, expected: 10.42 2022-12-11 actual: 10.40 2022-04-14
Regex version mismatch, expected: 10.42 2022-12-11 actual: 10.40 2022-04-14
# Add a manifest file
echo "include */*.so" > /scratch/rhauser/ipbus/ipbus-software/uhal/python/rpm/RPMBUILD/MANIFEST.in
# Change into rpm/pkg to finally run the customized setup.py
if [ -f setup.cfg ]; then cp setup.cfg /scratch/rhauser/ipbus/ipbus-software/uhal/python/rpm/RPMBUILD/ ; fi
cd /scratch/rhauser/ipbus/ipbus-software/uhal/python/rpm/RPMBUILD && \
  bindir=/scratch/rhauser/ipbus/install//bin python cactuscore-uhal-python312.py install --prefix=/scratch/rhauser/ipbus/install/ --exec-prefix=/scratch/rhauser/ipbus/install/
/cvmfs/sft-nightlies.cern.ch/lcg/views/dev3python312/Wed/x86_64-el9-gcc13-opt/lib/python3.12/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()
/cvmfs/sft-nightlies.cern.ch/lcg/views/dev3python312/Wed/x86_64-el9-gcc13-opt/lib/python3.12/site-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` and ``easy_install``.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://github.com/pypa/setuptools/issues/917 for details.
        ********************************************************************************

!!
  self.initialize_options()
Traceback (most recent call last):
  File "/tmp/tmpq5713hro.py", line 1, in <module>
    from distutils.util import byte_compile
ModuleNotFoundError: No module named 'distutils'
error: command '/cvmfs/sft-nightlies.cern.ch/lcg/views/dev3python312/Wed/x86_64-el9-gcc13-opt/bin/python' failed with exit code 1
make: *** [/scratch/rhauser/ipbus/ipbus-software/uhal/config/mfPythonRPMRules.mk:97: install] Error 1
pc-tbed-pub-06:/scratch/rhauser/ipbus/ipbus-software/uhal/python % 

I'm testing this with a nightly build of LCG like this on an Alma 9 host:

source /cvmfs/sft-nightlies.cern.ch/lcg/views/dev3python312/Wed/x86_64-el9-gcc13-opt/setup.sh
make Set=uhal BUILD_PUGIXML=1 EXTERN_BOOST_INCLUDE_PREFIX=/cvmfs/sft-nightlies.cern.ch/lcg/nightlies/dev3python312/Wed/Boost/1.86.0/x86_64-el9-gcc13-opt/include/ EXTERN_BOOST_LIB_PREFIX=/cvmfs/sft-nightlies.cern.ch/lcg/nightlies/dev3python312/Wed/Boost/1.86.0/x86_64-el9-gcc13-opt/lib prefix=/scratch/rhauser/ipbus/install