So, just for confirmation, looking at the recipe https://github.com/xrootd/xrootd/blob/v5.4.2/.github/workflows/build.yml#L100 should the python bindings will be automatically be found in
CMAKE_INSTALL_PREFIX/lib/python${PYTHON_VERSION}/site-packages
?
Correct. As an explicit example with the image built from this Dockerfile
located at docker/centos7/Dockerfile
in the XRootD
repo which installs the Python bindings during the CMake build with CMAKE_INSTALL_PREFIX=/usr/local/
ARG BASE_IMAGE=gitlab-registry.cern.ch/linuxsupport/cc7-base:latest
FROM ${BASE_IMAGE} as base
SHELL [ "/bin/bash", "-c" ]
RUN yum update -y && \
yum install -y \
cmake3 \
make \
krb5-devel \
libuuid-devel \
libxml2-devel \
openssl-devel \
systemd-devel \
zlib-devel \
devtoolset-7-gcc-c++ \
python3-devel \
python3-setuptools \
git \
cppunit-devel && \
yum clean all && \
python3 -m pip install --no-cache-dir --upgrade pip setuptools wheel
COPY . /code/xrootd
WORKDIR /code
RUN source scl_source enable devtoolset-7 && \
cmake3 \
-DCMAKE_INSTALL_PREFIX=/usr/local/ \
-DPYTHON_EXECUTABLE=$(command -v python3) \
-DENABLE_TESTS=ON \
-DPIP_VERBOSE=ON \
-S xrootd \
-B build && \
cmake3 build -LH && \
cmake3 \
--build build \
--clean-first \
--parallel $(($(nproc) - 1)) && \
cmake3 --build build --target install && \
cd / && \
python3 -m pip list
WORKDIR /
SHELL [ "/usr/bin/scl", "enable", "devtoolset-7", "/bin/bash", "-c"]
if you build it with
docker build . --file docker/centos7/Dockerfile --tag xrootd/xrootd:centos7
then you can see that the bindings got installed under the /usr/local/
site-packages
as expected:
$ docker run --rm xrootd/xrootd:centos7 /bin/bash -c 'python3 -m pip show xrootd'
Name: xrootd
Version: unknown
Summary: XRootD Python bindings
Home-page: http://xrootd.org
Author: XRootD Developers
Author-email: [log in to unmask]
License: LGPLv3+
Location: /usr/local/lib64/python3.6/site-packages
Requires:
Required-by:
now, on the matter of normal cmake based xrootd installation https://github.com/alisw/alidist/blob/dd7ddacd743649bf9ca06af474275d2551a5a80c/xrootd.sh#L49 it seems that there is a mismatch of sorts between what python should see .. in the log here: https://asevcenc.web.cern.ch/asevcenc/alibuild_xrootd542/log i get:
-- Processing: //home/physics-tools/alicebuild/sw/fedora35_x86-64/XRootD/xrootd-local2/share/man/man1/xrdmapc.1 Processing ./bindings/python Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Building wheels for collected packages: xrootd Building wheel for xrootd (setup.py): started Building wheel for xrootd (setup.py): finished with status 'done' Created wheel for xrootd: filename=xrootd-2022.310+3fae8a870-cp310-cp310-linux_x86_64.whl size=1492763 sha256=e4185db69b3c5f580864925e1236773cc63376870789c70d0b19bc29782919e3 Stored in directory: /home/physics-tools/alicebuild/xrootd/tmp/pip-ephem-wheel-cache-raoegz6i/wheels/ce/0c/a2/89fae19e44d0efe6e789006aab5941f99901c4b09d7843f92c Successfully built xrootd Installing collected packages: xrootd Attempting uninstall: xrootd Found existing installation: xrootd 5.4.1 ERROR: Cannot uninstall 'xrootd'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall. ++ popd /home/physics-tools/alicebuild/sw/BUILD/1a865e68f28dfe14782f80089dbddcdf3ec15076/XRootD
and this should not happen, as ideally nor the system installed python(and packages) nor user installed packages should be seen... this happens in an environment that have it's own python. until this is fixed in xrootd...
I don't think there's anything to fix here. The pip
error message explains things correctly. It seems that the v5.4.1
version was installed using distuils
through a direct invocation of python setup.py install
and not through pip
plus setuptools
.
c.f. pypa/pip#5247 (comment) for a concise technical summary of why errors like this happen from the PyPA.
This is strange though, as this sort of thing is what motivated me to open up Issue #1579 in the first place and make the changes that went into v5.4.1
.
I'm a bit confused too why v5.4.1
is showing up in your build logs, as you updated to v5.4.0
in alisw/alidist#3696 and are now trying to update to v5.4.2
in alisw/alidist#3933. This makes sense to me why you're seeing the distuils
error given that v5.4.0
was installed with the deprecated python setup.py install
and distutils
pattern, and now v5.4.2
is using the pip
+ setuptools
approach and so rightly complaining that it is lacking enough information to know how to remove the v5.4.0
distutils
based install. But you're showing v5.4.1
and not v5.4.0
in the logs. :? Can you explain what is happening here?
...is there anything that we can do to intercept and fix the problem?
The short answer is just manually delete the old distuitls
directory (the PyPA approach in pypa/pip#5247 (comment)), but ...
Maybe something like build without python, and then build/install the python bindings based on what we just compiled, but with our python infrastructure and locations?
...yes this will also indeed work. c.f. #1594 (comment) for an example of building with CMake and then installing the Python bindings from PyPI.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1