Print

---

So, just for confirmation, looking at the recipe https://github.com/xrootd/xrootd/blob/v5.4.2/.github/workflows/build.yml#L100 should the python bindings will be automatically be found in CMAKE_INSTALL_PREFIX/lib/python${PYTHON_VERSION}/site-packages ?

Correct. As an explicit example with the image built from this Dockerfile located at docker/centos7/Dockerfile in the XRootD repo which installs the Python bindings during the CMake build with CMAKE_INSTALL_PREFIX=/usr/local/

ARG BASE_IMAGE=gitlab-registry.cern.ch/linuxsupport/cc7-base:latest
FROM ${BASE_IMAGE} as base

SHELL [ "/bin/bash", "-c" ]

RUN yum update -y && \
    yum install -y \
        cmake3 \
        make \
        krb5-devel \
        libuuid-devel \
        libxml2-devel \
        openssl-devel \
        systemd-devel \
        zlib-devel \
        devtoolset-7-gcc-c++ \
        python3-devel \
        python3-setuptools \
        git \
        cppunit-devel && \
    yum clean all && \
    python3 -m pip install --no-cache-dir --upgrade pip setuptools wheel

COPY . /code/xrootd

WORKDIR /code
RUN source scl_source enable devtoolset-7 && \
    cmake3 \
        -DCMAKE_INSTALL_PREFIX=/usr/local/ \
        -DPYTHON_EXECUTABLE=$(command -v python3) \
        -DENABLE_TESTS=ON \
        -DPIP_VERBOSE=ON \
        -S xrootd \
        -B build && \
    cmake3 build -LH && \
    cmake3 \
        --build build \
        --clean-first \
        --parallel $(($(nproc) - 1)) && \
    cmake3 --build build --target install && \
    cd / && \
    python3 -m pip list

WORKDIR /

SHELL [ "/usr/bin/scl", "enable", "devtoolset-7", "/bin/bash", "-c"]

if you build it with

docker build . --file docker/centos7/Dockerfile --tag xrootd/xrootd:centos7

then you can see that the bindings got installed under the /usr/local/ site-packages as expected:

$ docker run --rm xrootd/xrootd:centos7 /bin/bash -c 'python3 -m pip show xrootd'
Name: xrootd
Version: unknown
Summary: XRootD Python bindings
Home-page: http://xrootd.org
Author: XRootD Developers
Author-email: [log in to unmask]
License: LGPLv3+
Location: /usr/local/lib64/python3.6/site-packages
Requires: 
Required-by: 

now, on the matter of normal cmake based xrootd installation https://github.com/alisw/alidist/blob/dd7ddacd743649bf9ca06af474275d2551a5a80c/xrootd.sh#L49 it seems that there is a mismatch of sorts between what python should see .. in the log here: https://asevcenc.web.cern.ch/asevcenc/alibuild_xrootd542/log i get:

-- Processing: //home/physics-tools/alicebuild/sw/fedora35_x86-64/XRootD/xrootd-local2/share/man/man1/xrdmapc.1
Processing ./bindings/python
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Building wheels for collected packages: xrootd
  Building wheel for xrootd (setup.py): started
  Building wheel for xrootd (setup.py): finished with status 'done'
  Created wheel for xrootd: filename=xrootd-2022.310+3fae8a870-cp310-cp310-linux_x86_64.whl size=1492763 sha256=e4185db69b3c5f580864925e1236773cc63376870789c70d0b19bc29782919e3
  Stored in directory: /home/physics-tools/alicebuild/xrootd/tmp/pip-ephem-wheel-cache-raoegz6i/wheels/ce/0c/a2/89fae19e44d0efe6e789006aab5941f99901c4b09d7843f92c
Successfully built xrootd
Installing collected packages: xrootd
  Attempting uninstall: xrootd
    Found existing installation: xrootd 5.4.1
ERROR: Cannot uninstall 'xrootd'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
++ popd
/home/physics-tools/alicebuild/sw/BUILD/1a865e68f28dfe14782f80089dbddcdf3ec15076/XRootD

and this should not happen, as ideally nor the system installed python(and packages) nor user installed packages should be seen... this happens in an environment that have it's own python. until this is fixed in xrootd...

I don't think there's anything to fix here. The pip error message explains things correctly. It seems that the v5.4.1 version was installed using distuils through a direct invocation of python setup.py install and not through pip plus setuptools.

c.f. pypa/pip#5247 (comment) for a concise technical summary of why errors like this happen from the PyPA.

This is strange though, as this sort of thing is what motivated me to open up Issue #1579 in the first place and make the changes that went into v5.4.1.

I'm a bit confused too why v5.4.1 is showing up in your build logs, as you updated to v5.4.0 in alisw/alidist#3696 and are now trying to update to v5.4.2 in alisw/alidist#3933. This makes sense to me why you're seeing the distuils error given that v5.4.0 was installed with the deprecated python setup.py install and distutils pattern, and now v5.4.2 is using the pip + setuptools approach and so rightly complaining that it is lacking enough information to know how to remove the v5.4.0 distutils based install. But you're showing v5.4.1 and not v5.4.0 in the logs. :? Can you explain what is happening here?

...is there anything that we can do to intercept and fix the problem?

The short answer is just manually delete the old distuitls directory (the PyPA approach in pypa/pip#5247 (comment)), but ...

Maybe something like build without python, and then build/install the python bindings based on what we just compiled, but with our python infrastructure and locations?

...yes this will also indeed work. c.f. #1594 (comment) for an example of building with CMake and then installing the Python bindings from PyPI.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.Message ID: <xrootd/xrootd/issues/1641/1065619152@github.com>

Use REPLY-ALL to reply to list

To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1