> So, just for confirmation, looking at the recipe https://github.com/xrootd/xrootd/blob/v5.4.2/.github/workflows/build.yml#L100 should the python bindings will be automatically be found in `CMAKE_INSTALL_PREFIX/lib/python${PYTHON_VERSION}/site-packages` ?
Correct. As an explicit example with the image built from this `Dockerfile` located at `docker/centos7/Dockerfile` in the `XRootD` repo which installs the Python bindings during the CMake build with `CMAKE_INSTALL_PREFIX=/usr/local/`
<details>
<summary>Dockerfile:</summary>
```Dockerfile
ARG BASE_IMAGE=gitlab-registry.cern.ch/linuxsupport/cc7-base:latest
FROM ${BASE_IMAGE} as base
SHELL [ "/bin/bash", "-c" ]
RUN yum update -y && \
yum install -y \
cmake3 \
make \
krb5-devel \
libuuid-devel \
libxml2-devel \
openssl-devel \
systemd-devel \
zlib-devel \
devtoolset-7-gcc-c++ \
python3-devel \
python3-setuptools \
git \
cppunit-devel && \
yum clean all && \
python3 -m pip install --no-cache-dir --upgrade pip setuptools wheel
COPY . /code/xrootd
WORKDIR /code
RUN source scl_source enable devtoolset-7 && \
cmake3 \
-DCMAKE_INSTALL_PREFIX=/usr/local/ \
-DPYTHON_EXECUTABLE=$(command -v python3) \
-DENABLE_TESTS=ON \
-DPIP_VERBOSE=ON \
-S xrootd \
-B build && \
cmake3 build -LH && \
cmake3 \
--build build \
--clean-first \
--parallel $(($(nproc) - 1)) && \
cmake3 --build build --target install && \
cd / && \
python3 -m pip list
WORKDIR /
SHELL [ "/usr/bin/scl", "enable", "devtoolset-7", "/bin/bash", "-c"]
```
</details>
if you build it with
```
docker build . --file docker/centos7/Dockerfile --tag xrootd/xrootd:centos7
```
then you can see that the bindings got installed under the `/usr/local/` `site-packages` as expected:
```
$ docker run --rm xrootd/xrootd:centos7 /bin/bash -c 'python3 -m pip show xrootd'
Name: xrootd
Version: unknown
Summary: XRootD Python bindings
Home-page: http://xrootd.org
Author: XRootD Developers
Author-email: [log in to unmask]
License: LGPLv3+
Location: /usr/local/lib64/python3.6/site-packages
Requires:
Required-by:
```
> now, on the matter of normal cmake based xrootd installation https://github.com/alisw/alidist/blob/dd7ddacd743649bf9ca06af474275d2551a5a80c/xrootd.sh#L49 it seems that there is a mismatch of sorts between what python should see .. in the log here: https://asevcenc.web.cern.ch/asevcenc/alibuild_xrootd542/log i get:
>
> ```
> -- Processing: //home/physics-tools/alicebuild/sw/fedora35_x86-64/XRootD/xrootd-local2/share/man/man1/xrdmapc.1
> Processing ./bindings/python
> Preparing metadata (setup.py): started
> Preparing metadata (setup.py): finished with status 'done'
> Building wheels for collected packages: xrootd
> Building wheel for xrootd (setup.py): started
> Building wheel for xrootd (setup.py): finished with status 'done'
> Created wheel for xrootd: filename=xrootd-2022.310+3fae8a870-cp310-cp310-linux_x86_64.whl size=1492763 sha256=e4185db69b3c5f580864925e1236773cc63376870789c70d0b19bc29782919e3
> Stored in directory: /home/physics-tools/alicebuild/xrootd/tmp/pip-ephem-wheel-cache-raoegz6i/wheels/ce/0c/a2/89fae19e44d0efe6e789006aab5941f99901c4b09d7843f92c
> Successfully built xrootd
> Installing collected packages: xrootd
> Attempting uninstall: xrootd
> Found existing installation: xrootd 5.4.1
> ERROR: Cannot uninstall 'xrootd'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
> ++ popd
> /home/physics-tools/alicebuild/sw/BUILD/1a865e68f28dfe14782f80089dbddcdf3ec15076/XRootD
> ```
>
> and this should not happen, as ideally nor the system installed python(and packages) nor user installed packages should be seen... this happens in an environment that have it's own python. until this is fixed in xrootd...
I don't think there's anything to fix here. The `pip` error message explains things correctly. It seems that the `v5.4.1` version was installed using `distuils` through a direct invocation of `python setup.py install` and not through `pip` plus `setuptools`.
c.f. https://github.com/pypa/pip/issues/5247#issuecomment-381550610 for a concise technical summary of why errors like this happen from the PyPA.
This is strange though, as this sort of thing is what motivated me to open up Issue #1579 in the first place and make the changes that _went into_ `v5.4.1`.
I'm a bit confused too why `v5.4.1` is showing up in your build logs, as you updated to `v5.4.0` in https://github.com/alisw/alidist/pull/3696 and are now trying to update to `v5.4.2` in https://github.com/alisw/alidist/pull/3933. This makes sense to me why you're seeing the `distuils` error given that `v5.4.0` was installed with the deprecated `python setup.py install` and `distutils` pattern, and now `v5.4.2` is using the `pip` + `setuptools` approach and so rightly complaining that it is lacking enough information to know how to remove the `v5.4.0` `distutils` based install. But you're showing `v5.4.1` and _not_ `v5.4.0` in the logs. :? Can you explain what is happening here?
> ...is there anything that we can do to intercept and fix the problem?
The short answer is just manually delete the old `distuitls` directory (the PyPA approach in https://github.com/pypa/pip/issues/5247#issuecomment-381550610), but ...
> Maybe something like build without python, and then build/install the python bindings based on what we just compiled, but with our python infrastructure and locations?
...yes this will also indeed work. c.f. https://github.com/xrootd/xrootd/issues/1594#issuecomment-1063761052 for an example of building with CMake and then installing the Python bindings from PyPI.
--
Reply to this email directly or view it on GitHub:
https://github.com/xrootd/xrootd/issues/1641#issuecomment-1065619152
You are receiving this because you are subscribed to this thread.
Message ID: <[log in to unmask]>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
