@adriansev @agoose77 Circling back to this very late, I think I have the start of an idea of the problem: The `RPATH` (CentOS) or `RUNPATH` (Debian) needs to be different if the installation is an egg or a wheel for things to work. This is because the `RPATH`/`RUNPATH` needs to take us from the library `.so` to the dependency library `.so`s.
## Wheel
If you have a wheel then from your `site-packages` you'll have a nice layout something like
```console
[root@e2b96418e2e5 site-packages]# tree -L 1 | grep -i "xrootd"
|-- XRootD
|-- pyxrootd
`-- xrootd-5.4.2-py3.6.egg-info
[root@e2b96418e2e5 site-packages]# tree -L 1 pyxrootd/
pyxrootd/
|-- __init__.py
|-- __pycache__
|-- client.cpython-36m-x86_64-linux-gnu.so
|-- include
|-- lib64
`-- share
4 directories, 2 files
```
and the important thing is that the `client` `.so` has a `RPATH`/`RUNPATH` that points to `$ORIGIN/lib64` (`$ORIGIN/lib` on Debian)
```console
[root@e2b96418e2e5 site-packages]# cd pyxrootd/
[root@e2b96418e2e5 pyxrootd]# objdump -x client.cpython-36m-x86_64-linux-gnu.so | grep PATH
RPATH $ORIGIN/lib64
[root@e2b96418e2e5 pyxrootd]# ls -l lib64/libXrdCl.so.3
lrwxrwxrwx 1 root root 17 Apr 5 23:09 lib64/libXrdCl.so.3 -> libXrdCl.so.3.0.0
[root@e2b96418e2e5 site-packages]# cd
[root@e2b96418e2e5 ~]# python3 -c 'from XRootD import client; print(client.FileSystem("root://someserver:1094"))'
<XRootD.client.filesystem.FileSystem object at 0x7fd541df7f98>
```
## egg
However, if you have an egg, then the layout changes and from your `site-packages` you'll have something like
```console
(venv) root@34321f613143:/code/venv/lib/python3.9/site-packages# tree -L 1 | grep -i "xrootd"
|-- pyxrootd
`-- xrootd-2022.415-py3.9-linux-x86_64.egg
(venv) root@34321f613143:/code/venv/lib/python3.9/site-packages# tree -L 2 xrootd-2022.415-py3.9-linux-x86_64.egg/
xrootd-2022.415-py3.9-linux-x86_64.egg/
|-- EGG-INFO
| |-- PKG-INFO
| |-- SOURCES.txt
| |-- dependency_links.txt
| |-- native_libs.txt
| |-- not-zip-safe
| `-- top_level.txt
|-- XRootD
| |-- __init__.py
| |-- __pycache__
| `-- client
`-- pyxrootd
|-- __init__.py
|-- __pycache__
|-- client.cpython-39-x86_64-linux-gnu.so
`-- client.py
6 directories, 10 files
```
and now the `client` `.so` has a `RPATH`/`RUNPATH` that points to `$ORIGIN/../../pyxrootd/lib64` (`$ORIGIN/../../pyxrootd/lib` on Debian)
```console
(venv) root@34321f613143:/code/venv/lib/python3.9/site-packages# cd xrootd-2022.415-py3.9-linux-x86_64.egg/pyxrootd/
(venv) root@34321f613143:/code/venv/lib/python3.9/site-packages/xrootd-2022.415-py3.9-linux-x86_64.egg/pyxrootd# objdump -x client.cpython-39-x86_64-linux-gnu.so | grep PATH
RUNPATH $ORIGIN/../../pyxrootd/lib
(venv) root@34321f613143:/code/venv/lib/python3.9/site-packages/xrootd-2022.415-py3.9-linux-x86_64.egg/pyxrootd# ls -l ../../pyxrootd/lib/libXrdCl.so.3
lrwxrwxrwx 1 root root 17 Apr 15 06:44 ../../pyxrootd/lib/libXrdCl.so.3 -> libXrdCl.so.3.0.0
(venv) root@34321f613143:/code/venv/lib/python3.9/site-packages/xrootd-2022.415-py3.9-linux-x86_64.egg/pyxrootd# cd
(venv) root@34321f613143:~# python3 -c 'from XRootD import client; print(client.FileSystem("root://someserver:1094"))'
<XRootD.client.filesystem.FileSystem object at 0x7f5105ee12b0>
```
This works without any `LD_LIBRARY_PATH` manipulation as `RPATH`/`RUNPATH` are actually being set correctly!
## Next question towards solution
As the `RPATH`/`RUNPATH` is dependent on the distribution layout then
https://github.com/xrootd/xrootd/blob/81e57b5d4bf4de3e9a126484afc1923b9b08d5db/bindings/python/CMakeLists.txt#L12-L16
needs to be more robust and be able to determine what the layout will be and set the `XRDCL_RPATH` accordingly if `PYPI_BUILD`:
* `"$ORIGIN/${CMAKE_INSTALL_LIBDIR}"` if wheel
* `"$ORIGIN/../../pyxrootd/${CMAKE_INSTALL_LIBDIR}"` if egg
## Question after all of this is fixed
Why are wheels not working with `build`?
## Summary
I don't have a working fix yet, but I think this is actually understandable now. :+1:
--
Reply to this email directly or view it on GitHub:
https://github.com/xrootd/xrootd/issues/1668#issuecomment-1099922724
You are receiving this because you are subscribed to this thread.
Message ID: <[log in to unmask]>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
