I came to this conclusion because it works for me

Please see #1474 (comment) and #1474 (comment) where this has been discussed.

Either I'm doing something wrong to reproduce, or this issue has been fixed. If it still does not work for you, please post the full history of commands you ran after starting the Ubuntu container and the full output of pip install --verbose xrootd.

It is not fixed. Please see #1474 (comment) where this has already been provided. Please check the logs there along with the relevant line of code

as there seem to be path differences with pyenv such that

https://github.com/xrootd/xrootd/blob/fa83acfdb5d7101bf8ac50b61a1018a366fa2d7a/packaging/wheel/install.sh#L40

is failing to place us in the proper location for the rest of the build.

My recommendation is to try to set CMAKE_PREFIX_PATH pointing to the non-standard location where your Python is installed.

Sorry, can you be more explicit about how you would want a user to do this when installing from PyPI? Currently packaging/wheel/setup.py doesn't contain any os.environ calls. I don't think that framing pyenv as "non-standard" is correct either — XRootD is making assumptions about directory structures that aren't necessarily valid given #1474 (comment).

If that doesn't solve the problem, then please post the complete output of pip install --verbose xrootd so that I can understand what is happening at configuration time

Sure. Here's a reproducible containerized workflow:

# Have to run as root to have install permissions with apt
$ docker run --rm -ti -u root matthewfeickert/pyenv-virtualenv-conda:latest
root@4a47d67a252c:~/data# apt update && apt install -y python3-dev cmake  # Need these as forced to build from sdist as no wheels on PyPI
root@4a47d67a252c:~/data# pyenv virtualenv 3.10.4 issue-1474 && pyenv activate issue-1474
(issue-1474) root@4a47d67a252c:~/data# pyenv which python3
/home/docker/.pyenv/versions/issue-1474/bin/python3
(issue-1474) root@4a47d67a252c:~/data# python -c "import sysconfig; print(sysconfig.get_path('include'))"
/home/docker/.pyenv/versions/3.10.4/include/python3.10
(issue-1474) root@4a47d67a252c:~/data# find $(python -c "import sysconfig; print(sysconfig.get_path('include'))") -type f -iname "Python.h"
/home/docker/.pyenv/versions/3.10.4/include/python3.10/Python.h
(issue-1474) root@4a47d67a252c:~/data# python -c "import sysconfig; print(sysconfig.get_config_var('LIBDIR'))"
/home/docker/.pyenv/versions/3.10.4/lib
(issue-1474) root@4a47d67a252c:~/data# ls -l $(python -c "import sysconfig; print(sysconfig.get_config_var('LIBDIR'))")
total 43464
-rwxr-xr-x  1 docker docker 44497800 May 12  2022 libpython3.10.a
drwxr-xr-x  2 docker docker     4096 May 12  2022 pkgconfig
drwxr-xr-x 36 docker docker     4096 May 12  2022 python3.10
(issue-1474) root@4a47d67a252c:~/data# python -m pip --quiet install --upgrade pip setuptools wheel
(issue-1474) root@4a47d67a252c:~/data# python -m pip list
Package    Version
---------- -------
pip        23.0
setuptools 67.2.0
wheel      0.38.4
(issue-1474) root@4a47d67a252c:~/data# python -m pip --verbose install 'xrootd==5.5.2'

Attached is the output of that as last command as pip-install-fail.log.txt

with the following being the same error snippet as from #1474 (comment)

Final error snippet:
...
 ./install.sh: line 40: cd: ../bindings/python: No such file or directory
  WARNING: The directory '/home/docker/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
  ERROR: Directory '.' is not installable. Neither 'setup.py' nor 'pyproject.toml' found.
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/tmp/pip-install-6qvvduhj/xrootd_2f5bca47c43d46b9bed7a1e7d7727459/setup.py", line 205, in <module>
      setup(
    File "/home/docker/.pyenv/versions/issue-1474/lib/python3.10/site-packages/setuptools/__init__.py", line 108, in setup
      return distutils.core.setup(**attrs)
    File "/home/docker/.pyenv/versions/issue-1474/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
    File "/home/docker/.pyenv/versions/issue-1474/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/home/docker/.pyenv/versions/issue-1474/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "/home/docker/.pyenv/versions/issue-1474/lib/python3.10/site-packages/setuptools/dist.py", line 1213, in run_command
      super().run_command(command)
    File "/home/docker/.pyenv/versions/issue-1474/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/tmp/pip-install-6qvvduhj/xrootd_2f5bca47c43d46b9bed7a1e7d7727459/setup.py", line 182, in run
      raise Exception( 'Install step failed!' )
  Exception: Install step failed!
  error: subprocess-exited-with-error
  
  × Running setup.py install for xrootd did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /home/docker/.pyenv/versions/issue-1474/bin/python -u -c '
  exec(compile('"'"''"'"''"'"'
  # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
  #
  # - It imports setuptools before invoking setup.py, to enable projects that directly
  #   import from `distutils.core` to work with newer packaging standards.
  # - It provides a clear error message when setuptools is not installed.
  # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
  #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
  #     manifest_maker: standard file '"'"'-c'"'"' not found".
  # - It generates a shim setup.py, for handling setup.cfg-only projects.
  import os, sys, tokenize
  
  try:
      import setuptools
  except ImportError as error:
      print(
          "ERROR: Can not execute `setup.py` since setuptools is not available in "
          "the build environment.",
          file=sys.stderr,
      )
      sys.exit(1)
  
  __file__ = %r
  sys.argv[0] = __file__
  
  if os.path.exists(__file__):
      filename = __file__
      with tokenize.open(__file__) as f:
          setup_py_code = f.read()
  else:
      filename = "<auto-generated setuptools caller>"
      setup_py_code = "from setuptools import setup; setup()"
  
  exec(compile(setup_py_code, filename, "exec"))
  '"'"''"'"''"'"' % ('"'"'/tmp/pip-install-6qvvduhj/xrootd_2f5bca47c43d46b9bed7a1e7d7727459/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' install --record /tmp/pip-record-hq84ulx4/install-record.txt --single-version-externally-managed --compile --install-headers /home/docker/.pyenv/versions/issue-1474/include/site/python3.10/xrootd
  cwd: /tmp/pip-install-6qvvduhj/xrootd_2f5bca47c43d46b9bed7a1e7d7727459/
  Running setup.py install for xrootd ... error
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> xrootd

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
(issue-1474) root@4a47d67a252c:~/data# 

This is in contrast to

...
(issue-1474) root@4a47d67a252c:~/data# pyenv deactivate 
root@4a47d67a252c:~/data# pyenv virtualenv-delete -y issue-1474
root@4a47d67a252c:~/data# pyenv global 3.10.4
root@4a47d67a252c:~/data# pyenv which python
/home/docker/.pyenv/versions/3.10.4/bin/python
root@4a47d67a252c:~/data# python -c "import sysconfig; print(sysconfig.get_path('include'))"
/home/docker/.pyenv/versions/3.10.4/include/python3.10
root@4a47d67a252c:~/data# find $(python -c "import sysconfig; print(sysconfig.get_path('include'))") -type f -iname "Python.h"
/home/docker/.pyenv/versions/3.10.4/include/python3.10/Python.h
root@4a47d67a252c:~/data# python -c "import sysconfig; print(sysconfig.get_config_var('LIBDIR'))"
/home/docker/.pyenv/versions/3.10.4/lib
root@4a47d67a252c:~/data# ls -l $(python -c "import sysconfig; print(sysconfig.get_config_var('LIBDIR'))")
total 43464
-rwxr-xr-x  1 docker docker 44497800 May 12  2022 libpython3.10.a
drwxr-xr-x  2 docker docker     4096 May 12  2022 pkgconfig
drwxr-xr-x 36 docker docker     4096 May 12  2022 python3.10
root@4a47d67a252c:~/data# pyenv which python3
/home/docker/.pyenv/versions/3.10.4/bin/python3
root@4a47d67a252c:~/data# python -m pip --quiet install --upgrade pip setuptools wheel
root@4a47d67a252c:~/data# python -m pip list
Package    Version
---------- -------
pip        23.0
setuptools 67.2.0
wheel      0.38.4
root@4a47d67a252c:~/data# python -m pip --verbose install 'xrootd==5.5.2'
...
Successfully installed xrootd-5.5.2
root@4a47d67a252c:~/data# python -m pip list
Package    Version
---------- -------
pip        23.0
setuptools 67.2.0
wheel      0.38.4
xrootd     5.5.2
root@4a47d67a252c:~/data#

as I showed in #1474 (comment).


Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.Message ID: <xrootd/xrootd/issues/1474/1428962163@github.com>

[ { "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": "https://github.com/xrootd/xrootd/issues/1474#issuecomment-1428962163", "url": "https://github.com/xrootd/xrootd/issues/1474#issuecomment-1428962163", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

Use REPLY-ALL to reply to list

To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1