 |
 |
Hello K.T., First of all, i wish to all the team a Merry Christmas and a Happy new year. Even if its documentation could be improved, eups is a good tool, it's code is clear and i think it will made qserv build process more modular, cross-platform, and maintainable. Futhermore, it seems eups will easilly allow to upgrade Qserv dependencies durign Qserv development process. Thanks for your help on this topic, it is very usefull. Thanks also to other eups pundits (i.e. Mario, Robert and Paul) for their support. It was a necessary condition to do the job. Nevertheless, i still need some help on some small details, in order to converge towards the right direction. *1. *At the moment, the build procedure relies on HSC build style (like it is what was recommended by Paul Price one month ago, cf. http://hsca.ipmu.jp:8080/question/128/more-information-about-eups-create-distrib/) It builds and installs nearly all Qserv source dependencies (but not scisql and Qserv itself) on both Ubuntu 13.04 and SL6.2. For information, i did not use nor lssteups or sconsutils (i only have the qserv package to build with scons, and Qserv only links with protobuf, xrootd and mysql). - Do you confirm i have to move all these build script to pkgbuild right now ? Is pkgbuild stable enough ? If yes, does https://github.com/RobertLuptonTheGood/eups.git master branch support pkgbuild, or shall i use an other branch ? - Do you think Qserv build should relies on sconsutils ? - Do you thinks sconsutils may be compatible with rpm format ? - Do you confirm Qserv build musn't rely on lssteups (I didn't notice the use of this tool while i investigated HSC build scripts.) ? For the next three points, i answer you between the lines below : On 12/20/2013 07:55 PM, Kian-Tat Lim wrote: > Fabrice, > >> Maybe it would be better if eups always run with system python ? > In the not-too-distant future (maybe even in January or > February), I think we want to convert the LSST Stack to run with the > system Python in all cases, including eups. > >> Indeed for now it switch to eups-installed python once command >> "setup python" is performed. > "setup python" would go away. But I'm not sure what the problem > is with switching Pythons. *2. *The problem is met with next hypothesis : - eups is compatible with python version i, but not with python version j, and eups is used to install a software, named SOFT, which require python version j. (note that j can even be lower than i) During SOFT install, eups will install python version j, and in current eups version, it will itself switch to python version j, and then the whole install procedure may break. Indeed all call to eups may fails because of the use of python version j, so it may even be not possible to go back (unsetup command may fails). That's why i propose that, during eups installation, all call to eups executables are forced to be done with python version i. Note that all other calls to python done by the build scripts will use environment python, i.e. python version j. (For exemple, to use this solution, eups/bin/eups would be simply updated like during by eups install process : --- a/bin/eups +++ b/bin/eups @@ -3,4 +3,4 @@ # #!/usr/bin/env python options # unset PYTHONSTARTUP -*python* -S $EUPS_DIR/bin/eups_impl.py "$@" +*/usr/bin/python* -S $EUPS_DIR/bin/eups_impl.py "$@" ) This solution, trivial to implement, would make eups more stable and more generic, don't you think so ? (I expose briefly this issue to Christian Arnault, french manager for LSST computing activities and author of CMT, the atlas build system, and he agreed with me.) Note that i met this issue while switching from system python to virtualenv python during Qserv build process (cf. http://hsca.ipmu.jp:8080/question/167/installing-virtualenv-with-eups/). > >>> Regarding its use with virtualenv, EUPS has been designed with >>> virtual-env like functionality in mind -- specifically, it manages the >>> PYTHON_PATH for you. So using the two together is likely to generate >>> conflicts (though it may not be impossible). >> One of the main advantage of using virtualenv is that install of >> python libraries via eups is completely transparent : you don't have >> to manage a dedicated PYTHONPATH in eups build scripts of python >> libraries. >> It also isolate eups-installed python and then prevent conflicts >> between system python user libraries and eups-installed python user >> libraries. >> Of course, if you prefer, i can set up build script to install each >> Qserv python lib in a dedicated directory and then add it to >> PYTHONPATH. > We already have the PYTHONPATH manipulation in the rest of the > LSST Stack, so I think it's best for now to continue with that. In the > longer run, I would like to see all LSST Stack Python modules installed > into a single directory, almost certainly an LSST-specific one and not > the system site-packages -- but they would be swapped in and out on > setup of the individual packages through the use of symbolic links. *3. *This long-term solution looks fine. So i confirm i will implement PYTHONPATH manipulation in my next version of Qserv packaging tool. > > One of the main reasons to use the system Python is to allow the > user to incorporate other packages into their usage of the LSST Stack. > virtualenv does not work well with this usage model. This may not be an > issue with qserv, however, which is server-side and can insist on a more > rigorously-controlled environment. > * 4. *Qserv now relies on virtualenv but it also may works with system-python. An interesting feature of virtualenv in a rigorously-controlled environment is that it may switch PYTHONPATH while its activated/deactivated. Please, note that in order to make eups fully complient with virtualenv, command "setup python" would require an extra feature : Indeed activating a virtualenv-installed python require /path_to_virtualenv_installed_python/bin/activate script to be sourced, and deactivating it require /path_to_virtualenv_installed_python/bin/deactivate script to be runnned. So i think that complete virtualenv support in eups would require and execute() function in table file. Do you think an alternate solution may be used ? Have a nice day. Fabrice ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the QSERV-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1