Hello K.T.,

First of all, i wish to all the team a Merry Christmas and a Happy new year.

Even if its documentation could be improved, eups is a good tool, it's code is clear and i think it will made qserv build process more modular, cross-platform, and maintainable.
Futhermore, it seems eups will easilly allow to upgrade Qserv dependencies durign Qserv development process.
Thanks for your help on this topic, it is very usefull. Thanks also to other eups pundits (i.e. Mario, Robert and Paul) for their support. It was a necessary condition to do the job.

Nevertheless, i still need some help on some small details, in order to converge towards the right direction.

1. At the moment, the build procedure relies on HSC build style (like it is what was recommended by Paul Price one month ago, cf. http://hsca.ipmu.jp:8080/question/128/more-information-about-eups-create-distrib/)
It builds and installs nearly all Qserv source dependencies (but not scisql and Qserv itself) on both Ubuntu 13.04 and SL6.2.
For information, i did not use nor lssteups or sconsutils (i only have the qserv package to build with scons, and Qserv only links with protobuf, xrootd and mysql).

- Do you confirm i have to move all these build script to pkgbuild right now ? Is pkgbuild stable enough ? If yes, does https://github.com/RobertLuptonTheGood/eups.git master branch support pkgbuild, or shall i use
an other branch ?
- Do you think Qserv build should relies on sconsutils ?
- Do you thinks sconsutils may be compatible with rpm format ?
- Do you confirm Qserv build musn't rely on lssteups (I didn't notice the use of this tool while i investigated HSC build scripts.) ?

For the next three points, i answer you between the lines below :

On 12/20/2013 07:55 PM, Kian-Tat Lim wrote:
[log in to unmask]" type="cite"> 
Fabrice,


Maybe it would be better if eups always run with system python ?

	In the not-too-distant future (maybe even in January or
February), I think we want to convert the LSST Stack to run with the
system Python in all cases, including eups.


Indeed for now it switch to eups-installed python once command
"setup python" is performed.

	"setup python" would go away.  But I'm not sure what the problem
is with switching Pythons.

2. The problem is met with next hypothesis :
- eups is compatible with python version i, but not with python version j, and eups is used to install a software, named SOFT, which require python version j. (note that j can even be lower than i)
During SOFT install, eups will install python version j, and in current eups version, it will itself switch to python version j, and then the whole install procedure may break. Indeed all call to eups may fails because of the use of python version j,
so it may even be not possible to go back (unsetup command may fails).

That's why i propose that, during eups installation, all call to eups executables are forced to be done with python version i. Note that all other calls to python done by the build scripts will use environment python, i.e. python version j.

(For exemple, to use this solution, eups/bin/eups would be simply updated like during by eups install process :
--- a/bin/eups
+++ b/bin/eups
@@ -3,4 +3,4 @@
 #  #!/usr/bin/env python options
 #
 unset PYTHONSTARTUP
-python -S $EUPS_DIR/bin/eups_impl.py "$@"
+/usr/bin/python -S $EUPS_DIR/bin/eups_impl.py "$@"
)

This solution, trivial to implement, would make eups more stable and more generic, don't you think so ?
(I expose briefly this issue to Christian Arnault, french manager for LSST computing activities and author of CMT, the atlas build system, and he agreed with me.)

Note that i met this issue while switching from system python to virtualenv python during Qserv build process (cf. http://hsca.ipmu.jp:8080/question/167/installing-virtualenv-with-eups/).
[log in to unmask]" type="cite"> 


	Regarding its use with virtualenv, EUPS has been designed with
virtual-env like functionality in mind -- specifically, it manages the
PYTHON_PATH for you. So using the two together is likely to generate
conflicts (though it may not be impossible).


      

        
One of the main advantage of using virtualenv is that install of
python libraries via eups is completely transparent : you don't have
to manage a dedicated PYTHONPATH in eups build scripts of python
libraries.
It also isolate eups-installed python and then prevent conflicts
between system python user libraries and eups-installed python user
libraries.
Of course, if you prefer, i can set up build script to install each
Qserv python lib in a dedicated directory and then add it to
PYTHONPATH.


      

      
	We already have the PYTHONPATH manipulation in the rest of the
LSST Stack, so I think it's best for now to continue with that.  In the
longer run, I would like to see all LSST Stack Python modules installed
into a single directory, almost certainly an LSST-specific one and not
the system site-packages -- but they would be swapped in and out on
setup of the individual packages through the use of symbolic links.

    

    3. This long-term solution looks fine. So i confirm i will
    implement PYTHONPATH manipulation in my next version of Qserv
    packaging tool.

    
[log in to unmask]"
      type="cite">
      

	One of the main reasons to use the system Python is to allow the
user to incorporate other packages into their usage of the LSST Stack.
virtualenv does not work well with this usage model.  This may not be an
issue with qserv, however, which is server-side and can insist on a more
rigorously-controlled environment.



    

    

      4. Qserv now relies on virtualenv but it also may works with
    system-python.

    

    An interesting feature of virtualenv in a rigorously-controlled
    environment is that it may switch PYTHONPATH while its
    activated/deactivated.

    

    Please, note that in order to make eups fully complient with
    virtualenv, command

    "setup python" would require an extra feature :

    Indeed activating a virtualenv-installed python require
    /path_to_virtualenv_installed_python/bin/activate script to be
    sourced,

    and deactivating it require
    /path_to_virtualenv_installed_python/bin/deactivate script to be
    runnned.

    So i think that complete virtualenv support in eups would require
    and execute() function in table file.

    Do you think an alternate solution may be used ?

    

    Have a nice day.

    

    Fabrice

    

  







Use REPLY-ALL to reply to list


To unsubscribe from the QSERV-L list, click the following link:

https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1