K-T,
I forgot to mention that the issue is only with python-side
of things. We already adopted lsst logging in Qserv C++ world.
> small Python-only
> scripts that are not expected to produce production logs
I suspect Qserv does not fully fall into that category,
although our python scripts are not huge.
So... if we
- use lsst logging in qserv-C++
- use python logging in qserv-python and point it to
lsst logging via config
- use lsst logging in python db module (which is called
from qserv, but only from qserv python admin scripts)
would that be reasonable?
Hmm, I think I heard sentiment within the group that admin
scripts would rather use standard logging, not lsst logging.
My feeling is that this is getting convoluted.
Jacek
On 03/03/2015 07:43 AM, Kian-Tat Lim wrote:
> Jacek,
>
>> We need to come back to the topic of logging. Andy (S) is telling
>> me that we recently decided to stick in Qserv with native python
>> log, and attach that log stream to the lsst logging. I do remember
>> we discussed this, but somehow it didn't fully occur to me that we
>> are not going to use our lsst logging in Qserv directly. Because...
>> if indeed that is the case, it does not make much sense to introduce
>> it in the "db" module (which I just did in DM-2137), which means
>> it does not make sense to use it in metaserv/imgserv/dbserv/webserv.
>>
>> So, I would like us to be completely clear about it before we
>> go that route. (I am not very comfortable with going that route).
>> And I want to make sure K-T is not going to shoot us when he
>> discovers our plans. :)
>
> C++ code needs to log via log4cxx. Having C++ log via the
> Python logger is, as far as I know, 1) difficult and 2) impossible if
> C++ code is not called from Python, which may still occur at times.
>
> Python code can log via log4cxx using lsst.log. It can also log
> via the Python library logger. The library logger can be configured to
> output to log4cxx, but I think that generally requires two configuration
> files, which I have generally resisted; I'm not sure if it also requires
> an actual code change to import lsst.log.
>
> The compromise I had approved before was that small Python-only
> scripts that are not expected to produce production logs could use the
> Python library logger only.
>
> But logging in packages that are shared with the Science
> Pipelines portion of the Stack (like db or sphgeom) needs to take more
> care to be compatible with C++ and its usage.
>
> The requirements I have in mind are:
>
> 1) Minimize the number of places that logging configuration is stored.
> Even if logging users tend to be more sophisticated developers, I think
> having many configuration locations makes our system hard to use.
>
> 2) Generate log outputs that are consistent in terms of format and
> content. Having non-ISO times (or no times at all), components in
> different orders, etc. makes interpretation hard.
>
> 3) Be able to configure logging at a fine grain (definitely package
> level; usually class level or even below).
>
> 4) Permit urgent log messages to be sent via DM Messages (currently
> known as ctrl_events). This will allow rapid response to problems as
> opposed to waiting for log files to be collected.
>
> 5) Have the primary log output from a single "application" go to a
> single place. I think it's a lot easier to debug if all the information
> is collected together and not scattered across multiple files.
>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
