Hello All -
I've pushed the latest iteration of the logging prototype, which can be found at:
https://dev.lsstcorp.org/cgit/LSST/DMS/qserv.git/tree/core/modules?h=u/bchick/protolog
The online documentation has also been updated and can be found at:
https://dev.lsstcorp.org/trac/wiki/db/Qserv/ProtoLog
Most of the requested features have been implemented. I look forward to receiving feedback.
-- Bill
Begin forwarded message:
> From: Kian-Tat Lim <[log in to unmask]>
> Subject: Re: [LSST-data] input about logging needed by mid next week
> Date: May 5, 2014 at 2:36:57 PM PDT
> To: Robert Lupton the Good <[log in to unmask]>
> Cc: LSST Data Management <[log in to unmask]>
>
> Robert and Jim and Paul,
>
> Robert wrote:
>> Is there support for python "with" statements (with logging context)
>
> Jim wrote:
>> My biggest concern at this stage is the way the context switching is
>> handled; having to manually pop the context seems like it's going against
>> the trend of how context management is being done in C++ and Python these
>> days. I think Robert's of supporting 'with' in Python is a good one, but I
>> think we should also continue to have log objects that know their own
>> context. RAII is the natural (and only safe, IMO) way to do resource
>> management in C++, and I think it's desirable (and convenient) to be able
>> to do the same thing in Python.
>
> It is easy to wrap the current pushContext and popContext calls within
> both C++ and Python objects (or replace them, although that may involve
> more SWIG). I have suggested this to Bill already.
>
> Robert wrote:
>> Having enough debug levels. I'd like to have at least 3 and maybe 5
>> different debug levels, and preferably control them by a numerical
>> parameter not a logger name (protolog.debug(3, ...) rather than
>> protolog.debug3(...)) to enable changing the level of groups of debug
>> commands. Yes, you can do this with mylog = {3: protolog.debug3);
>> mylog[3](...) but that's more work.
>
> Bill and I were thinking that having more loggers with their own names
> is similar to having more debug levels. Aside from syntactic sugar,
> which we can work out, I think you get near-equivalent functionality by
> turning on and off "mylog-level4" or "mylog-level5" instead of having
> "mylog" with level 4 or 5.
>
> Robert wrote:
>> output control. It is convenient to make different debug levels
>> format with different indents
>
> This may be a bit more difficult. log4j 2.0 shows how we could extend
> the log4cxx PatternLayout class to enable this, but it's something we'd
> have to add "under" (or "plug into") log4cxx.
>
> Robert wrote:
>> I'd like to suport boost::format if possible (more powerful/safer than
>> sprintf --- understands user types)
>
> I hear from Daniel and Serge that boost::format performance is very
> slow, but it shouldn't be hard to support, with no penalty for messages
> that don't meet the threshold.
>
> Robert wrote:
>> Does the log call prevent inlining of functions? I'm worried that it
>> might, esp. with varargs.
>
> Should you be logging inside an inlined function? You might have to
> compile out in this case.
>
> Jim wrote:
>> I'm also a little bothered by the extensive use of preprocessor macros; I'm
>> guessing that's at least somewhat related to the way the context is
>> handled, though they're also the easiest way to ensure that things compile
>> down to nothing when desired, and perhaps part of dealing with variadic
>> arguments for format strings. If it's at all possible to do things without
>> macros, I'd much prefer that, even if it involves a bit of C++ template
>> metaprogramming (note that with C++11, the variadic stuff at least becomes
>> much, much easier).
>
> Actually, I think the macros do only two things that can't (easily) be
> done with functions:
> * capture file/function/line location information
> * provide a place where we could insert #ifdefs to compile out
>
> Robert wrote:
>> I don't like the support for more than one interface (you illustrate 2
>> python options, one using python logging). But I suppose that we can
>> handle this by policy.
>
> Jim wrote:
>> I do definitely like the idea of being able to use Python's built-in
>> logging module with a custom handler. I don't think it'd be a bad idea to
>> just use Python's logging module as the front-end in all of our logging
>> code; would there be any major disadvantages to doing this?
>
> The thinking is that the Python logging interface is a simpler, more
> familiar, but less-performant and less-featureful interface that L3
> users can use. I think there are interoperability problems with using
> both, though: protolog only provides the back end, while Python logging
> provides its own separate logger name and threshold handling.
>
> My suggestion here would be to standardize on protolog until a more
> complete Python logging integration can be done (if that's even
> possible).
>
> Robert wrote:
>> I think we need to be able to change logging levels from an
>> interactive python prompt.
>
> Bill is quite sure this can be done.
>
> Robert wrote:
>> This is probably supported, but with
>> debug("afw.image", 1, ...)
>> debug("afw.image.fitsio", 2, ...)
>> Can I enable afw debug logging at level 1 (which applies to afw.image
>> and afw.image.fitsio), then switch afw.image to 2 (which also applies
>> to afw.image.fitsio), and then change afw.image.fitsio to yet another
>> log level?
>
> Yes, this should work.
>
> Jim wrote:
>> I'm not looking forward to doing a lot of configuration via XML, but I can
>> also see how trying to wrap the entire log4cxx configuration interface in
>> something friendlier wouldn't be a good use of our time either.
>
> log4cxx already supports Java property syntax, which is much more
> user-friendly.
>
> Paul wrote:
>> Haven't had a chance to read the docs yet, but one question I have beyond
>> RHL's is whether the logging will support multithreading.
>
> Yes, it does. The thread id can be placed in the log message.
>
> Paul wrote:
>> I also would desire that the logger be picklable.
>
> I'm not sure how hard this is.
>
> --
> Kian-Tat Lim, LSST Data Management, [log in to unmask]
> _______________________________________________
> LSST-data mailing list
> [log in to unmask]
> http://listserv.lsstcorp.org/mailman/listinfo/lsst-data
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
