From: Kian-Tat Lim <[log in to unmask]>

Subject: Re: [LSST-data] input about logging needed by mid next week

Date: May 5, 2014 at 2:36:57 PM PDT

To: Robert Lupton the Good <[log in to unmask]>

Cc: LSST Data Management <[log in to unmask]>

Robert and Jim and Paul,

Robert wrote:

Is there support for python "with" statements (with logging context)

Jim wrote:

My biggest concern at this stage is the way the context switching is
handled; having to manually pop the context seems like it's going against
the trend of how context management is being done in C++ and Python these
days.  I think Robert's of supporting 'with' in Python is a good one, but I
think we should also continue to have log objects that know their own
context.  RAII is the natural (and only safe, IMO) way to do resource
management in C++, and I think it's desirable (and convenient) to be able
to do the same thing in Python.

It is easy to wrap the current pushContext and popContext calls within
both C++ and Python objects (or replace them, although that may involve
more SWIG).  I have suggested this to Bill already.

Robert wrote:

Having enough debug levels.  I'd like to have at least 3 and maybe 5
different debug levels, and preferably control them by a numerical
parameter not a logger name (protolog.debug(3, ...) rather than
protolog.debug3(...)) to enable changing the level of groups of debug
commands. Yes, you can do this with  mylog = {3: protolog.debug3);
mylog[3](...) but that's more work.

Bill and I were thinking that having more loggers with their own names
is similar to having more debug levels.  Aside from syntactic sugar,
which we can work out, I think you get near-equivalent functionality by
turning on and off "mylog-level4" or "mylog-level5" instead of having
"mylog" with level 4 or 5.

Robert wrote:

output control.  It is convenient to make different debug levels
format with different indents

This may be a bit more difficult.  log4j 2.0 shows how we could extend
the log4cxx PatternLayout class to enable this, but it's something we'd
have to add "under" (or "plug into") log4cxx.

Robert wrote:

I'd like to suport boost::format if possible (more powerful/safer than
sprintf --- understands user types)

I hear from Daniel and Serge that boost::format performance is very
slow, but it shouldn't be hard to support, with no penalty for messages
that don't meet the threshold.

Robert wrote:

Does the log call prevent inlining of functions?  I'm worried that it
might, esp. with varargs.

Should you be logging inside an inlined function?  You might have to
compile out in this case.

Jim wrote:

I'm also a little bothered by the extensive use of preprocessor macros; I'm
guessing that's at least somewhat related to the way the context is
handled, though they're also the easiest way to ensure that things compile
down to nothing when desired, and perhaps part of dealing with variadic
arguments for format strings.  If it's at all possible to do things without
macros, I'd much prefer that, even if it involves a bit of C++ template
metaprogramming (note that with C++11, the variadic stuff at least becomes
much, much easier).

Actually, I think the macros do only two things that can't (easily) be
done with functions:
* capture file/function/line location information
* provide a place where we could insert #ifdefs to compile out

Robert wrote:

I don't like the support for more than one interface (you illustrate 2
python options, one using python logging).  But I suppose that we can
handle this by policy.

Jim wrote:

I do definitely like the idea of being able to use Python's built-in
logging module with a custom handler.  I don't think it'd be a bad idea to
just use Python's logging module as the front-end in all of our logging
code; would there be any major disadvantages to doing this?

The thinking is that the Python logging interface is a simpler, more
familiar, but less-performant and less-featureful interface that L3
users can use.  I think there are interoperability problems with using
both, though: protolog only provides the back end, while Python logging
provides its own separate logger name and threshold handling.

My suggestion here would be to standardize on protolog until a more
complete Python logging integration can be done (if that's even
possible).

Robert wrote:

I think we need to be able to change logging levels from an
interactive python prompt.

Bill is quite sure this can be done.

Robert wrote:

This is probably supported, but with
debug("afw.image", 1, ...)
debug("afw.image.fitsio", 2, ...)
Can I enable afw debug logging at level 1 (which applies to afw.image
and afw.image.fitsio), then switch afw.image to 2 (which also applies
to afw.image.fitsio), and then change afw.image.fitsio to yet another
log level?

Yes, this should work.

Jim wrote:

I'm not looking forward to doing a lot of configuration via XML, but I can
also see how trying to wrap the entire log4cxx configuration interface in
something friendlier wouldn't be a good use of our time either.

log4cxx already supports Java property syntax, which is much more
user-friendly.

Paul wrote:

Haven't had a chance to read the docs yet, but one question I have beyond
RHL's is whether the logging will support multithreading.

Yes, it does.  The thread id can be placed in the log message.

Paul wrote:

I also would desire that the logger be picklable.

I'm not sure how hard this is.

--
Kian-Tat Lim, LSST Data Management, [log in to unmask]
_______________________________________________
LSST-data mailing list
[log in to unmask]
http://listserv.lsstcorp.org/mailman/listinfo/lsst-data