Attendees: Serge, AndyH, Daniel, Douglas, Bill, Jacek

sourceId larger than 64 bits
  - if we have to... probably best to have composite key
    instead of sourceId
  - impact on qserv?
    - duplicator cares, needs to remap ids
    - larger sourceIds would be an issue if we decide
      to duplicate refSourceMatch
  - worry: people will rely on meaning of individual bits
    - can't easily enforced to not do that
    - knowledge how bits are used easily gets lots
    - BaBar, Atlas had very bad experience with that!
      --> need to talk to KT and others


new xrootd client
  - thread-pool model instead of thread-per-request model
  - completely async, with real callbacks
  - at least as fast as the old version, in some cases
    10x faster, (eg, metadata requests)
  - interfaces different
  - new client not released, but stable version ready
  - expect few more months before it reaches production
    quality
  - don't try to use it for large scale tests in June/July


xrootd - new interface that does not look like a file system
  - think: rpc instead of files to communicate
  - will simplify potential migration if we ever decide to
    replace backend in the future
  - all tied to a single communication (sending query and
    retrieving results is now in two streams)
  - integrating with qserv will take ~ a month
  - time scale: this year


parser
  - adding support for spatial constraints
  - expecting to have queries running next week


Serge
  - finalizing unit tests, then documentation
  - next step: integrating with install scripts
    (Douglas/IN2P3/Serge)


PipeQA
  - dusting off pipeQA code used for W13, need to
    be committed, merged etc, work in #2773
  - ready to copy the qserved W13 to ncsa
     - but need space on /usr/data/mysql
       (backup still not moved off to other place)


metadata
  - first iteration of integration with qserv done (#2725)
  - still to do: fixing lightweight tests of parser
    and improving robustness


lsst-dbdev{1-6}
  - want to do code dev there, need to find if
    our home directories are backed up


copyright line in file headers inconsistent,
would be nice to fix


buildbot
  - only rebuild qserv
  - redo dependencies as needed (very infrequently) manually


Error handling (Bill)
  - many errors that were previously killing qserv are now
    gracefully caught, 1st error nicely reported back to user
  - activities logged per query in memory, then can be dumped
    to a database. 3 use cases:
    - always dump if query fails
    - turn on full logging for debugging
    - dump statistics (one row per query)
  --> merge master into the ticket, request review, push to
      master
  - then deal with more failure modes (eg try to corrupt
    data stream, database file etc and gracefully handle it)


cutting release
  - cut one now, include essential fixes (eg workaround
    described in 2786)
  - cut new releases every few months as needed [Douglas]


test on 120 yillies up & working, full table scan ~50 sec
  - document in a trac page [Douglas]
  - run stress test focused on concurrency issues [Douglas]


qserv admin reworked, supports multi-node now

Daniel will document how to rewrite @poly query for pipeQA

Jacek

########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1