Attendees: Serge, AndyH, Daniel, Douglas, Bill, Jacek
sourceId larger than 64 bits
- if we have to... probably best to have composite key
instead of sourceId
- impact on qserv?
- duplicator cares, needs to remap ids
- larger sourceIds would be an issue if we decide
to duplicate refSourceMatch
- worry: people will rely on meaning of individual bits
- can't easily enforced to not do that
- knowledge how bits are used easily gets lots
- BaBar, Atlas had very bad experience with that!
--> need to talk to KT and others
new xrootd client
- thread-pool model instead of thread-per-request model
- completely async, with real callbacks
- at least as fast as the old version, in some cases
10x faster, (eg, metadata requests)
- interfaces different
- new client not released, but stable version ready
- expect few more months before it reaches production
quality
- don't try to use it for large scale tests in June/July
xrootd - new interface that does not look like a file system
- think: rpc instead of files to communicate
- will simplify potential migration if we ever decide to
replace backend in the future
- all tied to a single communication (sending query and
retrieving results is now in two streams)
- integrating with qserv will take ~ a month
- time scale: this year
parser
- adding support for spatial constraints
- expecting to have queries running next week
Serge
- finalizing unit tests, then documentation
- next step: integrating with install scripts
(Douglas/IN2P3/Serge)
PipeQA
- dusting off pipeQA code used for W13, need to
be committed, merged etc, work in #2773
- ready to copy the qserved W13 to ncsa
- but need space on /usr/data/mysql
(backup still not moved off to other place)
metadata
- first iteration of integration with qserv done (#2725)
- still to do: fixing lightweight tests of parser
and improving robustness
lsst-dbdev{1-6}
- want to do code dev there, need to find if
our home directories are backed up
copyright line in file headers inconsistent,
would be nice to fix
buildbot
- only rebuild qserv
- redo dependencies as needed (very infrequently) manually
Error handling (Bill)
- many errors that were previously killing qserv are now
gracefully caught, 1st error nicely reported back to user
- activities logged per query in memory, then can be dumped
to a database. 3 use cases:
- always dump if query fails
- turn on full logging for debugging
- dump statistics (one row per query)
--> merge master into the ticket, request review, push to
master
- then deal with more failure modes (eg try to corrupt
data stream, database file etc and gracefully handle it)
cutting release
- cut one now, include essential fixes (eg workaround
described in 2786)
- cut new releases every few months as needed [Douglas]
test on 120 yillies up & working, full table scan ~50 sec
- document in a trac page [Douglas]
- run stress test focused on concurrency issues [Douglas]
qserv admin reworked, supports multi-node now
Daniel will document how to rewrite @poly query for pipeQA
Jacek
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
