-------- Original Message --------
Subject: [LSST-data] belate notes from qserv mtg Apr 11
Date: Thu, 18 Apr 2013 22:21:56 -0700
From: Jacek Becla <[log in to unmask]>
To: LSST Data Management <[log in to unmask]>
Attendees: Daniel, Serge, Douglas, Bill, Jacek
- start trac page "do and don't do", things like
"don't use port numbers except for proxy and qms,
instead rely on sockets", "abstract away mysql
connection and don't bypass it"
- #2805 (install in scons) - need more info what is
needed: "make clean"? "rm -rf dist"? something else?
[Jacek]
- parser work
- technically, it more than parser, it is parser,
query generation, query dispatch...
- count * queries work, dispatch much faster
120 node setup on yilis
- Douglas will post how to run
- Serge will reproduce concurrency issues and will
make tweaks to fix
- full rewrite of threading model not necessary,
because new xrootd client will solve that issue
- lower ulimit and demonstrate qserv can handle it
w13
- not fully loaded to qserv, something silently failed
- try lsst-dev
- try new partitioner
- implementing checking after every step to catch
potential issues
- rewriting to use any number of batch machines
- eg last time took 10 days, now on 40 batch
jobs finished in 10h
- what to do when adding data to the same table fails?
- reload, but with "update"
buildbot
- bootstraping qserv now reuires scons, which
was not on the machine Robyn used
install script by default wipes out everything,
not ideal for developers, easy to trigger by
mistake rebuilding everything.
- add some protection against it
nice progress with message reporting
https://dev.lsstcorp.org/trac/wiki/db/Qserv/MessageReporting
- next step: try to pretend data corruption,
network disruption, and catch the errors gracefully
Jacek
_______________________________________________
LSST-data mailing list
[log in to unmask]
http://listserv.lsstcorp.org/mailman/listinfo/lsst-data
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
