Attendees: Serge, Douglas, Bill, Jacek
300 node test
- "select from obj join source using objectId where objectId = <xx>"
results in full scan, objectId index not recognized, probably
subchunks materialized too. Code needs to be fixed
- still some issues with getting cluster correctly configured
for smaller # nodes
- need to run few low-volume queries (eg "select from source
in a very small region")
- run near neighbor for 10x10 deg
- returning the nodes ~Sept 24
bugs
- worker still hosed under some conditions, writing/reading
results is synchronized incorrectly: results are read
before they are populated
- some bugs Bill found are handled through fault tolerance,
code not yet merged to master
logging
- Bill wrote some code for logging, thread safe,
outputs buffers one full line at a time, with thread
id and time stamp
- using boost filtered strings, not using any package
like logcxx
- it will be easy to migrate once we settle on the package
we should use long term
what is next (not - till the construction starts)
- high priority: finalize documentation or FDR
- then: code refactoring
Qserv code refactoring between now and start of
the Construction - two options:
a) bottom up (start from the existing prototype
and cleanup/fix/tweak/improve)
or
b) top down (come up with a fresh clean design based
on all lessons learned, reuse pieces from the existing
prototype)
- overall sentiment based on very preliminary discussion
(without Daniel!): "b"
- main reason: existing prototype has too many layers,
very hard to debug
- other points made:
- would be good to implement geometry support
in c++, not from python
- would be good to get rid of subchunks
- would be good to push more code from python to C++
- also... want to switch to the MariaDb distribution
The topic of code refactoring will be discussed
in details in the next few weeks...
Jacek
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
