Fabrice, Serge, Daniel, Douglas, Jacek,
Agenda:
* quick round table status report
* outstanding action items
* coding standards
* layout for Qserv hackathon day 1
* Qserv modularization
quick round table status report
===============================
Daniel
- join syntax, including logic for substituting names
for overlap, previous version was "terrible", current
version is more general
Serge
- concurrency-related tickets
- progress with SciDB tests, but run into hardware problems
Fabrice
- working on auto testing, have clean system
Jacek
- final touches to W14 Qserv plan
- prep for hackathon:
- proposal for day 1 layout on trac
- added "tentative list of outcomes", see
https://dev.lsstcorp.org/trac/wiki/db/Qserv/RedesignFY2014/201311_hackathon
- coordinated build/package discussion time
w/Mario and Robert (Mon 9:30am)
- arranged meeting rooms
- points from wishlists documented in appropriate places
(where obvious)
- documented python usage, see:
https://dev.lsstcorp.org/trac/wiki/db/Qserv/RedesignFY2014/pythonCPP
- researched and documented mysql wrappers in python, see:
https://dev.lsstcorp.org/trac/wiki/db/MySQLWrapper4Python
- started work on interfaces for Data Loading, see:
https://dev.lsstcorp.org/trac/wiki/db/Qserv/DataLoading/Interfaces/InputData
- working on letter of collaboration for Florin's DOE grant
Douglas
- evaluated Zookeeper
- now looking into Doozerd, etcd, Concoord and Openreplica
- all documented at:
https://dev.lsstcorp.org/trac/wiki/db/Qserv/ClusterControl
Side discussions:
- wrapper
- qserv might need more raw access that other parts that
talk to mysql
- want to avoid accessing mysql through python in Qserv core,
only from admin
- qms likely goes away, replaced by something like Zookeeper
- btw, we have another wrapper in admin used for data loader
Cluster mgmt tools
- Qserv will likely run on variety of clusters, some of
them not under our control, and they might require
specialized cluster mgmt tools (and won't accept our).
We can't support all of them, we will pick one and will
support it, if others want to use different cluster mgmt
tools, they will have to do the porting
One large ticket vs several smaller tickets?
- from review point, smaller tickets ar better
- but there is overhead per ticket (branch, merging etc)
Qserv code reviews
- Issue K-T raises from time to time: want non-qserv
people to review Qserv tickets
- Often push back "why was I assigned to this ticket?"
- We still want to review our own tickets to keep up
to date with changes
Keeping documentation of tests run by Fabrice on in2p3 servers
- not big issue for now, but better to keep all
in one place, eg. on trac
Monitor trac changes daily
- e.g., use timeline, or "watch" pages
outstanding action items
========================
- #3007 CLOSED
- #3013 reviewed, change owners to FJ and close
- #3004 depends on #3013, close when #3013 closed
- then close #3006 when above done [FJ]
- #3019 (concurrency) almost done
- #3020 not done yet, unlikely to happen in the
next few days
- cut release
- include xrootd patch 5
- will cut another release with 3019, 3020
- document what is now done in python DONE
- hackathon planning DONE
- document ideas about user friendly communication
and syntax parsing ("--unsafe flag") STILL TODO [DW]
- modularization DONE
- lay out list of things that can be done
on either worker or master STILL TODO [DW]
- bring Mario/Robert discussion about
building/packaging, during hackathon week DONE
- add to trac hackathon planning: "tentative list
of outcomes" DONE
- take apart wishlists and document each point
raised in appropriate place DONE
- work on objectId index STILL TODO, CAN WAIT [DW]
- join syntax improvements DONE
- small tweaks to shared scan ticket DONE
- follow up with Florin Rusu, check his code DONE
- talk to Robyn about integrating with buildbot
sometime later next week DONE
- send Fabrice info about policies/restrictions
regarding in2p3 contributions *during construction*
DONE (Dominique plugged into the loop)
- work on collaboration with Florin Rusu ON TRACK
coding standards
================
- most files in admin don't have any headers, comments
see https://dev.lsstcorp.org/trac/wiki/CodeStandards
- raising/handling errors, incl throwing exceptions
- would be useful to have one style and stick with it
Qserv modularization
====================
See:
https://listserv.slac.stanford.edu/cgi-bin/wa?A2=ind1311&L=QSERV-L&F=&S=&P=5294
- this covers C++ only
- note, that replaces common, worker and master directories
- looks fine at first glance
- need to cover python to see complete flow
- think about modules for admin [FJ, DS]
- would be useful to prefix some files,
otherwise once we add (many) files from admin,
it'll be too messy
- e.g., prefix all files that belong to master (with 'm'?)
and all files that belong to admin (with 'a'?)
proposed layout for Qserv hackathon day 1
=========================================
https://listserv.slac.stanford.edu/cgi-bin/wa?A2=ind1311&L=QSERV-L&F=&S=&P=574
sounds like a good plan for 1st day, then we will revisit
- possibly, might need few larger mtgs
Jacek
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
