Quick summary of what we (AndyS, Nate and me) discussed. 
We will discuss it more at the Qserv meeting…


* Zookeeper is not great for quick and frequent updates, so implementing  
  things like synchronizing access to resources such as table state/metadata
  in zookeeper is questionable
* if we combine it with fact that watchers are not reliable and with the pains 
  with c++ bindings, we’d basically want to …. get rid of zookeeper

Proposal
 * push CSS metadata into transactional mysql 
 * register every query in query metadata, including interactive
 * change Facade API: new API would only have “fetch all metadata for a given
   table”
 * KVInterface - stays as is (granularity: key/value). Add support for updates
   (Eg we need to set flags used to sync access to tables and that’d be done
   through KVInterface)
 * implement new MySQLKvInterface
 * need to think how to store CSS info internally in mysql. Pack as JSON?
 * high availability: standby replica(s) + auto-fail over. Two modes
    1) Don’t handle new queries while fail over is happening, or
    2) we could maintain a stand-by snapshot in czar, and work off the snapshot 
        while fail over is happening
 * need a flag in mysql per table to synchronize access
   -  process that wants to delete a table would lock it (exclusive lock)
   - czar would lock it (shared read lock) when it starts processing new query
 * Need python interface for qservAdmin type things, Could wrap the C++ one


I hope that is enough to plan it in your heads. Think about it over the next
day or so, and we will discuss more on Wed.

Jacek


########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1