Hello,

In the minutes of your last meeting, we've seen that you are trying to
load W13 data in Qserv. This interests us a lot because we may plan to use
this dataset on CCIN2P3 250 nodes cluster.

As asked by Jacek, we've worked on tests/runTests.py script and it seems
to works quite well now (even if some improvments are still needed).

That's why Emmanuel Gangler subtly suggests us to give you some feedback
about this testing procedure, because in fact it contains an algorithm
which partition/load/generate meta for a small dataset.

In tests/runTests.py, the function loadPartitionedData() has been rewrited
w.r.t. Jacek comments, and it seems it could be interesting to generalize
it for various LSST data. It relies on partition.py (a bit patched for tsv
support) and load.py (patched with ugly debug messages)

I will be on holidays next week, but Emmanuel Medernach may work on it,
and study how to partition, load, and generate meta for PT1.1 Source and
Object data with this function. I hope and think that
loadPartitionedData() algorithm should't be to much impacted by this
generalization, but only the configuration parameters of the input data
(hard-coded at the beginning of the function, but which could be easily
readable from a config file). If this algorithm could be used for data
loading, some interesting job would also remains in order to distribute
loading, and why not, meta-generation.

Hope this helps.

Cheers,

Fabrice

P.S. :

You can try it by launching tests/runTests.py (assuming you've got a
qserv.conf in your #/.lsst dir, as explained in README.txt).
(In addtion this ticket has been merged with a templating system update,
and a minor bugfix in qserv-stop in order to stop also mysqld, cmsd and
xrootd).

########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1