Qserv: Questions from in2p3 to SLAC and related answers

Emmiters				Emmanuel Medernach, computer scientist, developer and EGI TIER2 manager
					Fabrice Jammes, computer scientist, developer and head of LPC/in2p3 IT department

Approval		6Feb2013	Emmanuel Gangler, scientist, head of LSST team at LPC/in2p3 
Diffusion
For action | SLAC (Douglas Smith, Daniel Wang, Jacek Becla)
For information | LSST-France
Change history
Date | Author | Description of the update |
6Feb2013 | LPC | Creation

N° | Theme | Priority | Remark | Status |

1.

Install

 Ticket/ #2511

High

IN2P3  Question (submitted on 1Feb2013) :
Is the code review for ticket #2511 (scons automated install) done ? If not, what is the timeline ? This is a priority item as further development like automated tests, are dependent on this ticket.

SLAC Answer :

Open

2.

Data-loading

Med

IN2P3  Question (submitted on 1Feb2013) :
Could we have some information about PT1.1 'Source' loading, as it would allow to complete the data loading and to execute queries for Qserv validation. 
Is the next command ok :
/opt/qserv-dev/bin/python /opt/qserv-dev/qserv/master/examples/partition.py --output-dir /data/lsst/pt11_partition --chunk-prefix Source --theta-column 33 --phi-column 34 /data/lsst/pt11/Source.txt --num-stripes 10 --num-sub-stripes 2 
are these parameters correct : --theta-column 33 --phi-column 34 

SLAC Answer :

Open

3.

Automated tests

Low

IN2P3  Question (submitted on 6Feb2013) :
 Is the development on tests/runTest.py inside the perimeter of ticket #2014 or will a new ticket be created ? At the moment development is done in branch “u/fjammes/automatedTest”
 
SLAC Answer :

Open

4.

Automated tests
/
Data-loading

Low

IN2P3  Question (submitted on 6Feb2013) :
The automated tests with qserv mode needs to load partitioned data. In your tests/runTest.py comments,  you propose this : 
#use the loadPartitionedObjectTables.py script to generate loadO
but  loadPartitionedObjectTables.py doesn't exist.
Should we use instead master/examples/loader.py, or re-implement in Python the load() function of admin/custom/bin/qserv-admin.pl ?

SLAC Answer :

Open

5.

Data-loading

Med

IN2P3  Question (submitted on 6Feb2013) :
Furthermore, in order to allow Qserv users to save expensive storage cost., we propose to develop a dynamic loader, which load data in DB directly at partitionning, without having to store partitioned data in intermediate files ?  Do you think it is interesting ?

SLAC Answer :

Open

6.

Minutes

Med

IN2P3  Question  : Wiki page for Phone meeting minutes (submitted 6Feb2013)
We would like to have a centralized place to put phone meeting minutes, could it be possible to use your wiki pages for that ?

SLAC Answer :

Open

7.

Benchmarking

Med

IN2P3  Question : Standard performance tests for QServ (submitted 6Feb2013)
Thanks for giving us next link :  https://dev.lsstcorp.org/trac/wiki/dbQserv/perTableKeyCache .
Does the queries defined in also fit for Qserv master and node performance and benchmarking tests ? Beyond individual queries, is there a document describing a typical queries workload to be used for benchmarking ? Could we define standard performance tests for QServ master and nodes ? Could you provide the ones you used before ?

SLAC Answer :

Open