On 02/06/2013 05:13 PM, Douglas Smith wrote:
> Hello -
>
> Ok, we should go over things for the meeting tomorrow, and the
> phone conference has been setup:
>
> Thu, Feb 7, 2013, 9:00 AM Pacific Standard Time
> Meeting Type: Audio Only
> Audio Details:
> Dial-In Number(s):
> U.S. & Canada: 866.740.1260
> U.S. Toll: 303.248.0285
> France: 0800942408 (Toll Free)
> Access Code: 9268664
>
>
> The meeting will be from 9am to 10am tomorrow, since it seems
> to have a slightly later meeting is agreeable to all.
>
> Some question answers, from me at least, there might be other opinions:
>
> #2: data loading - wait didn't we do the source partitioning and loading
> also
> yet? Um, looking at the Source.sql in the pt11 is looks like what you need
> is: "--theta-column 6 --phi-column 9". The rest of the options should work
> fine. Actually I think I looked that before, and the other columns
> would also
> work, it seems that there are repeated ra and dec fields in the table,and
> looking at the data quickly the values appear to be the same.
Ok, just adding more info here. So it looks like the fields raObject and
declObject are what you need for the partitioning of the source. But
you need to count the fields starting with '0', and you do that these
are fields 32 and 33, so you want "--theta-column 32 --phi-column 33"
for the options there.
>
> #3: Well, yes, that can be part of the ticket. I think the important
> point there
> is that the ticket should be for only running runTest.py from the
> installer. Other
> dev. of the use of runTest.py should be another ticket.
>
> #4: Hmm... well good question. Sounds like we need to create a re-usable
> python loader class, maybe need a new ticket there?
Ok, send me the command line example you used here for the loader.py,
and I'll look at the code and port over what is needed from qserv-admin.
Probably needs a new ticket.
>
> #5: Yeah, well, we've thought of this, and there was some talk with Serge
> about having a type of partitioner which is more of a streaming executable.
> Taking data from one input, and loading it into the database as output.
> There is a concern, since on large data sets, these processes take days,
> and are then quite fragile for errors, and you don't want the loading left
> in some unknown state on failure, such that you need to re-start everything
> from the beginning. It would be good to be able to batch the
> partitioning and
> loading into smaller when contained parts, and be able to re-start on
> the part
> left over on failure. But not sure that is the best thing to work on
> just yet,
> I think we might need more experience on handle large datasets with the
> partitioning and loading still.
>
> #6: Um, maybe, does trac scale enough to have a new page added for each
> meeting?
So, the answer is yes, please put the meeting minutes in the trac
wiki pages, linked from the Qserv pages:
https://dev.lsstcorp.org/trac/wiki/db/Qserv
Best to create the new page name starting with "Qserv/" like:
"Qserv/IN2P3meeting" or something like that.
>
> Douglas
>
>
> On 02/06/2013 09:58 AM, Fabrice Jammes wrote:
>> Hello Daniel, Douglas and Jacek,
>>
>> Qserv is working fine on 4 nodes, with Object PT1.1 data, at CCIN2P3,
>> thanks to your help.
>>
>> Today, we send a questions list to you, as an attached document. We
>> propose you to answer in the same document, above each questions. Your
>> answers will help us a lot in continuing efficiently our job on Qserv.
>>
>> If you think a meeting would be useful in order to answer to these
>> questions, we will be available by phone tomorrow from 9:00 pm to 10:00pm
>> (SF time).
>> Thanks for answering us today about this potential meeting.
>>
>> Yours sincerely,
>>
>> Emmanuel and Fabrice
>>
>> ########################################################################
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the QSERV-L list, click the following link:
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
> ########################################################################
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the QSERV-L list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
|
