LISTSERV mailing list manager LISTSERV 16.5

Help for QSERV-L Archives


QSERV-L Archives

QSERV-L Archives


QSERV-L@LISTSERV.SLAC.STANFORD.EDU


View:

Message:

[

First

|

Previous

|

Next

|

Last

]

By Topic:

[

First

|

Previous

|

Next

|

Last

]

By Author:

[

First

|

Previous

|

Next

|

Last

]

Font:

Proportional Font

LISTSERV Archives

LISTSERV Archives

QSERV-L Home

QSERV-L Home

QSERV-L  February 2013

QSERV-L February 2013

Subject:

Re: Proposition of discussion for our phone meeting

From:

Douglas Smith <[log in to unmask]>

Reply-To:

General discussion for qserv (LSST prototype baseline catalog)

Date:

Wed, 6 Feb 2013 17:13:48 -0800

Content-Type:

text/plain

Parts/Attachments:

Parts/Attachments

text/plain (90 lines)

Hello -

Ok, we should go over things for the meeting tomorrow, and the
phone conference has been setup:

Thu, Feb 7, 2013, 9:00 AM Pacific Standard Time
Meeting Type:	 Audio Only
Audio Details:	
Dial-In Number(s):
U.S. & Canada:    866.740.1260
U.S. Toll:    303.248.0285
France: 0800942408 (Toll Free)
Access Code: 9268664


The meeting will be from 9am to 10am tomorrow, since it seems
to have a slightly later meeting is agreeable to all.

Some question answers, from me at least, there might be other opinions:

#2: data loading - wait didn't we do the source partitioning and loading 
also
yet?  Um, looking at the Source.sql in the pt11 is looks like what you need
is: "--theta-column 6 --phi-column 9".  The rest of the options should work
fine.  Actually I think I looked that before, and the other columns 
would also
work, it seems that there are repeated ra and dec fields in the table,and
looking at the data quickly the values appear to be the same.

#3: Well, yes, that can be part of the ticket.  I think the important 
point there
is that the ticket should be for only running runTest.py from the 
installer.  Other
dev. of the use of runTest.py should be another ticket.

#4: Hmm... well good question.  Sounds like we need to create a re-usable
python loader class, maybe need a new ticket there?

#5:  Yeah, well, we've thought of this, and there was some talk with Serge
about having a type of partitioner which is more of a streaming executable.
Taking data from one input, and loading it into the database as output.
There is a concern, since on large data sets, these processes take days,
and are then quite fragile for errors, and you don't want the loading left
in some unknown state on failure, such that you need to re-start everything
from the beginning.  It would be good to be able to batch the 
partitioning and
loading into smaller when contained parts, and be able to re-start on 
the part
left over on failure.  But not sure that is the best thing to work on 
just yet,
I think we might need more experience on handle large datasets with the
partitioning and loading still.

#6: Um, maybe, does trac scale enough to have a new page added for each
meeting?

Douglas


On 02/06/2013 09:58 AM, Fabrice Jammes wrote:
> Hello Daniel, Douglas and Jacek,
>
> Qserv is working fine on 4 nodes, with Object PT1.1 data, at CCIN2P3,
> thanks to your help.
>
> Today, we send a questions list to you, as an attached document. We
> propose you to answer in the same document, above each questions. Your
> answers will help us a lot in continuing efficiently our job on Qserv.
>
> If you think a meeting would be useful in order to answer to these
> questions, we will be available by phone tomorrow from 9:00 pm to 10:00pm
> (SF time).
> Thanks for answering us today about this potential meeting.
>
> Yours sincerely,
>
> Emmanuel and Fabrice
>
> ########################################################################
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the QSERV-L list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1

########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1

Top of Message | Previous Page | Permalink

Advanced Options


Options

Log In

Log In

Get Password

Get Password


Search Archives

Search Archives


Subscribe or Unsubscribe

Subscribe or Unsubscribe


Archives

March 2018
February 2018
January 2018
December 2017
August 2017
December 2016
November 2016
October 2016
September 2016
August 2016
July 2016
June 2016
May 2016
April 2016
March 2016
February 2016
January 2016
December 2015
November 2015
October 2015
September 2015
August 2015
July 2015
June 2015
May 2015
April 2015
March 2015
February 2015
January 2015
December 2014
November 2014
October 2014
September 2014
August 2014
July 2014
June 2014
May 2014
April 2014
March 2014
February 2014
January 2014
December 2013
November 2013
October 2013
September 2013
August 2013
July 2013
June 2013
May 2013
April 2013
March 2013
February 2013
January 2013
December 2012

ATOM RSS1 RSS2



LISTSERV.SLAC.STANFORD.EDU

Secured by F-Secure Anti-Virus CataList Email List Search Powered by the LISTSERV Email List Manager

Privacy Notice, Security Notice and Terms of Use