LISTSERV mailing list manager LISTSERV 16.5

Help for QSERV-L Archives


QSERV-L Archives

QSERV-L Archives


QSERV-L@LISTSERV.SLAC.STANFORD.EDU


View:

Message:

[

First

|

Previous

|

Next

|

Last

]

By Topic:

[

First

|

Previous

|

Next

|

Last

]

By Author:

[

First

|

Previous

|

Next

|

Last

]

Font:

Proportional Font

LISTSERV Archives

LISTSERV Archives

QSERV-L Home

QSERV-L Home

QSERV-L  April 2013

QSERV-L April 2013

Subject:

Fwd: [LSST-data] belate notes from qserv mtg Apr 11

From:

Jacek Becla <[log in to unmask]>

Reply-To:

General discussion for qserv (LSST prototype baseline catalog)

Date:

Thu, 18 Apr 2013 22:35:27 -0700

Content-Type:

text/plain

Parts/Attachments:

Parts/Attachments

text/plain (72 lines)

-------- Original Message --------
Subject: [LSST-data] belate notes from qserv mtg Apr 11
Date: Thu, 18 Apr 2013 22:21:56 -0700
From: Jacek Becla <[log in to unmask]>
To: LSST Data Management <[log in to unmask]>

Attendees: Daniel, Serge, Douglas, Bill, Jacek

- start trac page "do and don't do", things like
    "don't use port numbers except for proxy and qms,
    instead rely on sockets", "abstract away mysql
    connection and don't bypass it"

- #2805 (install in scons) - need more info what is
    needed: "make clean"? "rm -rf dist"? something else?
    [Jacek]

- parser work
     - technically, it more than parser, it is parser,
       query generation, query dispatch...
     - count * queries work, dispatch much faster

120 node setup on yilis
   - Douglas will post how to run
   - Serge will reproduce concurrency issues and will
     make tweaks to fix
   - full rewrite of threading model not necessary,
     because new xrootd client will solve that issue
   - lower ulimit and demonstrate qserv can handle it


w13
   - not fully loaded to qserv, something silently failed
   - try lsst-dev
   - try new partitioner
   - implementing checking after every step to catch
     potential issues
   - rewriting to use any number of batch machines
     - eg last time took 10 days, now on 40 batch
       jobs finished in 10h
   - what to do when adding data to the same table fails?
      - reload, but with "update"


buildbot
   - bootstraping qserv now reuires scons, which
     was not on the machine Robyn used


install script by default wipes out everything,
not ideal for developers, easy to trigger by
mistake rebuilding everything.
   - add some protection against it


nice progress with message reporting
https://dev.lsstcorp.org/trac/wiki/db/Qserv/MessageReporting
   - next step: try to pretend data corruption,
     network disruption, and catch the errors gracefully

Jacek
_______________________________________________
LSST-data mailing list
[log in to unmask]
http://listserv.lsstcorp.org/mailman/listinfo/lsst-data

########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1

Top of Message | Previous Page | Permalink

Advanced Options


Options

Log In

Log In

Get Password

Get Password


Search Archives

Search Archives


Subscribe or Unsubscribe

Subscribe or Unsubscribe


Archives

March 2018
February 2018
January 2018
December 2017
August 2017
December 2016
November 2016
October 2016
September 2016
August 2016
July 2016
June 2016
May 2016
April 2016
March 2016
February 2016
January 2016
December 2015
November 2015
October 2015
September 2015
August 2015
July 2015
June 2015
May 2015
April 2015
March 2015
February 2015
January 2015
December 2014
November 2014
October 2014
September 2014
August 2014
July 2014
June 2014
May 2014
April 2014
March 2014
February 2014
January 2014
December 2013
November 2013
October 2013
September 2013
August 2013
July 2013
June 2013
May 2013
April 2013
March 2013
February 2013
January 2013
December 2012

ATOM RSS1 RSS2



LISTSERV.SLAC.STANFORD.EDU

Secured by F-Secure Anti-Virus CataList Email List Search Powered by the LISTSERV Email List Manager

Privacy Notice, Security Notice and Terms of Use