LISTSERV mailing list manager LISTSERV 16.5

Help for QSERV-L Archives


QSERV-L Archives

QSERV-L Archives


QSERV-L@LISTSERV.SLAC.STANFORD.EDU


View:

Message:

[

First

|

Previous

|

Next

|

Last

]

By Topic:

[

First

|

Previous

|

Next

|

Last

]

By Author:

[

First

|

Previous

|

Next

|

Last

]

Font:

Proportional Font

LISTSERV Archives

LISTSERV Archives

QSERV-L Home

QSERV-L Home

QSERV-L  April 2013

QSERV-L April 2013

Subject:

Re: Some important points about Qserv, and the integration of our work.

From:

"Daniel L. Wang" <[log in to unmask]>

Reply-To:

General discussion for qserv (LSST prototype baseline catalog)

Date:

Thu, 4 Apr 2013 11:03:27 -0700

Content-Type:

text/plain

Parts/Attachments:

Parts/Attachments

text/plain (129 lines)

Hello,

> - qserv-0.3.0rc6.1.tar.gz : was done the 27 Nov 2012, before we came at
> SLAC, that's why the code related to automatic install isn't in this
> package.
Uh, I'm not sure why there is a rc6.1 release, except that maybe 
something happened in the packaging, because it should be the same code 
as rc6.

> - qserv-0.4.0.tar.gz doesn't contains the complete automated install
> procedure (for example : admin/bootstrap isn't in this directory)
Let's define a "release" as a code version that is stable enough to 
function and run some basic queries. Releases should, in general, 
represent temporary pauses in development and imply that some (minor) 
stabilization work has been done. Releases should represent a state of 
the code that can be explained. During development, code is often in a 
state where several pieces are being developed at once, and it may be 
hard to explain what works and what doesn't.

None of the qserv releases are "stable" in the typical meaning, because 
we are still breaking and rewriting major portions. No part of qserv is 
safe from rewriting and refactoring.

> That's why we thinks it would be a good idea to make a release
> containing the full automated install procedure.
Releases should always be made off the master, not on a ticket. This 
implies that the review processes, documentation, testing, and 
stabilization work has been done. The idea is that the ticket developer 
believes the ticket is done.

> In order to do that, the commit 98e3b5e1afac03a91b1f7ca3acdd09326435374a
> of ticket #2511 contains a valid install procedure as this version runs
> on several nodes with PT1.1 objects.
> And we think that the commit 8110ed9df7ce3ac872fd4e9e1c11d0f67ad55d0b of
> the master branch, which was the one done at the merge of #2511 ticket,
> contains the valid install procedure. Do you want us to fully test this
> again this in order to make a new qserv release containing this
> automated install procedure ? Or shall we wait in order to make a
> release containing also #2014 test procedure ?
I will leave it to Douglas to decide when to cut a release (0.x.* -> 
0.x+1.0), but in general, releases should happen at stabilization 
points, and hopefully, only one rc is needed (which becomes the 
release). I am in favor of rapid release cycles.

> Futhermore it seems serv-0.3.0rc6.1.tar.gz and qserv-0.4.0.tar.gz are
> not tagged in git repository. So we can't determine from with
> commit/branch they were produced.
I think this was an oversight. We should ensure that the tags are pushed 
(via git push --tags) before making and publishing a release tarball.

> Mono-node support :
> -------------------
>
> - Is mono-node supported by current master branch ? If no, thanks to
> make it works because, as asked by Jacek, ticket #2014 was implemented
> for mono-node only. Mono-node support is a pre-requisite so that we can
> merge the #2014 ticket.
To me, it "works," but my definition of "works" may be different.

>
> New mysqld-based export path support :
> --------------------------------------
>
> This functionnality is very usefull, thanks for implementing this.
> Nevertheless, our automated install procedure needs to be adapted in
> order to support it.
The appropriate thing to do is to cut a new ticket off of master to 
implement the updates to the installation procedure, get it done, 
reviewed, and merged again. It may be a small change, but that's 
great--it can be merged quickly!


> GRANT ALL ON qservw_worker.* TO 'qsmaster'@'localhost';
As Jacek said, the permissions are sloppy right now. This is because we 
have not implemented a way to pass authentication credentials into the 
ofs/oss plugins. Can you grant read-only permissions to everyone on that db?

> No XRDLCLROOT set. Bug in xrootd?
> No databases found to export.
Actually, XRDLCLROOT might not be necessary anymore, so I will have a 
look and perhaps remove the warning message. Not finding any Dbs is 
probably a permission problem, but maybe there's a better message to report.

> And in cmsd failed to start and produce next cmsd.log :

> 130404 17:06:51 001 oss_getPlugin: Unable to open libqserv_workerCmsd.so
> libqserv_workerCmsd.so: cannot open shared object file: No such file or
> directory
Looks like the cmsd was unable to load the plugin. Perhaps a path is not 
set? The path was not necessary for cmsd before because it did not need 
a plugin before.

> Could we have some more elements to make it works.
> A full install procedure for this feature, at least for mono-node
> installation, would be welcome.
Sorry, there hasn't been one documented--but hopefully the above 
comments are enough. If not, please ask again. :)

> Continuous integration :
> ------------------------
>
> We think that having a continuous integration server which would launch
> the automated install procedure and the data tests could be a good
> solution to improve collaborative work and the quality of the software.
We agree! We are in the process of getting the LSST buildbot configured 
for such a task.

> Do you thinks we could make this work after the #2014 merge in master
> branch ?
We are trying to get the system up, as we have resources to work on it. 
This may be before or after the 2014 merge (which could take longer 
because it looks like a big ticket and might take a while to review).
>
> 300 nodes à CC-IN2P3 :
> ----------------------
>
> Which release do you plan to install at CC-in2p3 ?
Something stable? I want the new parser/query-generation/manipulation 
framework to be ready but it's not certain it will be.

Hope this helps,
-Daniel

########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1

Top of Message | Previous Page | Permalink

Advanced Options


Options

Log In

Log In

Get Password

Get Password


Search Archives

Search Archives


Subscribe or Unsubscribe

Subscribe or Unsubscribe


Archives

March 2018
February 2018
January 2018
December 2017
August 2017
December 2016
November 2016
October 2016
September 2016
August 2016
July 2016
June 2016
May 2016
April 2016
March 2016
February 2016
January 2016
December 2015
November 2015
October 2015
September 2015
August 2015
July 2015
June 2015
May 2015
April 2015
March 2015
February 2015
January 2015
December 2014
November 2014
October 2014
September 2014
August 2014
July 2014
June 2014
May 2014
April 2014
March 2014
February 2014
January 2014
December 2013
November 2013
October 2013
September 2013
August 2013
July 2013
June 2013
May 2013
April 2013
March 2013
February 2013
January 2013
December 2012

ATOM RSS1 RSS2



LISTSERV.SLAC.STANFORD.EDU

Secured by F-Secure Anti-Virus CataList Email List Search Powered by the LISTSERV Email List Manager

Privacy Notice, Security Notice and Terms of Use