LISTSERV mailing list manager LISTSERV 16.5

Help for HPS-SOFTWARE Archives


HPS-SOFTWARE Archives

HPS-SOFTWARE Archives


HPS-SOFTWARE@LISTSERV.SLAC.STANFORD.EDU


View:

Message:

[

First

|

Previous

|

Next

|

Last

]

By Topic:

[

First

|

Previous

|

Next

|

Last

]

By Author:

[

First

|

Previous

|

Next

|

Last

]

Font:

Monospaced Font

LISTSERV Archives

LISTSERV Archives

HPS-SOFTWARE Home

HPS-SOFTWARE Home

HPS-SOFTWARE  September 2012

HPS-SOFTWARE September 2012

Subject:

Re: HPS conditions framework

From:

"McCormick, Jeremy I." <[log in to unmask]>

Reply-To:

Software for the Heavy Photon Search Experiment <[log in to unmask]>

Date:

Fri, 14 Sep 2012 12:37:12 -0700

Content-Type:

multipart/mixed

Parts/Attachments:

Parts/Attachments

text/plain (52 lines) , winmail.dat (52 lines)

Hi,

Thanks for the email.

This is a discussion that should be shared on the hps-software mailing list, as there are several people working with this more closely that should be in the loop, namely Matt, Sho, and Omar. Maurik should also be aware of this because he's in charge of the HPS software effort. I've added all of these people and the list to this message.

My opinion is that a MySQL database is overly complicated for what we're currently doing, and I don't see anyone needing anything like this for now. Recently a bunch of files were removed from the hps-detectors CVS, which is the source for HPS conditions, and put into the resources area of the hps-java project where they can be accessed as plain old Java resources. This data was from the test run, and reason for doing this was that the conditions had to all be copied between the different detector model directories to be accessed, which, as you can imagine, involves an enormous amount of data duplication. And many of these data sets are being used to analyze ALL data from the test run across all runs, so putting them into a conditions system by run or time does not make that much sense.

The problem with a database, as I see it, is that the user either has to run this themselves on their local machine, which is an extra amount of installation and setup, OR there has to be a shared instance that hps-java accesses, especially for jobs run on batch farms or the grid. (The database also has to be kept up to date, which is more complicated than the current method of 'cvs up' to get new text files.) Either one of these working methods could cause problems with how we currently do things. For one, many batch systems are not that friendly when it comes to connecting out to databases unless this is preconfigured by their admins to allow it. And having users run their own MySQL database just strikes me as an unnecessary headache. We already have enough troubles getting people to install and run the software on their own machines, and this would add another hurdle that frankly isn't necessary right now.

For the analysis of test run data and for the foreseeable future, we are fine in terms of LCSim infrastructure in this area, in my opinion. Given that the next HPS "run" is not going to be for several years, at least, as the JLab schedule was pushed back, I'd like to see attention focused elsewhere, namely on JAS/AIDA development.

We've started to collect a list of requested features and bug fixes here.

https://confluence.slac.stanford.edu/display/hpsg/HPS+Software+Wish+List

What you added to the current conditions system for accessing databases should be fine, at least for people just to play around with and experiment. There isn't anything on that list having to do with conditions, but people can add it if they wish or see the need. To me, the need for this kind of "proper" conditions system with run/time tagging and a database backend is just so far into the future that it is practically irrelevant right now.

That's mostly just my opinion though. I wonder if others have anything to add to the discussion...?

--Jeremy

-----Original Message-----
From: Onoprienko, Dmitry
Sent: Friday, September 14, 2012 11:24 AM
To: Neal, Homer A.; Johnson, Tony; McCormick, Jeremy I.; Graf, Norman A.; Nelson, Timothy Knight
Subject: HPS conditions framework

Hello Everyone,

I just want to touch base regarding the HPS use of org.lcsim conditions framework.

A few months ago, I was asked by Tony (the original request came from Homer I believe) to modify the framework and enable reading conditions from MySQL database instead of a zip file. I was also asked to make it possible for conditions to be run-dependent. Since HPS was preparing for the beam test at that time, Norman and Jeremy stressed the importance of keeping changes to org.lcsim code to absolute minimum. I also found that HPS was already using the conditions framework in the traditional way - getting data from zip files - so that capability had to be preserved.

Given these constraints, I made a few minor changes to standard implementations on ConditionManager and ConditionsReader, and created a hook that can be used to load experiment-specific conditions reader. The changes are backward compatible and transparent to the user - the framework will still look for a conditions archive or directory with a name derived from the detector name it finds in the org.lcsim event. However, if the detector.properties file in that archive contains a line in
"ConditionsReader: <ClassName>" format, the framework will instantiate <ClassName> class and use it in place of the standard reader.

There is an example of use in org.lcsim.hps.conditions package (hps-java project, see package javadoc for instructions on how to run it). Since we knew nothing about the structure of the actual HPS conditions data, the example is just a trivial demo that shows how to fetch something from a database and retrieve it through the conditions framework.

Now that people seem to be actually using conditions, I wonder if it's time to take the next step. Is the scheme of loading a custom conditions reader through an entry in the detector.properties file convenient ? There are alternatives that would make the framework behavior a bit more transparent at a price of requiring more substantial changes to the codebase. Is anyone familiar with HPS data interested in working with me to get some more realistic conditions into the database ?

Best Regards,
- Dmitry.


########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the HPS-SOFTWARE list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1

Top of Message | Previous Page | Permalink

Advanced Options


Options

Log In

Log In

Get Password

Get Password


Search Archives

Search Archives


Subscribe or Unsubscribe

Subscribe or Unsubscribe


Archives

April 2024
March 2024
February 2024
January 2024
December 2023
November 2023
October 2023
September 2023
August 2023
July 2023
June 2023
May 2023
April 2023
March 2023
February 2023
January 2023
December 2022
November 2022
October 2022
September 2022
August 2022
June 2022
April 2022
March 2022
February 2022
January 2022
December 2021
November 2021
October 2021
September 2021
August 2021
July 2021
June 2021
May 2021
April 2021
March 2021
February 2021
January 2021
December 2020
November 2020
October 2020
September 2020
August 2020
July 2020
June 2020
May 2020
April 2020
March 2020
February 2020
January 2020
December 2019
November 2019
October 2019
September 2019
August 2019
July 2019
June 2019
May 2019
April 2019
March 2019
February 2019
January 2019
December 2018
November 2018
October 2018
September 2018
August 2018
July 2018
June 2018
May 2018
April 2018
March 2018
February 2018
January 2018
December 2017
November 2017
October 2017
September 2017
August 2017
July 2017
June 2017
May 2017
April 2017
March 2017
February 2017
January 2017
December 2016
November 2016
October 2016
September 2016
August 2016
July 2016
June 2016
May 2016
April 2016
March 2016
February 2016
January 2016
December 2015
November 2015
October 2015
September 2015
August 2015
July 2015
June 2015
May 2015
April 2015
March 2015
February 2015
January 2015
December 2014
November 2014
October 2014
September 2014
August 2014
July 2014
June 2014
May 2014
April 2014
March 2014
February 2014
January 2014
December 2013
November 2013
October 2013
September 2013
August 2013
July 2013
June 2013
May 2013
April 2013
March 2013
February 2013
January 2013
December 2012
November 2012
October 2012
September 2012
August 2012
July 2012
June 2012
May 2012
April 2012
March 2012
February 2012
January 2012
December 2011
November 2011
October 2011
September 2011
August 2011
July 2011
June 2011
May 2011
April 2011
March 2011

ATOM RSS1 RSS2



LISTSERV.SLAC.STANFORD.EDU

Secured by F-Secure Anti-Virus CataList Email List Search Powered by the LISTSERV Email List Manager

Privacy Notice, Security Notice and Terms of Use