Hello Brad,
So, I tried some more things and it finally worked. It seems that there
were two, separate, issues that occurred. The first was that the batch
system likely failed because it accessed a local version of the conditions
database that is out-of-date with the newest version, and doesn't have the
newer detectors yet. I need to test to verify this, but can't yet as I
don't have the administrative privileges to update the local database.
The second issue did, oddly, appear to be the fieldmap. Running just the
command, independent of the batch system, allows HPS-Java to access the
SLAC database (the batch system has no internet connectivity). Doing this,
with the fieldmap present, actually works. I didn't really think that
spacing used the fieldmap, but I guess the LCIOReader tries to initialize
it by default.
Thanks,
Kyle
On Thu, Feb 4, 2016 at 6:15 PM, Bradley T Yale <[log in to unmask]>
wrote:
> Hey Kyle,
>
>
> I think I may have found the problem.
>
>
> First, I managed to successfully filter the events after linking with the
> fieldmap (seems to be a common theme among errors lately...)
>
>
> ln -s /u/group/hps/hps_soft/fieldmaps fieldmap
>
> java -cp
> /u/group/hps/hps_soft/svn/hps-java-trunk/distribution/target/hps-distribution-3.5-SNAPSHOT-bin.jar
> org.hps.users.meeg.FilterMCBunches
> /cache/mss/hallb/hps/production/PhysicsRun2016/slic/moller/2pt3/molv1_HPS-PhysicsRun2016-Nominal-v4-4-fieldmap_478.slcio
> Mol_v4-4-fieldmap_filt.slcio -e250 -E0.05 -w20000000
>
>
> Second, for whatever reason, the new detectors after 2pt2-v0 are NOT in
> the latest jar contained in the usual 'hps-java' directory
> (/u/group/hps/hps_soft/hps-java/hps-distribution-3.5-20160125.215702-33-bin.jar)
> even after getting the latest 3.5-SNAPSHOT from NEXUS.
>
>
> However, they ARE found in the updated local trunk:
>
>
>
> "/u/group/hps/hps_soft/svn/hps-java-trunk/distribution/target/hps-distribution-3.5-SNAPSHOT-bin.jar"
>
>
> So using that jar should work.
>
>
>
> ------------------------------
> *From:* Kyle McCarty <[log in to unmask]>
> *Sent:* Thursday, February 4, 2016 4:27 PM
> *To:* Bradley T Yale
> *Cc:* McCormick, Jeremy I.; Nathan Baltzell; hps-software
>
> *Subject:* Re: Conditions Drivers Error for New Detector
>
> Hello Brad,
>
> I have always been able to space files in the past without issue using
> this command, but it is possible that something changed more recently.
> Ultimately, though, I am pretty confident that this issue has something to
> do with the detector properties not being found. It crashes immediately
> upon being run and produces an error; if it were just not writing files
> because it was given a write maximum of zero, I would expect the process to
> complete without crashing.
>
> Also note that I used this same command to space the previous batch of 2.2
> GeV files with the old detector and did not have any issues.
>
> Thanks,
>
> Kyle
>
> On Thu, Feb 4, 2016 at 4:22 PM, Bradley T Yale <[log in to unmask]>
> wrote:
>
>> By the way, the flags which are normally set when spacing events are
>>
>>
>> -e250 -E0.05 -w20000000 (or some large integer that isn't a 'long')
>>
>> Looking at the code, the 'a' flag is only tied to how many events are
>> read.
>>
>> However, if you leave out the 'w' option, which tells you how many events
>> to write, then it seems that none get written.
>>
>> Right, Sho?
>>
>>
>> That may solve the mystery of your empty filtered files...
>>
>>
>> ------------------------------
>> *From:* [log in to unmask] <[log in to unmask]>
>> on behalf of Kyle McCarty <[log in to unmask]>
>> *Sent:* Thursday, February 4, 2016 4:02 PM
>> *To:* McCormick, Jeremy I.
>> *Cc:* Nathan Baltzell; hps-software
>>
>> *Subject:* Re: Conditions Drivers Error for New Detector
>>
>> Hello Jeremy,
>>
>> There are multiple ways to get this, but the easiest is probably the
>> command that I sent you earlier:
>>
>> java -cp $HPS_JAVA org.hps.users.meeg.FilterMCBunches
>>> raw/molv1_HPS-PhysicsRun2016-Nominal-v4-4-fieldmap_100.slcio
>>> molv1_HPS-PhysicsRun2016-Nominal-v4-4-fieldmap_100-spaced.slcio -e500 -a
>>
>>
>> You get a similar error when trying to open the file in JAS3 as well.
>> Nathan found the calorimeter properties file here:
>>
>>
>> http://java.freehep.org/svn/repos/hps/list/java/trunk/detector-data/detectors/HPS-PhysicsRun2016-Nominal-v4-4/SamplingFractions/?revision=HEAD&bypassEmpty=true
>>
>> Do you know where I need to put this?
>>
>> Thanks,
>>
>> Kyle
>>
>> On Thu, Feb 4, 2016 at 3:59 PM, McCormick, Jeremy I. <
>> [log in to unmask]> wrote:
>>
>>> Hi, Kyle.
>>>
>>> Can you paste the exact command you're using on the command line here?
>>>
>>> --Jeremy
>>> ------------------------------
>>> *From:* [log in to unmask] <[log in to unmask]>
>>> on behalf of Kyle McCarty <[log in to unmask]>
>>> *Sent:* Thursday, February 4, 2016 12:33 PM
>>> *To:* Nathan Baltzell
>>> *Cc:* hps-software
>>> *Subject:* Re: Conditions Drivers Error for New Detector
>>>
>>> Hello hps-software,
>>>
>>> Jeremy says that this error means that either the conditions database is
>>> not being initialized correctly, or that the sampling fractions .PROP file
>>> is missing from the detector description. I believe that the conditions
>>> database is being read, so I think the missing file is the issue. Can
>>> someone of the detector end confirm whether this is or is not present? Is
>>> there something that I need to do to obtain it or enable it?
>>>
>>> Thanks,
>>>
>>> Kyle
>>>
>>> On Wed, Feb 3, 2016 at 10:32 PM, Kyle McCarty <[log in to unmask]> wrote:
>>>
>>>> Hello Nathan,
>>>>
>>>> Looking further, I figured out part of the problem: the spaced Møller
>>>> files have no events (file size 0), but for some reason wrote files anyway.
>>>> Looking into the log files for the spacing jobs, I see that the following
>>>> error occurred:
>>>>
>>>> Exception in thread "main" java.lang.NullPointerException
>>>>> at
>>>>> org.lcsim.event.base.SamplingFractionManager.getCorrectedEnergy(SamplingFractionManager.java:34)
>>>>> at
>>>>> org.lcsim.event.base.BaseCalorimeterHit.calculateCorrectedEnergy(BaseCalorimeterHit.java:116)
>>>>> at
>>>>> org.lcsim.lcio.SIOSimCalorimeterHit.<init>(SIOSimCalorimeterHit.java:68)
>>>>> at
>>>>> org.lcsim.lcio.SIOSimCalorimeterHitBlockHandler.addCollectionElements(SIOSimCalorimeterHitBlockHandler.java:23)
>>>>> at
>>>>> org.lcsim.lcio.AbstractBlockHandler.readCollection(AbstractBlockHandler.java:39)
>>>>> at
>>>>> org.lcsim.lcio.AbstractBlockHandler.readBlock(AbstractBlockHandler.java:32)
>>>>> at org.lcsim.lcio.LCIOReader.read(LCIOReader.java:77)
>>>>> at
>>>>> org.hps.users.meeg.FilterMCBunches.main(FilterMCBunches.java:177)
>>>>>
>>>>
>>>> Note that I also get this error if I try to open the input file in JAS3.
>>>>
>>>> Thanks,
>>>>
>>>> Kyle
>>>>
>>>> On Wed, Feb 3, 2016 at 10:10 PM, Nathan Baltzell <[log in to unmask]>
>>>> wrote:
>>>>
>>>>> See line 959 in JobControlManager, per your stack trace:
>>>>>
>>>>> LOGGER.info("Job took " + elapsed + " which is " + elapsed /
>>>>> processedEvents + " ms/event.");
>>>>>
>>>>> I guess processEvents is zero, and JobControlManager prefers to crash
>>>>> rather than deal with it?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Feb 3, 2016, at 9:44 PM, Kyle McCarty <[log in to unmask]> wrote:
>>>>>
>>>>> > Hello hps-software,
>>>>> >
>>>>> > Nathan pointed out that I was using 2015 instead of 2016 for some
>>>>> reason. Correcting that, I now get a different error:
>>>>> >
>>>>> > SEVERE: A fatal error occurred during the job.
>>>>> > java.lang.ArithmeticException: / by zero
>>>>> > at
>>>>> org.lcsim.job.JobControlManager.run(JobControlManager.java:959)
>>>>> > at org.hps.job.JobManager.run(JobManager.java:61)
>>>>> > at org.hps.job.JobManager.main(JobManager.java:27)
>>>>> >
>>>>> > I haven't ever seen this from the job manager before. Any idea why
>>>>> it would be producing this error? I have attached the steering file, if
>>>>> that helps.
>>>>> >
>>>>> > Thanks,
>>>>> >
>>>>> > Kyle
>>>>> >
>>>>> > On Wed, Feb 3, 2016 at 9:27 PM, Kyle McCarty <[log in to unmask]>
>>>>> wrote:
>>>>> > Hello hps-software,
>>>>> >
>>>>> > I am attempting to run readout on 2.3 GeV data with the new
>>>>> detector, but I obtain the following error:
>>>>> >
>>>>> > Exception in thread "main" java.lang.RuntimeException: Error
>>>>> initializing conditions from ConditionsDriver.
>>>>> > at
>>>>> org.hps.conditions.ConditionsDriver.initialize(ConditionsDriver.java:90)
>>>>> > at
>>>>> org.hps.job.JobManager.setupConditionsDriver(JobManager.java:84)
>>>>> > at org.hps.job.JobManager.setup(JobManager.java:49)
>>>>> > at
>>>>> org.lcsim.job.JobControlManager.setup(JobControlManager.java:1062)
>>>>> > at
>>>>> org.lcsim.job.JobControlManager.parse(JobControlManager.java:665)
>>>>> > at org.hps.job.JobManager.main(JobManager.java:26)
>>>>> > Caused by:
>>>>> org.lcsim.conditions.ConditionsManager$ConditionsNotFoundException:
>>>>> Conditions not found for detector HPS-PhysicsRun2015-Nominal-v4-4
>>>>> > at
>>>>> org.lcsim.conditions.ConditionsReader.create(ConditionsReader.java:203)
>>>>> > at
>>>>> org.lcsim.conditions.ConditionsReader.create(ConditionsReader.java:214)
>>>>> > at
>>>>> org.lcsim.conditions.ConditionsManagerImplementation.setDetector(ConditionsManagerImplementation.java:40)
>>>>> > at
>>>>> org.hps.conditions.database.DatabaseConditionsManager.initialize(DatabaseConditionsManager.java:716)
>>>>> > at
>>>>> org.hps.conditions.database.DatabaseConditionsManager.setDetector(DatabaseConditionsManager.java:1023)
>>>>> > at
>>>>> org.hps.conditions.ConditionsDriver.initialize(ConditionsDriver.java:84)
>>>>> > ... 5 more
>>>>> > Caused by: java.net.UnknownHostException: www.lcsim.org
>>>>> > at
>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>>>> > at
>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>>>>> > at
>>>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>>>> > at
>>>>> java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>>>>> > at
>>>>> sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1675)
>>>>> > at
>>>>> sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1673)
>>>>> > at java.security.AccessController.doPrivileged(Native Method)
>>>>> > at
>>>>> sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1671)
>>>>> > at
>>>>> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1244)
>>>>> > at
>>>>> org.lcsim.util.cache.FileCache.getCachedFile(FileCache.java:95)
>>>>> > at
>>>>> org.lcsim.conditions.ConditionsReader.downloadDetectorDescription(ConditionsReader.java:268)
>>>>> > at
>>>>> org.lcsim.conditions.ConditionsReader.create(ConditionsReader.java:194)
>>>>> > ... 10 more
>>>>> > Caused by: java.net.UnknownHostException: www.lcsim.org
>>>>> > at
>>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178)
>>>>> > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>>>>> > at java.net.Socket.connect(Socket.java:579)
>>>>> > at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
>>>>> > at
>>>>> sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
>>>>> > at
>>>>> sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
>>>>> > at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
>>>>> > at sun.net.www.http.HttpClient.New(HttpClient.java:308)
>>>>> > at sun.net.www.http.HttpClient.New(HttpClient.java:326)
>>>>> > at
>>>>> sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:996)
>>>>> > at
>>>>> sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:932)
>>>>> > at
>>>>> sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:850)
>>>>> > at
>>>>> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1300)
>>>>> > at
>>>>> sun.net.www.protocol.http.HttpURLConnection.getHeaderField(HttpURLConnection.java:2678)
>>>>> > at
>>>>> java.net.URLConnection.getHeaderFieldLong(URLConnection.java:639)
>>>>> > at
>>>>> java.net.URLConnection.getContentLengthLong(URLConnection.java:511)
>>>>> > at
>>>>> java.net.URLConnection.getContentLength(URLConnection.java:495)
>>>>> > at
>>>>> org.lcsim.util.cache.FileCache.getCachedFile(FileCache.java:94)
>>>>> > ... 12 more
>>>>> >
>>>>> >
>>>>> > It looks like conditions are not present for the detector
>>>>> "HPS-PhysicsRun2015-Nominal-v4-4" yet. Are they present on the main
>>>>> conditions database branch? We might just need to clone the current version
>>>>> to the UNH server (the batch system there doesn't have online access).
>>>>> >
>>>>> > Thanks,
>>>>> >
>>>>> > Kyle
>>>>> >
>>>>> >
>>>>> > Use REPLY-ALL to reply to list
>>>>> >
>>>>> > To unsubscribe from the HPS-SOFTWARE list, click the following link:
>>>>> >
>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1
>>>>> >
>>>>> > <TestTrigger_2pt2_MollerN2.lcsim>
>>>>>
>>>>>
>>>>
>>>
>>> ------------------------------
>>>
>>> Use REPLY-ALL to reply to list
>>>
>>> To unsubscribe from the HPS-SOFTWARE list, click the following link:
>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1
>>>
>>> ------------------------------
>>>
>>> Use REPLY-ALL to reply to list
>>>
>>> To unsubscribe from the HPS-SOFTWARE list, click the following link:
>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1
>>>
>>
>>
>> ------------------------------
>>
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the HPS-SOFTWARE list, click the following link:
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1
>>
>
>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the HPS-SOFTWARE list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1
