Here is a link to the smallest file on dropbox:
https://www.dropbox.com/s/pep68o1tfc5ugzd/ap6.6gev050mev.slcio
I have a terrible upload speed, so I couldn't get the larger ones up in any
reasonable time frame. Hopefully this one will do!
On an unrelated note, I have a probably stupid question. In JAS3, how do I
load an AIDA file as part of a script? All the examples online seem to
generate a new random file to do analysis on, and there doesn't seem to be
any clear example of what command loads a pre-existing file to modify.
On Mon, Oct 14, 2013 at 2:29 PM, Sho Uemura <[log in to unmask]> wrote:
> Dropbox is fine, and no rush - spacing events should make things work for
> you, so finding this memory leak is not critical.
>
> You want to make it use the collection name EcalTriggerClusters (defined
> in the steering file) instead of EcalClusters.
>
>
> On Mon, 14 Oct 2013, Kyle McCarty wrote:
>
> If you give me about a half-hour to hour, I can try to upload the files to
>> dropbox and see if I can get them to you that way. I am not in a position
>> where I can do that at the moment, unfortunately, and I do not have a JLab
>> account yet.
>>
>> I tried using the HPS2014ReadoutNoPileup.lcsim driver. By itself, it works
>> great and seems to resolve the massive memory issue. However, when I input
>> the HPSTriggerPlotsDriver, it gives the following error:
>>
>> java.lang.**IllegalArgumentException: Unknown event component
>> EcalClusters
>> at hep.physics.event.BaseEvent.**get(BaseEvent.java:48)
>> at org.lcsim.event.base.**BaseLCSimEvent.get(**
>> BaseLCSimEvent.java:105)
>> at
>> org.lcsim.hps.analysis.ecal.**HPSEcalTriggerPlotsDriver.**process(**
>> HPSEcalTriggerPlotsDriver.**java:113)
>> at org.lcsim.util.Driver.**doProcess(Driver.java:273)
>> at org.lcsim.util.Driver.**processChildren(Driver.java:**284)
>> at org.lcsim.util.Driver.process(**Driver.java:198)
>> at org.lcsim.util.DriverAdapter.**recordSupplied(DriverAdapter.**
>> java:74)
>> at
>> org.freehep.record.loop.**DefaultRecordLoop.**consumeRecord(**
>> DefaultRecordLoop.java:832)
>> at
>> org.freehep.record.loop.**DefaultRecordLoop.loop(**
>> DefaultRecordLoop.java:668)
>> at
>> org.freehep.record.loop.**DefaultRecordLoop.execute(**
>> DefaultRecordLoop.java:566)
>> at org.lcsim.util.loop.LCSimLoop.**loop(LCSimLoop.java:153)
>> at org.lcsim.job.**JobControlManager.run(**
>> JobControlManager.java:393)
>> at org.lcsim.job.**JobControlManager.main(**
>> JobControlManager.java:169)
>>
>> Any idea on why it fails when inserted into the same spot on this driver,
>> but not the old one?
>> I guess you don't have a JLab computing account?
>>
>> Is there any web server at UNH that you can upload these files to so I can
>> grab them?
>>
>> The easiest thing to do would be to use HPS2014ReadoutNoPileup.lcsim -
>> that
>> doesn't simulate pileup at all, so it's safe to use with unspaced events,
>> and is otherwise a drop-in replacement for HPS2014ReadoutToLcio.lcsim.
>> I'll
>> add a note about that on the Confluence page.
>>
>> You can also use the instructions for spacing photon beam data for readout
>> simulation, given on Confluence - that hasn't been tested for A' events
>> but
>> should work. But I would only do that if you find there's something about
>> HPS2014ReadoutNoPileup.lcsim that doesn't work for you; and let me know in
>> that case.
>>
>> On Mon, 14 Oct 2013, Kyle McCarty wrote:
>>
>> I could upload the SLCIO files somewhere if you can suggest a good place.
>> I
>>
>>> am unsure of the best way to host them - they are between 0.25 - 2.5 GB,
>>> so
>>> they are rather large.
>>>
>>> I had not considered spacing. The background events I had been working
>>> with
>>> were already spaced with many empty events, so I suppose I assumed the A'
>>> files would be too. I'm happy to apply some spacing if you send me the
>>> instructions.
>>> On Oct 14, 2013 1:26 PM, "Sho Uemura" <[log in to unmask]> wrote:
>>>
>>> Is there any way you can make your input LCIO files available?
>>>
>>>>
>>>> Your events have a lot of hits because you have no spacing between A'
>>>> events (we normally add empty bunches so that A' events only occur once
>>>> every 500 events) - that's probably exacerbating any memory problems.
>>>> You
>>>> should add empty bunches, since without empty bunches your results are
>>>> going to be nonsense; in production MC this is done at the stdhep level,
>>>> but I can work out simpler instructions for you to do it with hps-java.
>>>>
>>>> But I agree that your symptoms suggest some memory leak that needs to be
>>>> looked at. We can help.
>>>>
>>>> On Mon, 14 Oct 2013, Kyle McCarty wrote:
>>>>
>>>> Hello hps-software,
>>>>
>>>>
>>>>> I have been running some A' events through the lcsim software and have
>>>>> been
>>>>> running into memory problems.
>>>>>
>>>>> System Information:
>>>>> OS: Red Hat Enterprise Linux Server release 5.7 (Tikanga)
>>>>> RAM: 22 GB
>>>>>
>>>>> SLCIO File Generation Information:
>>>>> SLIC: 4.9.6
>>>>> GEANT4: 9.6.1
>>>>> Geometry: HPS-Proposal2014-v5-6pt6.lcdd
>>>>> Input Files: ap6.6gevXXXmev.stdhep
>>>>> where XXX = { 050, 100, 200, 300, 400, 500, 600 } are the A' masses.
>>>>>
>>>>> LCSim Information:
>>>>> hps-java: 1.8
>>>>> Drivers:
>>>>> - EventMarkerDriver
>>>>> - CalibrationDriver
>>>>> - TestRunTriggeredReconToLcio
>>>>> - FADCEcalReadoutDriver
>>>>> - EcalRawConverterDriver
>>>>> - CTPEcalClusterer
>>>>> - FADCTriggerDriver
>>>>> - SimpleSvtReadout
>>>>> - HPSEcalTriggerPlotsDriver
>>>>> - AidaSaveDriver
>>>>> - ClockDriver
>>>>> The steering file is attached for more detailed reference. It is a
>>>>> modified
>>>>> version of Sho's HPS2014ReadoutToLcio.lcsim.
>>>>>
>>>>> Problem Manifestation:
>>>>> When I started running the A' events through LCSim, I got heap errors
>>>>> and
>>>>> OutOfMemoryErrors. These were intially resolved by including the
>>>>> -Xmx[Amount] option when running, but for the larger files (>1 GB,
>>>>>
>>>>> 100,000
>>>>>>
>>>>>> events) I still received memory errors even when I allotted Java the
>>>>> entirety of the server's available memory. I was ultimately able to get
>>>>> all
>>>>> the files to run by downloading them to my personal machine (a Windows
>>>>> device) and running hps-java there, but it was necessary to allot Java
>>>>> approximately 55 GB of RAM to accomplish this.
>>>>>
>>>>> I ran some diagnostics while the LCSim software was running on my local
>>>>> machine and observed the memory footprint of the software. I found that
>>>>> it
>>>>> started low, but continually increased throughout the duration of the
>>>>> run.
>>>>> My guess from what I saw is that the Java virtual machine is not
>>>>> correctly
>>>>> cleaning old objects from memory, so they are building up causing the
>>>>> large
>>>>> event files to rapidly expand in memory.
>>>>>
>>>>> I have attached two log files. The first is from the 2.5 GB A' file for
>>>>> 600
>>>>> MeV masses. This run was ultimately terminated by me because it reached
>>>>> the
>>>>> maximum amount of server memory that I could allot it and then froze
>>>>> while
>>>>> it tried to get more memory. The second log files are from another run
>>>>> where it did yield an OutOfMemoryError.
>>>>>
>>>>> Any ideas as to cause of this?
>>>>>
>>>>> ##############################******##########################**
>>>>> ##**##**
>>>>>
>>>>> ############
>>>>> Use REPLY-ALL to reply to list
>>>>>
>>>>> To unsubscribe from the HPS-SOFTWARE list, click the following link:
>>>>> https://listserv.slac.****stanfo**rd.edu/cgi-bin/wa?****SUBED1=**<http://rd.edu/cgi-bin/wa?**SUBED1=**>
>>>>> HPS-SOFTWARE&A=1<http://**stanford.edu/cgi-bin/wa?****
>>>>> SUBED1=HPS-SOFTWARE&A=1<http://stanford.edu/cgi-bin/wa?**SUBED1=HPS-SOFTWARE&A=1>
>>>>> >
>>>>> <https://**listserv.slac.**stanford.edu/**cgi-bin/wa?**SUBED1=HPS-**<http://listserv.slac.stanford.edu/**cgi-bin/wa?SUBED1=HPS-**>
>>>>> SOFTWARE&A=1<https://listserv.**slac.stanford.edu/cgi-bin/wa?**
>>>>> SUBED1=HPS-SOFTWARE&A=1<https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1>
>>>>> >
>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> ##############################******##########################**
>>>>> ##**##**
>>>>>
>>>> ############
>>>> Use REPLY-ALL to reply to list
>>>>
>>>> To unsubscribe from the HPS-SOFTWARE list, click the following link:
>>>> https://listserv.slac.****stanfo**rd.edu/cgi-bin/wa?****SUBED1=**<http://rd.edu/cgi-bin/wa?**SUBED1=**>
>>>> HPS-SOFTWARE&A=1<http://**stanford.edu/cgi-bin/wa?****
>>>> SUBED1=HPS-SOFTWARE&A=1<http://stanford.edu/cgi-bin/wa?**SUBED1=HPS-SOFTWARE&A=1>
>>>> >
>>>> <https://**listserv.slac.**stanford.edu/**cgi-bin/wa?**SUBED1=HPS-**<http://listserv.slac.stanford.edu/**cgi-bin/wa?SUBED1=HPS-**>
>>>> SOFTWARE&A=1<https://listserv.**slac.stanford.edu/cgi-bin/wa?**
>>>> SUBED1=HPS-SOFTWARE&A=1<https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1>
>>>> >
>>>>
>>>>>
>>>>>
>>>>
>>>> ##############################****############################**##**
>>> ############
>>> Use REPLY-ALL to reply to list
>>>
>>> To unsubscribe from the HPS-SOFTWARE list, click the following link:
>>> https://listserv.slac.**stanfo**rd.edu/cgi-bin/wa?**SUBED1=**
>>> HPS-SOFTWARE&A=1<http://stanford.edu/cgi-bin/wa?**SUBED1=HPS-SOFTWARE&A=1>
>>> <https://**listserv.slac.stanford.edu/**cgi-bin/wa?SUBED1=HPS-**
>>> SOFTWARE&A=1<https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1>
>>> >
>>>
>>>
>>>
>> ##############################**##############################**
>> ############
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the HPS-SOFTWARE list, click the following link:
>> https://listserv.slac.**stanford.edu/cgi-bin/wa?**SUBED1=HPS-SOFTWARE&A=1<https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1>
>>
>>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the HPS-SOFTWARE list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1
