I have never been able to get LCSim to run properly through
Eclipse/NetBeans, so I am just using the command line directly. Is there a
way to run profiling without them?
The files are, to the best of my knowledge, just A' events, though Maurik
will have to confirm that, as I got the files from him. The largest two
files had ~170,000 events and came out to 1.4 and 2.5 GB respectively. I'm
not, admittedly, sure what went into the original StdHEP file generation
though. The source file was 78 MB, if that helps.
On Oct 14, 2013 1:23 PM, "Graham, Mathew Thomas" <[log in to unmask]>
wrote:
> Yeah, definitely a memory leak somewhere. If you are using netbeans or
> eclipse, you can run the profiler to pinpoint just where it's leaking. I'm
> not sure based on your list, but I typically run fewer events-per-job.
>
> Mostly I'm chiming in because of this…
>
> but for the larger files (>1 GB, >100,000 events)
>
>
> …is that really the size of these files? I.e 100k events=1GB? Are these
> A' only? Because my A' only events are ~10x smaller than this…even pileup
> shouldn't effect things much (unless truth info is kept for every beam
> electron…then it get's big).
>
> On Oct 14, 2013, at 10:12 AM, Kyle McCarty <[log in to unmask]> wrote:
>
> Hello hps-software,
>
> I have been running some A' events through the lcsim software and have
> been running into memory problems.
>
> System Information:
> OS: Red Hat Enterprise Linux Server release 5.7 (Tikanga)
> RAM: 22 GB
>
> SLCIO File Generation Information:
> SLIC: 4.9.6
> GEANT4: 9.6.1
> Geometry: HPS-Proposal2014-v5-6pt6.lcdd
> Input Files: ap6.6gevXXXmev.stdhep
> where XXX = { 050, 100, 200, 300, 400, 500, 600 } are the A' masses.
>
> LCSim Information:
> hps-java: 1.8
> Drivers:
> - EventMarkerDriver
> - CalibrationDriver
> - TestRunTriggeredReconToLcio
> - FADCEcalReadoutDriver
> - EcalRawConverterDriver
> - CTPEcalClusterer
> - FADCTriggerDriver
> - SimpleSvtReadout
> - HPSEcalTriggerPlotsDriver
> - AidaSaveDriver
> - ClockDriver
> The steering file is attached for more detailed reference. It is a
> modified version of Sho's HPS2014ReadoutToLcio.lcsim.
>
> Problem Manifestation:
> When I started running the A' events through LCSim, I got heap errors and
> OutOfMemoryErrors. These were intially resolved by including the
> -Xmx[Amount] option when running, but for the larger files (>1 GB, >100,000
> events) I still received memory errors even when I allotted Java the
> entirety of the server's available memory. I was ultimately able to get all
> the files to run by downloading them to my personal machine (a Windows
> device) and running hps-java there, but it was necessary to allot Java
> approximately 55 GB of RAM to accomplish this.
>
> I ran some diagnostics while the LCSim software was running on my local
> machine and observed the memory footprint of the software. I found that it
> started low, but continually increased throughout the duration of the run.
> My guess from what I saw is that the Java virtual machine is not correctly
> cleaning old objects from memory, so they are building up causing the large
> event files to rapidly expand in memory.
>
> I have attached two log files. The first is from the 2.5 GB A' file for
> 600 MeV masses. This run was ultimately terminated by me because it reached
> the maximum amount of server memory that I could allot it and then froze
> while it tried to get more memory. The second log files are from another
> run where it did yield an OutOfMemoryError.
>
> Any ideas as to cause of this?
>
> ------------------------------
>
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the HPS-SOFTWARE list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1
> <hps-java.log><hpsjava_2.log><hpsjava_2_err.log><ecalAnalysis.lcsim>
>
>
>
> ------------------------------
>
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the HPS-SOFTWARE list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1
>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the HPS-SOFTWARE list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1
