 |
 |
I do think repeatability is absolutely vital when running _reconstruction_ over the same data set; for simulation, and readout is a simulation step, I think it’s desirable as well. Ideally, we should be able to supply the simulation(s) a random seed and get the same hits/clusters/tracks every time we as long as we use the same random seed (on the same architecture etc..). I’m not sure how practical this is though. On Sep 18, 2014, at 9:25 AM, McCormick, Jeremy I. <[log in to unmask]> wrote: > Anyone have any additional thoughts about this... > > On Sep 18, 2014, at 9:06 AM, Jeremy McCormick <[log in to unmask]> wrote: > >> Hi, >> >> I understand there is built-in random noise in the ReadoutChip classes within the digitization code, so that’s one know source of (intended) variation. It would be nice if the random seeds for these objects were somehow externally settable so that results could be made deterministic if required. For testing the MC recon, I got around this by using a digi Driver that has a NoiselessReadoutChip. That seemed to reduce the variation somewhat. >> >> As for the other point, I do understand there will be some variation of results between runs, due to iterating over hash map value lists, which are determined by object memory ordering that changes each time you run the recon. I’ve actually done a little bit of hacking to try and replace all usages of HashMap with LinkedHashMap in the tracking in lcsim. But I still get slight differences on the level of 1 or 2 more or fewer tracks on the order of every 1000 recon events. >> >> This concerns me, because it means at some level that our MC results are not deterministically repeatable, and it also means that writing meaningful, integrated test cases that use the MC recon is basically impossible. >> >> This is not the case for the ECAL reconstruction, which always gives the same number of digitized hits and clusters when noise is turned off. >> >> On a related note, the ECAL readout simulation itself still has some amount of variation in the number of events it writes, even when noise is turned off. For instance, across multiple runs on 2.5 million input events, the output would vary (seemingly randomly) between 944 and 945 triggered events. >> >> Do you think this stuff is worth looking into? Is having the readout sim and recon be perfectly repeatable a desirable or necessary goal? >> >> BTW, I noticed this stuff when writing a test case where I tried to assert that X number of tracks should be found in a given recon run on a known input file. This is basically impossible because the number of tracks always varies slightly. I also wanted to write a chaining test to run the readout and then the recon, and again, it wasn’t possible because the number of output events from the trigger simulation was varying slightly between each run. >> >> —Jeremy >> >> On Sep 18, 2014, at 8:38 AM, Sho Uemura <[log in to unmask]> wrote: >> >>> SimpleSvtReadout: <addNoise>false</addNoise> >>> >>> Ordering of hits in collections is probably going to be random since we're iterating over HashMaps. >>> >>> I think CDFSiSensorSim is deterministic but I don't know for sure. Ask Tim. >>> >>> Can't guarantee there isn't anything else. This is not something we've been careful about. >>> >>> On Thu, 18 Sep 2014, McCormick, Jeremy I. wrote: >>> >>>> If I turn off noise in the FADC Driver, is there any other source of randomness in the readout simulation that would prevent event processing from being exactly repeatable? >>>> >>>> If so, how can I turn it off? >>>> >>>> >>>> >> > > ######################################################################## > Use REPLY-ALL to reply to list > > To unsubscribe from the HPS-SOFTWARE list, click the following link: > https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1 ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the HPS-SOFTWARE list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1