 |
 |
Actually since we are running many CPF processes in parallel it is impossible to remove duplicates if you read the same hbook file or the same run in two different jobs. About the creation of the reduced collections you mean you are using the same tcl file we used to produce hbook files. If any run is present in two different tcl's it produces for sure duplicated events, correct? Daniele On Wed, 6 Feb 2002, Riccardo Faccini wrote: > Hi, > one of things Cecilia noticed in using CPFramework is that the duplicate > removal seems not to be turned on by default (I did not remember this). Tu > turn it on you should toggle cpf/nomulti. > As far as the reduced collection creation the way it works is that for > each event we check if it is in the list. The only way to get duplicates > is to run twice on the same event AND to have the TS twice in the files. > > Of course I cannot completely exclude an operational error such as filling > twice the same reduced collection. > It might be worthwhile making a passa at removing duplicates from the > reduced collections I am doing right now > ciao > ric > > > > On Wed, 6 Feb 2002, Daniele del Re wrote: > > > > > I looked at the TS files for run1 and I discovered that some duplicated > > are present already at TS file level. > > The effect seems to be around 0.5% of the events for Dstar0, 0.3% for Dc, > > no effect for Dstar and D0. > > Actually it is possible that the TS were produced running twice on the > > same hbook but in prinicple the Riccardo script had to remove them when > > collections were produced. > > Anyway the effect seems to be small and the new reduced collections we are > > preparing (including also the run1) should be unaffected by this > > duplication problem. > > > > Daniele > > > > On Wed, 6 Feb 2002, Alessio Sarti wrote: > > > > > Hi all, > > > some BAD news on the timestamp side. > > > Working hard on cutting the duplicates I've always assumed that among the > > > same mode (& superblock) there are not duplicated events. > > > Checking on the timestamp provided by CPFramework (but it was a very quick > > > check) I had validated this assumption. > > > Now the problem comes out again: > > > running on Dstar0 (and now I'm trying to check also the other seeds) > > > reduced entuples I found that in the SAME REDUCED ROOT FILE and NTUPLE > > > there are duplicated events. > > > The checked file is : > > > > > > /u/ec/ursl/d/output/breco-112101/data/ > > > dstar0_110901_superblock2_2000_b1-s2_aa.root (.hbook) > > > > > > I have ran CPF and anaRecoil on this file and I got some duplicated > > > events: > > > first chunk > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E2D/C03C52A3:P run=00012318 > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E38/AB59C9FB:J run=00012318 > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E43/CE1A9DE3:J run=00012318 > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E06/4A2C19EF:S run=00012318 > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E0F/47D4E057:S run=00012318 > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E31/5FE2AB1B:K run=00012318 > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E13/A575A2C3:S run=00012318 > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E30/8B7862E3:J run=00012318 > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E30/AFA6889B:M run=00012318 > > > second chunk > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E2D/C03C52A3:P run=00012318 > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E38/AB59C9FB:J run=00012318 > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E43/CE1A9DE3:J run=00012318 > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E06/4A2C19EF:S run=00012318 > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E0F/47D4E057:S run=00012318 > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E31/5FE2AB1B:K run=00012318 > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E13/A575A2C3:S run=00012318 > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E30/8B7862E3:J run=00012318 > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E30/AFA6889B:M run=00012318 > > > > > > It is really difficult to find those events because they are replicated in > > > the same order but in really different parts of the output files > > > > > > (from the first to the second chunk I got ~5000 events....) > > > > > > We really have to find out how this is possible.... Now we are using > > > ntuples that may contain duplicated SIGNAL events. > > > > > > I'm going to quantify the problem and to check it among the various seeds > > > but I think that is a very hard work. > > > > > > Does anybody has an idea on how this can happen? > > > Cheers, > > > Alessio > > > > > > ______________________________________________________ > > > Alessio Sarti > > > Universita' & I.N.F.N. Ferrara > > > tel +39-0532-781928 Ferrara > > > > > > "Quod non fecerunt barbari, fecerunt Berlusconi" > > > > > > "Che il bianco sia bianco, e che il nero sia nero > > > che uno e uno fanno due e che la scienza dice il vero.... > > > DIPENDE !" > > > > > > > > > > > > >