 |
 |
> > Actually since we are running many CPF processes in parallel it is > impossible to remove duplicates if you read the same hbook file or the > same run in two different jobs. > About the creation of the reduced collections you mean you are using the > same tcl file we used to produce hbook files. If any run is present in two > different tcl's it produces for sure duplicated events, correct? yep ric > > Daniele > > On Wed, 6 Feb 2002, Riccardo Faccini wrote: > > > Hi, > > one of things Cecilia noticed in using CPFramework is that the duplicate > > removal seems not to be turned on by default (I did not remember this). Tu > > turn it on you should toggle cpf/nomulti. > > As far as the reduced collection creation the way it works is that for > > each event we check if it is in the list. The only way to get duplicates > > is to run twice on the same event AND to have the TS twice in the files. > > > > Of course I cannot completely exclude an operational error such as filling > > twice the same reduced collection. > > It might be worthwhile making a passa at removing duplicates from the > > reduced collections I am doing right now > > ciao > > ric > > > > > > > > On Wed, 6 Feb 2002, Daniele del Re wrote: > > > > > > > > I looked at the TS files for run1 and I discovered that some duplicated > > > are present already at TS file level. > > > The effect seems to be around 0.5% of the events for Dstar0, 0.3% for Dc, > > > no effect for Dstar and D0. > > > Actually it is possible that the TS were produced running twice on the > > > same hbook but in prinicple the Riccardo script had to remove them when > > > collections were produced. > > > Anyway the effect seems to be small and the new reduced collections we are > > > preparing (including also the run1) should be unaffected by this > > > duplication problem. > > > > > > Daniele > > > > > > On Wed, 6 Feb 2002, Alessio Sarti wrote: > > > > > > > Hi all, > > > > some BAD news on the timestamp side. > > > > Working hard on cutting the duplicates I've always assumed that among the > > > > same mode (& superblock) there are not duplicated events. > > > > Checking on the timestamp provided by CPFramework (but it was a very quick > > > > check) I had validated this assumption. > > > > Now the problem comes out again: > > > > running on Dstar0 (and now I'm trying to check also the other seeds) > > > > reduced entuples I found that in the SAME REDUCED ROOT FILE and NTUPLE > > > > there are duplicated events. > > > > The checked file is : > > > > > > > > /u/ec/ursl/d/output/breco-112101/data/ > > > > dstar0_110901_superblock2_2000_b1-s2_aa.root (.hbook) > > > > > > > > I have ran CPF and anaRecoil on this file and I got some duplicated > > > > events: > > > > first chunk > > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E2D/C03C52A3:P run=00012318 > > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E38/AB59C9FB:J run=00012318 > > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E43/CE1A9DE3:J run=00012318 > > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E06/4A2C19EF:S run=00012318 > > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E0F/47D4E057:S run=00012318 > > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E31/5FE2AB1B:K run=00012318 > > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E13/A575A2C3:S run=00012318 > > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E30/8B7862E3:J run=00012318 > > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E30/AFA6889B:M run=00012318 > > > > second chunk > > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E2D/C03C52A3:P run=00012318 > > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E38/AB59C9FB:J run=00012318 > > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E43/CE1A9DE3:J run=00012318 > > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E06/4A2C19EF:S run=00012318 > > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E0F/47D4E057:S run=00012318 > > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E31/5FE2AB1B:K run=00012318 > > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E13/A575A2C3:S run=00012318 > > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E30/8B7862E3:J run=00012318 > > > > CPHbook/BCH_3_stprova.ts_3:7F:7FFFFF:00154E30/AFA6889B:M run=00012318 > > > > > > > > It is really difficult to find those events because they are replicated in > > > > the same order but in really different parts of the output files > > > > > > > > (from the first to the second chunk I got ~5000 events....) > > > > > > > > We really have to find out how this is possible.... Now we are using > > > > ntuples that may contain duplicated SIGNAL events. > > > > > > > > I'm going to quantify the problem and to check it among the various seeds > > > > but I think that is a very hard work. > > > > > > > > Does anybody has an idea on how this can happen? > > > > Cheers, > > > > Alessio > > > > > > > > ______________________________________________________ > > > > Alessio Sarti > > > > Universita' & I.N.F.N. Ferrara > > > > tel +39-0532-781928 Ferrara > > > > > > > > "Quod non fecerunt barbari, fecerunt Berlusconi" > > > > > > > > "Che il bianco sia bianco, e che il nero sia nero > > > > che uno e uno fanno due e che la scienza dice il vero.... > > > > DIPENDE !" > > > > > > > > > > > > > > > > > > > > > >