Hi Tim,
> 1) Your version of LCIOWriter works fine. The only problem
> is that we need new constructors in
> LCIODriver to pass the ignore list to LCIOWriter.
The extra constructors are now added.
> 2) LCSimLoop has one very minor problem and one major
> problem. The minor problem is that the skip method in
> LCSimLoop should be skip(long nskip) in order to be
> consistent with the long argument of
> SequentialRecordLoopImpl.loop(long n) .
> Then the loop variable in skip should also be long. In
> addition, with this change you need to cast the nskip
> variable to int for the rss.goToRecord call.
Fixed.
> The major problem is that LCSimLoop is unacceptably
> inefficient when you have to skip large numbers of records.
> It is necessary to skip large numbers of records, for
> example, when you want to use the first half of files for
> training a neural net and the second half for doing analysis
> with the trained neural net.
> With my version of LCSimLoop it takes 130 seconds of "don"
> cputime to skip 120000 ZHH events; with your version a medium
> queue job on "don" hit its time limit of 1000 seconds before
> it had finished skipping the same 120000 ZHH events.
I was not able to reproduce this. I see about 1ms/event skip time, which is consistent with your time. I have now put a small improvement in which speeds this up to about .5ms/event. (It should be possible to speed this up still further but I did not have time yet).
Can you point me to your data file? Perhaps there is some other change between the older code you were using and the current CVS code?
Tony
|