On Tuesday, June 19, 2012 09:03:48 AM Brian Bockelman wrote:
> Certainly... but I can't provide any guidance if it's not in the git repo.
> One particular limitation in the existing client I'd like to see solved is
> scatter-gather I/O.
We will put it in for the September release. In the mean time I will give
you a github link, I just need to put all this licensing info (LGPL) in before
I make it public.
> > need to rewrite the ROOT plugin for the new client from scratch anyways,
> > so perhaps your stuff could be generalized and put into ROOT?
>
> It's not TXNetFile specifically, but the ROOT TFile infrastructure as a
> whole is starting to become so far behind CMS I/O, that it would be a
> fairly large project.
>
> The things we have added:
> 1) Exceptions. We provide informative, useful C++ exceptions on failures.
> For example, we propagate the server which caused the failure upward,
> eventually to the user. 2) Enforcing invariants. Our TFileAdaptor layer
> allows us to enforce invariants (such as "no operations on a closed file")
> that must be re-implemented in each TFile plugin. This particular
> invariant is not true in TXNetFile, for example. 3) Statistics gathering.
> ROOT's performance statistics are not thread-safe, bug-ridden, and/or
> inconsistently implemented. We implement more thorough statistics at the
> wrapper layer, meaning it's only implemented once (as opposed to in every
> plugin). 4) Scatter-gather I/O. We have vector reads into non-contiguous
> buffers (in everything except the Xrootd plugin, unfortunately), which
> allows for read coalescing. 5) Monitoring integration. We pass monitoring
> information from the client to the server; in case of failed jobs, we can
> trace back 6) Lazy-download. Through our wrapper layer, we have
> implemented local caching. This is used select workflows, including
> fast-merging (where TTreeCloner I/O is known to be very poor) and for
> generator files (where each job will need a copy of a specific small file
> for several files; on dCache, an open file handle is quite expensive).
Thanks for the info! ROOT's IO infrastructure clearly needs a rewrite.
Cheers,
Lukasz
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
|