 |
 |
On Tuesday, June 19, 2012 09:03:48 AM Brian Bockelman wrote: > Certainly... but I can't provide any guidance if it's not in the git repo. > One particular limitation in the existing client I'd like to see solved is > scatter-gather I/O. We will put it in for the September release. In the mean time I will give you a github link, I just need to put all this licensing info (LGPL) in before I make it public. > > need to rewrite the ROOT plugin for the new client from scratch anyways, > > so perhaps your stuff could be generalized and put into ROOT? > > It's not TXNetFile specifically, but the ROOT TFile infrastructure as a > whole is starting to become so far behind CMS I/O, that it would be a > fairly large project. > > The things we have added: > 1) Exceptions. We provide informative, useful C++ exceptions on failures. > For example, we propagate the server which caused the failure upward, > eventually to the user. 2) Enforcing invariants. Our TFileAdaptor layer > allows us to enforce invariants (such as "no operations on a closed file") > that must be re-implemented in each TFile plugin. This particular > invariant is not true in TXNetFile, for example. 3) Statistics gathering. > ROOT's performance statistics are not thread-safe, bug-ridden, and/or > inconsistently implemented. We implement more thorough statistics at the > wrapper layer, meaning it's only implemented once (as opposed to in every > plugin). 4) Scatter-gather I/O. We have vector reads into non-contiguous > buffers (in everything except the Xrootd plugin, unfortunately), which > allows for read coalescing. 5) Monitoring integration. We pass monitoring > information from the client to the server; in case of failed jobs, we can > trace back 6) Lazy-download. Through our wrapper layer, we have > implemented local caching. This is used select workflows, including > fast-merging (where TTreeCloner I/O is known to be very poor) and for > generator files (where each job will need a copy of a specific small file > for several files; on dCache, an open file handle is quite expensive). Thanks for the info! ROOT's IO infrastructure clearly needs a rewrite. Cheers, Lukasz ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1