 |
 |
On Jun 19, 2012, at 8:49 AM, Lukasz Janyst wrote: > Hi Brian, > > I'm finishing up the copy command and still need to implement failure > recovery (going back to the redirectors on failure), otherwise it works well > enough so that we started to experiment with putting it into EOS. > Certainly... but I can't provide any guidance if it's not in the git repo. One particular limitation in the existing client I'd like to see solved is scatter-gather I/O. > In any case. Is there any specific reason why you have this adaptor? We will > need to rewrite the ROOT plugin for the new client from scratch anyways, so > perhaps your stuff could be generalized and put into ROOT? > It's not TXNetFile specifically, but the ROOT TFile infrastructure as a whole is starting to become so far behind CMS I/O, that it would be a fairly large project. The things we have added: 1) Exceptions. We provide informative, useful C++ exceptions on failures. For example, we propagate the server which caused the failure upward, eventually to the user. 2) Enforcing invariants. Our TFileAdaptor layer allows us to enforce invariants (such as "no operations on a closed file") that must be re-implemented in each TFile plugin. This particular invariant is not true in TXNetFile, for example. 3) Statistics gathering. ROOT's performance statistics are not thread-safe, bug-ridden, and/or inconsistently implemented. We implement more thorough statistics at the wrapper layer, meaning it's only implemented once (as opposed to in every plugin). 4) Scatter-gather I/O. We have vector reads into non-contiguous buffers (in everything except the Xrootd plugin, unfortunately), which allows for read coalescing. 5) Monitoring integration. We pass monitoring information from the client to the server; in case of failed jobs, we can trace back 6) Lazy-download. Through our wrapper layer, we have implemented local caching. This is used select workflows, including fast-merging (where TTreeCloner I/O is known to be very poor) and for generator files (where each job will need a copy of a specific small file for several files; on dCache, an open file handle is quite expensive). Thanks! Brian ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1