 |
 |
Hi Alex, you can use ZIP format without compression ... this allows to extract single pieces without reading a whole file. If the files are in ROOT format, ROOT supports ZIP archives, if not you can access the extracted files with zip + FUSE mount. Cheers Andreas. On Mon, Nov 26, 2012 at 5:41 PM, Alexandru Dan Sicoe < [log in to unmask]> wrote: > Hi, > > In ATLAS TDAQ we have the following problem: there is a huge number of > small files (~5 million) that we are storing in EOS as compressed tar > archives. Then to read a single small file we have to bring back from EOS > an entire archive file which can be up to 6 GBs. This is very inneficient > and takes a lot of time. Is there a way to programatically extract just the > small file we need from the archive directly on the EOS side? Or do you > know a file format which can be used instead of tar that will return us > only a subset of data from EOS, not only a complete file? What can the > XrootD API give to us? > > Cheers, > Alex > > ---- > Alexandru Dan Sicoe > MEng, Research Fellow > CERN, PH-ADT > Office: 40-5-D01 > Phone: +41 (0) 22 76 71113 > > ------------------------------ > > Use REPLY-ALL to reply to list > > To unsubscribe from the XROOTD-L list, click the following link: > https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 > ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1