Hi Alex,
The current xrootd does not provide support for archive extraction.
However, it is not difficult to add using the “frm” interface. The idea here is
that one can treat an archive as external storage. A request for a file in the
archive that hasn’t been extracted yet could trigger a trip through the frm
which would then extract the file and place it on disk so you can actually use
it. Of course, this present clean-up issues which would need to be addresed.
Should such a mechanism be provided, it would likely be done my opening the
file
<fname_in_archive?oss.arch=<archive_name>
which is fairly straightforward.
The EOS team would likely be the ones to provide such a feature as it
appears that is where most of these concerns come from.
Andy
Sent: Monday, November 26, 2012 8:41 AM
Subject: issue potentially related to XRootD usage
Hi,
In
ATLAS TDAQ we have the following problem: there is a huge number of small files
(~5 million) that we are storing in EOS as compressed tar archives. Then to read
a single small file we have to bring back from EOS an entire archive file which
can be up to 6 GBs. This is very inneficient and takes a lot of time. Is there a
way to programatically extract just the small file we need from the archive
directly on the EOS side? Or do you know a file format which can be used instead
of tar that will return us only a subset of data from EOS, not only a complete
file? What can the XrootD API give to us?
Cheers,
Alex
----
Alexandru Dan Sicoe
MEng, Research Fellow
CERN,
PH-ADT
Office: 40-5-D01
Phone: +41 (0) 22 76
71113
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-L list, click the following
link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
