In ATLAS TDAQ we have the following problem: there is a huge number of small files (~5 million) that we are storing in EOS as compressed tar archives. Then to read a single small file we have to bring back from EOS an entire archive file which can be up to
6 GBs. This is very inneficient and takes a lot of time. Is there a way to programatically extract just the small file we need from the archive directly on the EOS side? Or do you know a file format which can be used instead of tar that will return us only
a subset of data from EOS, not only a complete file? What can the XrootD API give to us?
Alexandru Dan Sicoe
MEng, Research Fellow
Phone: +41 (0) 22 76 71113