Hello,
I've implemented a python os.walk style function using xrootd bindings in place of the local file system calls. One of the ingredients of this is testing whether or not a directory listing is a file or a folder. Initially I was using the dirlist command with the DirListFlags.STAT flag. However, for large directories (>30K files) this takes an exceedingly long time (even in the command line client version). Example commands for python can be in [1] and for the command line can be found in [2].
Another option is to get the dirlist without requesting the stat information and then to call stat on each file/folder individually. However, this means that the server will receive many requests quite rapidly. This could, in principle, cause server side issues.
Is there a recommendation on how to identify the directories and/or files in a dirlist without returning all of the stat information?
Thank you,
Alexx
[1]
from XRootD import client
from XRootD.client.flags import DirListFlags, StatInfoFlags
xrdfs = client.FileSystem(xrootd_endpoint)
status, listing = xrdfs.dirlist(directory,DirListFlags.STAT)
[2]
xrdfs root:/// ls -l
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1