Print

Print

Hello,

I've implemented a python os.walk style function using xrootd bindings in place of the local file system calls. One of the ingredients of this is testing whether or not a directory listing is a file or a folder. Initially I was using the dirlist command with the DirListFlags.STAT flag. However, for large directories (>30K files) this takes an exceedingly long time (even in the command line client version). Example commands for python can be in [1] and for the command line can be found in [2].

Another option is to get the dirlist without requesting the stat information and then to call stat on each file/folder individually. However, this means that the server will receive many requests quite rapidly. This could, in principle, cause server side issues.

Is there a recommendation on how to identify the directories and/or files in a dirlist without returning all of the stat information?

Thank you,
Alexx

[1]
from XRootD import client
from XRootD.client.flags import DirListFlags, StatInfoFlags
xrdfs = client.FileSystem(xrootd_endpoint)
status, listing = xrdfs.dirlist(directory,DirListFlags.STAT)

[2]
xrdfs root:/// ls -l

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/xrootd/xrootd","title":"xrootd/xrootd","subtitle":"GitHub repository","main_image_url":"https://github.githubassets.com/images/email/message_cards/header.png","avatar_image_url":"https://github.githubassets.com/images/email/message_cards/avatar.png","action":{"name":"Open in GitHub","url":"https://github.com/xrootd/xrootd"}},"updates":{"snippets":[{"icon":"DESCRIPTION","message":"Directory listing with the stat option takes a long time (#962)"}],"action":{"name":"View Issue","url":"https://github.com/xrootd/xrootd/issues/962"}}} [ { "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": "https://github.com/xrootd/xrootd/issues/962", "url": "https://github.com/xrootd/xrootd/issues/962", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

Use REPLY-ALL to reply to list

To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1