 |
 |
Hi, Thanks for the information. I built something that would audit the links vs the files and discovered the source of the problem. I am not sure what the solution entails yet. We normally mount four partitions as /xrd_cache_[1-4]. This dataserver was incorrectly setup and /xrd_cache_1 and /xrd_cache_4 are mounting the same device! I think I need to turn off xrootd, unmount the disks, clean up /etc/fstab, remount the disks, modify the links under /xrd to replace references to /xrd_cache_4 with /xrd_cache_1 and then restart xrootd. Is the naming structure guaranteed to be the same under different partitions to the name of the file?: for instance: /xrd/datadisk/some/path is a link to /xrd_cache_4/ATLASDATADISK/xx/HASH% If the file should be on /xrd_cache_1, would xx and HASH% be same regardless of partition? I am debating about allowing some access (read, maybe delete) to xrootd while I am renaming links since this would allow the correct two partitions to operate. Patrick On 03/10/2017 02:48 PM, Andrew Hanushevsky wrote: > Hi Patrick, > > Hmm, well audit space does check that the files are actually linked via the name > space. So, if it tells you all of tehm are reachable then they should be. On the > other hand it also said, if I am not mistaken, that > the space had no files in it (which I find hard to beleive). So, something else > must be going on. You can rerrun audit space but specify -d (debug) on the > command line to see what it actually is doing. > > Andy > > > On Fri, 10 Mar 2017, Yang, Wei wrote: > >> HI Patrick, >> >> I think the best way is to rebuild the symlinks in name space (/xrd). For each >> file in /xrd_cache_*, you can find what its symlinks should be by doing >> something like this: >> >> $ getfattr -d >> //atlas/xrdcache/sdb/ATLASDATADISK/22/CB0C6157C00C00000000ac172c8922E9010014D% >> getfattr: Removing leading '/' from absolute path names >> # file: >> atlas/xrdcache/sdb/ATLASDATADISK/22/CB0C6157C00C00000000ac172c8922E9010014D% >> user.XrdCks.adler32=0sYWRsZXIzMgAAAAAAAAAAAAAAAABXRH7/ADNKMQAAAATRL9rKAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA >> >> user.XrdFrm.Pfn="/atlas/xrootd/atlasdatadisk/rucio/mc15_5TeV/97/7d/log.08435138._004167.job.log.tgz.1" >> >> >> After you rebuild the symlinks in name space, you can then decide which files >> are not needed (or rely on the RUCIO team to delete for you). >> >> regards, >> -- >> Wei Yang | [log in to unmask] | 650-926-3338 >> >> ________________________________________ >> From: [log in to unmask] <[log in to unmask]> on behalf of >> Patrick McGuigan <[log in to unmask]> >> Sent: Friday, March 10, 2017 12:06 PM >> To: xrootd-l >> Subject: audit with frm_admin >> >> Hi, >> >> I have an odd situation with a dataserver that uses XA space names and how space >> usage is being reported. >> >> I export an area named /xrd >> >> We are not using mig or stage options on the export. >> >> >> I have a named space: >> oss.space ATLASDATADISK /xrd_cache_* >> >> >> >> At the system level, only files written to /xrd/datadisk should be in the >> ATLASDATADISK space. >> >> I have a script that walks the exported directory to discover the files being >> stored on the server and record date and size information: >> >> find -L /xrd -type f -fprintf /tmp/xrd_files '%p %T@ %s\n' >> >> >> If I run 'frm_admin -c /etc/xrootd/xrootd-clustered.cfg audit usage' I get >> amounts for claimed and actual bytes used by the various named spaces including >> ATLASDATADISK. >> >> The problem is that the amount claimed in the audit output is 20TB (33%) larger >> than what I expect by walking the path /xrd/datadisk. >> >> It looks like there are files in /xrd_cache_x/ATLASDATDISK/... that do not have >> links in /xrd/datadisk. >> >> What is the correct way to rectify this issue? >> >> I tried to run >> >> frm_admin -c /etc/xrootd/xrootd-clustered.cfg audit space ATLASDATADISK >> >> 0 problems found; 0 fixed. >> Space ATLASDATADISK has 0 files with 0 bytes in use (0 unreachable). >> >> >> >> Should I run an audit names? >> >> Should the audit invocation look like: >> ... audit name /xrd/datadisk >> ? >> >> >> Thanks for any insights, >> >> Patrick >> >> ######################################################################## >> Use REPLY-ALL to reply to list >> >> To unsubscribe from the XROOTD-L list, click the following link: >> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 >> ######################################################################## >> Use REPLY-ALL to reply to list >> >> To unsubscribe from the XROOTD-L list, click the following link: >> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 >> > > ######################################################################## > Use REPLY-ALL to reply to list > > To unsubscribe from the XROOTD-L list, click the following link: > https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1