 |
 |
Hi Patrick, On Fri, 10 Mar 2017, Patrick McGuigan wrote: > We normally mount four partitions as /xrd_cache_[1-4]. This dataserver was > incorrectly setup and /xrd_cache_1 and /xrd_cache_4 are mounting the same > device! OK, that just means the partition for cache_1 and cache_4 were used twice as often than would be expected. > I think I need to turn off xrootd, unmount the disks, clean up /etc/fstab, > remount the disks, modify the links under /xrd to replace references to > /xrd_cache_4 with /xrd_cache_1 and then restart xrootd. Indeed the symlink needs to change all 4's need to become 1's. > Is the naming structure guaranteed to be the same under different partitions > to the name of the file?: > > for instance: > /xrd/datadisk/some/path is a link to /xrd_cache_4/ATLASDATADISK/xx/HASH% > > > If the file should be on /xrd_cache_1, would xx and HASH% be same regardless > of partition? Well, you are in luck here because the cache file paths were identical in length. So, you should be able to simply change the symlink and not bother with anything in the cache. > > > I am debating about allowing some access (read, maybe delete) to xrootd while > I am renaming links since this would allow the correct two partitions to > operate. I don't immediately see how. Once you separate the caches, any reference to a cache_4 file will fail because cache_4 is now empty, right? As I see it, all files in cache_4 also exist in cache_1. So, you should be able to rename the links to point to their Cache_1 image. Then fix up the mount point after that's done. You can allow read access during this. If you also want to have write access then remove cache_4 from the list of available spaces (i.e. specify cache_1, 2, and 3 explicitly -- don't use an asterisk. Make sense? Andy > > > Patrick > > On 03/10/2017 02:48 PM, Andrew Hanushevsky wrote: >> Hi Patrick, >> >> Hmm, well audit space does check that the files are actually linked via the >> name >> space. So, if it tells you all of tehm are reachable then they should be. >> On the >> other hand it also said, if I am not mistaken, that >> the space had no files in it (which I find hard to beleive). So, something >> else >> must be going on. You can rerrun audit space but specify -d (debug) on the >> command line to see what it actually is doing. >> >> Andy >> >> >> On Fri, 10 Mar 2017, Yang, Wei wrote: >> >>> HI Patrick, >>> >>> I think the best way is to rebuild the symlinks in name space (/xrd). For >>> each >>> file in /xrd_cache_*, you can find what its symlinks should be by doing >>> something like this: >>> >>> $ getfattr -d >>> //atlas/xrdcache/sdb/ATLASDATADISK/22/CB0C6157C00C00000000ac172c8922E9010014D% >>> getfattr: Removing leading '/' from absolute path names >>> # file: >>> atlas/xrdcache/sdb/ATLASDATADISK/22/CB0C6157C00C00000000ac172c8922E9010014D% >>> user.XrdCks.adler32=0sYWRsZXIzMgAAAAAAAAAAAAAAAABXRH7/ADNKMQAAAATRL9rKAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA >>> >>> user.XrdFrm.Pfn="/atlas/xrootd/atlasdatadisk/rucio/mc15_5TeV/97/7d/log.08435138._004167.job.log.tgz.1" >>> >>> >>> After you rebuild the symlinks in name space, you can then decide which >>> files >>> are not needed (or rely on the RUCIO team to delete for you). >>> >>> regards, >>> -- >>> Wei Yang | [log in to unmask] | 650-926-3338 >>> >>> ________________________________________ >>> From: [log in to unmask] <[log in to unmask]> on behalf of >>> Patrick McGuigan <[log in to unmask]> >>> Sent: Friday, March 10, 2017 12:06 PM >>> To: xrootd-l >>> Subject: audit with frm_admin >>> >>> Hi, >>> >>> I have an odd situation with a dataserver that uses XA space names and how >>> space >>> usage is being reported. >>> >>> I export an area named /xrd >>> >>> We are not using mig or stage options on the export. >>> >>> >>> I have a named space: >>> oss.space ATLASDATADISK /xrd_cache_* >>> >>> >>> >>> At the system level, only files written to /xrd/datadisk should be in the >>> ATLASDATADISK space. >>> >>> I have a script that walks the exported directory to discover the files >>> being >>> stored on the server and record date and size information: >>> >>> find -L /xrd -type f -fprintf /tmp/xrd_files '%p %T@ %s\n' >>> >>> >>> If I run 'frm_admin -c /etc/xrootd/xrootd-clustered.cfg audit usage' I get >>> amounts for claimed and actual bytes used by the various named spaces >>> including >>> ATLASDATADISK. >>> >>> The problem is that the amount claimed in the audit output is 20TB (33%) >>> larger >>> than what I expect by walking the path /xrd/datadisk. >>> >>> It looks like there are files in /xrd_cache_x/ATLASDATDISK/... that do not >>> have >>> links in /xrd/datadisk. >>> >>> What is the correct way to rectify this issue? >>> >>> I tried to run >>> >>> frm_admin -c /etc/xrootd/xrootd-clustered.cfg audit space ATLASDATADISK >>> >>> 0 problems found; 0 fixed. >>> Space ATLASDATADISK has 0 files with 0 bytes in use (0 unreachable). >>> >>> >>> >>> Should I run an audit names? >>> >>> Should the audit invocation look like: >>> ... audit name /xrd/datadisk >>> ? >>> >>> >>> Thanks for any insights, >>> >>> Patrick >>> >>> ######################################################################## >>> Use REPLY-ALL to reply to list >>> >>> To unsubscribe from the XROOTD-L list, click the following link: >>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 >>> ######################################################################## >>> Use REPLY-ALL to reply to list >>> >>> To unsubscribe from the XROOTD-L list, click the following link: >>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 >>> >> >> ######################################################################## >> Use REPLY-ALL to reply to list >> >> To unsubscribe from the XROOTD-L list, click the following link: >> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 > ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1