 |
 |
Hi Patrick, oss.space public /data/* directive is used to point where to hold the data files. /atlas/local/* contains symlinks to the data files. I removed the cms.space directive from the configuration file without success. Also SELinux is running in permissive mode so I do not think that that is the issue. Doug On 03/25/2015 04:41 PM, Patrick McGuigan wrote: > Hi Doug, > > My suspicion: > > 1) I don't see where /data/* is supporting anything under the /atlas namespace, so I won't look at that. > > 2) You are specifying that xrootd keep a minimum free space of 10GB and you only have 7.7GB free under /atlas. Hence > it won't write there. > > Patrick > > On 03/25/2015 04:32 PM, Doug Benjamin wrote: >> Hi, >> >> I having a heck of a time trying to debug this problem. >> >> I have a stand alone file server with two partitions /data/1 and /data/2 that >> are glued together with xrootd >> >> Here is the xrootd config file important bits - >> >> all.adminpath /var/spool/xrootd/admin >> all.export /atlas r/w >> oss.space public /data/* >> xrootd.chksum max 3 adler32 /usr/bin/xrdadler32 >> xrootd.seclib /usr/lib64/libXrdSec.so >> sec.protocol /usr/lib64 unix >> acc.authdb /etc/xrootd/auth_file >> ofs.authorize >> cms.space min 10g 15g >> xrootd.trace all >> >> >> >> Both of the data partitions have sufficient space and inodes free. >> >> [root@atlasfs ~]# df -h /data/* >> Filesystem Size Used Avail Use% Mounted on >> /dev/mapper/vgdata1-lvdata1 >> 19T 17T 2.2T 89% /data/1 >> /dev/mapper/vgdata2-lvdata2 >> 19T 16T 3.0T 84% /data/2 >> [root@atlasfs ~]# df -ih /data/* >> Filesystem Inodes IUsed IFree IUse% Mounted on >> /dev/mapper/vgdata1-lvdata1 >> 3.7G 211K 3.7G 1% /data/1 >> /dev/mapper/vgdata2-lvdata2 >> 3.7G 165K 3.7G 1% /data/2 >> >> The exported area /atlas is on a disk with sufficient space and inodes : >> >> >> [root@atlasfs ~]# df -ih /atlas >> Filesystem Inodes IUsed IFree IUse% Mounted on >> /dev/mapper/sysvg-LVsys >> 1.6M 639K 926K 41% / >> [root@atlasfs ~]# df -h /atlas >> Filesystem Size Used Avail Use% Mounted on >> /dev/mapper/sysvg-LVsys >> 24G 16G 7.7G 67% / >> >> >> Yet the xrootd data server is reporting no space left on the device. >> >> >> 150325 16:04:16 10920 ?:7@atlas28 XrootdProtocol: 0000 req=3007 dlen=71 >> 150325 16:04:16 10920 dbenjami.9782:7@atlas28 XrootdResponse: 0000 sending 23 >> data bytes; status=0 >> 150325 16:04:16 10920 dbenjami.9782:7@atlas28 XrootdProtocol: 0000 req=3000 dlen=21 >> 150325 16:04:16 10920 dbenjami.9782:7@atlas28 XrootdResponse: 0000 sending OK >> 150325 16:04:16 10920 XrootdXeq: dbenjami.9782:7@atlas28 pub IPv4 login as >> dbenjamin >> 150325 16:04:16 10920 dbenjami.9782:7@atlas28 XrootdProtocol: 0100 req=3017 dlen=39 >> 150325 16:04:16 10920 dbenjami.9782:7@atlas28 XrootdProtocol: 0100 rc=0 stat >> /atlas/local/dbenjamin/xroot_copy_test/ >> 150325 16:04:16 10920 dbenjami.9782:7@atlas28 XrootdResponse: 0100 sending 36 >> data bytes >> 150325 16:04:16 10920 dbenjami.9782:7@atlas28 XrootdProtocol: 0100 req=3010 dlen=94 >> 150325 16:04:16 10920 dbenjami.9782:7@atlas28 XrootdProtocol: 0100 open unmat >> /atlas/local/dbenjamin/xroot_copy_test//NTUP_SMWZ.01122074._000001.root.1?oss.asize=3303014400 >> >> 150325 16:04:16 10920 ofs_open: dbenjami.9782:7@atlas28 Unable to create >> /atlas/local/dbenjamin/xroot_copy_test/NTUP_SMWZ.01122074._000001.root.1; no >> space left on device >> 150325 16:04:16 10920 dbenjami.9782:7@atlas28 XrootdResponse: 0100 sending err >> 3009: Unable to create >> /atlas/local/dbenjamin/xroot_copy_test/NTUP_SMWZ.01122074._000001.root.1; no >> space left on device >> 150325 16:04:16 10920 XrootdXeq: dbenjami.9782:7@atlas28 disc 0:00:00 >> 150325 16:09:19 10950 ?:18@atlas17 XrootdProtocol: 0000 req=3007 dlen=84 >> 150325 16:09:19 10950 xrootd.7213:18@atlas17 XrootdResponse: 0000 sending 23 >> data bytes; status=0 >> 150325 16:09:19 10950 xrootd.7213:18@atlas17 XrootdProtocol: 0000 req=3000 dlen=18 >> 150325 16:09:19 10950 xrootd.7213:18@atlas17 XrootdResponse: 0000 sending OK >> >> Also using xrd - >> >> [dbenjamin@atlas28 bnl_openstack]$ xrd atlasfs.hep.anl.gov statvfs /atlas/local >> r/w nodes: 1 r/w free space: 3064292 r/w utilization: 0 staging nodes: 1 staging >> free space: 3064292 staging utilization: 0 >> >> [dbenjamin@atlas28 bnl_openstack]$ xrd atlasfs.hep.anl.gov queryspace /atlas/local >> Disk space approximations (MB): >> Total : 38139916 >> Free : 5324910 >> Used : 0 >> Largest chunk : 3064292 >> >> >> >> What am I missing? >> >> Thanks, >> >> Doug Benjamin >> >> ######################################################################## >> Use REPLY-ALL to reply to list >> >> To unsubscribe from the XROOTD-L list, click the following link: >> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 > > ######################################################################## > Use REPLY-ALL to reply to list > > To unsubscribe from the XROOTD-L list, click the following link: > https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1