Hi Patrick,
oss.space public /data/* directive is used to point where to hold the data files.
/atlas/local/* contains symlinks to the data files.
I removed the cms.space directive from the configuration file without success.
Also SELinux is running in permissive mode so I do not think that that is the issue.
Doug
On 03/25/2015 04:41 PM, Patrick McGuigan wrote:
> Hi Doug,
>
> My suspicion:
>
> 1) I don't see where /data/* is supporting anything under the /atlas namespace, so I won't look at that.
>
> 2) You are specifying that xrootd keep a minimum free space of 10GB and you only have 7.7GB free under /atlas. Hence
> it won't write there.
>
> Patrick
>
> On 03/25/2015 04:32 PM, Doug Benjamin wrote:
>> Hi,
>>
>> I having a heck of a time trying to debug this problem.
>>
>> I have a stand alone file server with two partitions /data/1 and /data/2 that
>> are glued together with xrootd
>>
>> Here is the xrootd config file important bits -
>>
>> all.adminpath /var/spool/xrootd/admin
>> all.export /atlas r/w
>> oss.space public /data/*
>> xrootd.chksum max 3 adler32 /usr/bin/xrdadler32
>> xrootd.seclib /usr/lib64/libXrdSec.so
>> sec.protocol /usr/lib64 unix
>> acc.authdb /etc/xrootd/auth_file
>> ofs.authorize
>> cms.space min 10g 15g
>> xrootd.trace all
>>
>>
>>
>> Both of the data partitions have sufficient space and inodes free.
>>
>> [root@atlasfs ~]# df -h /data/*
>> Filesystem Size Used Avail Use% Mounted on
>> /dev/mapper/vgdata1-lvdata1
>> 19T 17T 2.2T 89% /data/1
>> /dev/mapper/vgdata2-lvdata2
>> 19T 16T 3.0T 84% /data/2
>> [root@atlasfs ~]# df -ih /data/*
>> Filesystem Inodes IUsed IFree IUse% Mounted on
>> /dev/mapper/vgdata1-lvdata1
>> 3.7G 211K 3.7G 1% /data/1
>> /dev/mapper/vgdata2-lvdata2
>> 3.7G 165K 3.7G 1% /data/2
>>
>> The exported area /atlas is on a disk with sufficient space and inodes :
>>
>>
>> [root@atlasfs ~]# df -ih /atlas
>> Filesystem Inodes IUsed IFree IUse% Mounted on
>> /dev/mapper/sysvg-LVsys
>> 1.6M 639K 926K 41% /
>> [root@atlasfs ~]# df -h /atlas
>> Filesystem Size Used Avail Use% Mounted on
>> /dev/mapper/sysvg-LVsys
>> 24G 16G 7.7G 67% /
>>
>>
>> Yet the xrootd data server is reporting no space left on the device.
>>
>>
>> 150325 16:04:16 10920 ?:7@atlas28 XrootdProtocol: 0000 req=3007 dlen=71
>> 150325 16:04:16 10920 dbenjami.9782:7@atlas28 XrootdResponse: 0000 sending 23
>> data bytes; status=0
>> 150325 16:04:16 10920 dbenjami.9782:7@atlas28 XrootdProtocol: 0000 req=3000 dlen=21
>> 150325 16:04:16 10920 dbenjami.9782:7@atlas28 XrootdResponse: 0000 sending OK
>> 150325 16:04:16 10920 XrootdXeq: dbenjami.9782:7@atlas28 pub IPv4 login as
>> dbenjamin
>> 150325 16:04:16 10920 dbenjami.9782:7@atlas28 XrootdProtocol: 0100 req=3017 dlen=39
>> 150325 16:04:16 10920 dbenjami.9782:7@atlas28 XrootdProtocol: 0100 rc=0 stat
>> /atlas/local/dbenjamin/xroot_copy_test/
>> 150325 16:04:16 10920 dbenjami.9782:7@atlas28 XrootdResponse: 0100 sending 36
>> data bytes
>> 150325 16:04:16 10920 dbenjami.9782:7@atlas28 XrootdProtocol: 0100 req=3010 dlen=94
>> 150325 16:04:16 10920 dbenjami.9782:7@atlas28 XrootdProtocol: 0100 open unmat
>> /atlas/local/dbenjamin/xroot_copy_test//NTUP_SMWZ.01122074._000001.root.1?oss.asize=3303014400
>>
>> 150325 16:04:16 10920 ofs_open: dbenjami.9782:7@atlas28 Unable to create
>> /atlas/local/dbenjamin/xroot_copy_test/NTUP_SMWZ.01122074._000001.root.1; no
>> space left on device
>> 150325 16:04:16 10920 dbenjami.9782:7@atlas28 XrootdResponse: 0100 sending err
>> 3009: Unable to create
>> /atlas/local/dbenjamin/xroot_copy_test/NTUP_SMWZ.01122074._000001.root.1; no
>> space left on device
>> 150325 16:04:16 10920 XrootdXeq: dbenjami.9782:7@atlas28 disc 0:00:00
>> 150325 16:09:19 10950 ?:18@atlas17 XrootdProtocol: 0000 req=3007 dlen=84
>> 150325 16:09:19 10950 xrootd.7213:18@atlas17 XrootdResponse: 0000 sending 23
>> data bytes; status=0
>> 150325 16:09:19 10950 xrootd.7213:18@atlas17 XrootdProtocol: 0000 req=3000 dlen=18
>> 150325 16:09:19 10950 xrootd.7213:18@atlas17 XrootdResponse: 0000 sending OK
>>
>> Also using xrd -
>>
>> [dbenjamin@atlas28 bnl_openstack]$ xrd atlasfs.hep.anl.gov statvfs /atlas/local
>> r/w nodes: 1 r/w free space: 3064292 r/w utilization: 0 staging nodes: 1 staging
>> free space: 3064292 staging utilization: 0
>>
>> [dbenjamin@atlas28 bnl_openstack]$ xrd atlasfs.hep.anl.gov queryspace /atlas/local
>> Disk space approximations (MB):
>> Total : 38139916
>> Free : 5324910
>> Used : 0
>> Largest chunk : 3064292
>>
>>
>>
>> What am I missing?
>>
>> Thanks,
>>
>> Doug Benjamin
>>
>> ########################################################################
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the XROOTD-L list, click the following link:
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
>
> ########################################################################
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the XROOTD-L list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
|