Print

Print


Dear Expert:

After checking the log I found that two disks (localdisk3 and localdisk4) on valtical00 were not taken into account when xrootd starts up:

Config using configuration file /opt/vdt-xrootd/xrootd/etc/xrootd.cfg
=====> all.adminpath /opt/vdt-xrootd/xrootd/var/admin
=====> xrd.port 1094
Config maximum number of connections restricted to 1024
Copr. 2007 Stanford University/SLAC cmsd.
++++++ [log in to unmask] phase 1 initialization started.
=====> all.export /localdisk/xrootd
=====> all.export /localdisk/proofbox r/o
=====> all.adminpath /opt/vdt-xrootd/xrootd/var/admin
=====> all.manager valtical.cern.ch:1213
=====> all.role server
=====> oss.cache public /localdisk/xrootd/*
=====> oss.cache public /localdisk2/xrootd
=====> oss.cache public /localdisk5/xrootd
=====> oss.cache public /localdisk6/xrootd
The following paths are available to the redirector:
r /localdisk/proofbox
w /localdisk/xrootd

------ [log in to unmask] phase 1 server initialization completed.
++++++ [log in to unmask] phase 2 server initialization started.
130118 15:26:12 001 Meter: Duplicate filesystem /localdisk/xrootd/proof skipped for free space.
130118 15:26:12 001 Meter: Duplicate filesystem /localdisk/xrootd/productions skipped for free space.
130118 15:26:12 001 Meter: Duplicate filesystem /localdisk/xrootd/users skipped for free space.
130118 15:26:12 001 Meter: Duplicate filesystem /localdisk/xrootd/test skipped for free space.
130118 15:26:12 001 Meter: Found 4 filesystem(s); 7TB total (0% util); 1TB free (1TB max)


But actually localdisk3 and localdisk4 are configured in the configuration file /opt/vdt-xrootd/xrootd/etc/xrootd.cfg:

all.role server
xrootd.chksum max 3 adler32 ${xrootdlocation}/bin/xrdadler32
cms.perf int 2m pgm ${xrootdlocation}/etc/XrdOlbMonPerf 120
set osscachepath = /localdisk/xrootd
oss.cache public $(osscachepath)/* xa
oss.cache public /localdisk2/xrootd xa
oss.cache public /localdisk4/xrootd xa
oss.cache public /localdisk3/xrootd xa
oss.cache public /localdisk5/xrootd xa
oss.cache public /localdisk6/xrootd xa
oss.usage log ${xrootdlocation}/var/admin

any idea ?

Cheers,Gang

On 01/18/2013 12:35 PM, qing wrote:
[log in to unmask]" type="cite">Dear Expert:

   Today when I try to upload some files, the following error appeared:

root://valtical.cern.ch//localdisk/xrootd/test1.root

Last server error 3011 ('No servers are available to write the file, however, on one file server there are still ~ 5TB free space.

valtical.cern.ch is the xrdr, while valtical00 is one of the disk servers which has a lot of free space, the other disk servers are all full at this time.

Here is the xrootd configuration file on valtical00:
------------------------------------------------------------------------------------------------------------------------

xrootd.fslib ${xrootdlocation}/lib/libXrdOfs.so

if $(xrdr)
        all.role manager

        # CNSD_XRDR_BEGIN
        # broadcast mv, rm, rmdir and trunc operations to all data servers, and
        # redirect the client to CNS (the second xrootd instance on redirector)
        # to do the operation on shadow file/dir.
        ofs.forward 3way $(xrdr):2094 mv rm rmdir trunc
        # CNSD_XRDR_END
else

        all.role server
        xrootd.chksum max 3 adler32 ${xrootdlocation}/bin/xrdadler32
        cms.perf int 2m pgm ${xrootdlocation}/etc/XrdOlbMonPerf 120
        set osscachepath = /localdisk/xrootd
        oss.cache public $(osscachepath)/* xa
        oss.cache public /localdisk2/xrootd xa
        oss.cache public /localdisk3/xrootd xa
        oss.cache public /localdisk4/xrootd xa
        oss.cache public /localdisk5/xrootd xa
        oss.cache public /localdisk6/xrootd xa
        oss.usage log ${xrootdlocation}/var/admin

        # CNSD_BEGIN
        ofs.notify create closew mkdir | ${xrootdlocation}/bin/XrdCnsd -d -l ${xrootdlocation}/var/logs/cnsd.log root://$(xrdr):2094
        ofs.notifymsg create $TID create $FMODE $LFN?$CGI
        ofs.notifymsg closew $TID closew $LFN $FSIZE
        # CNSD_END
--------------------------------------------------------------------------------------------------------------------------------------

As you see, on valtical00 we still have 3 disks empty
[valtical00] /afs/cern.ch/user/q/qing > df -h | grep localdisk
/dev/sdb1             1.8T  1.7T     0 100% /localdisk
/dev/sdc1             1.8T  1.7T     0 100% /localdisk2
/dev/sdd1             1.8T  192G  1.6T  11% /localdisk3
/dev/sde1             1.8T   43G  1.7T   3% /localdisk4
/dev/sdg1             1.8T  1.7T     0 100% /localdisk5
/dev/sdh1             1.8T  196M  1.7T   1% /localdisk6

  Any idea what can be done to use these 3 disks?

  Cheers,Gang







Use REPLY-ALL to reply to list

To unsubscribe from the XROOTD-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1