URL:
<http://savannah.cern.ch/bugs/?81584>
Summary: XrdCnsd looses connection and stop exporting log
files.
Project: XROOTD
Submitted by: abh
Submitted on: 2011-04-27 22:35
Severity: 4 - Important
Priority: 7 - High
Status: None
Privacy: Public
Assigned to: abh
Originator Email:
Open/Closed: Open
Discussion Lock: Any
Fixed by commit(s):
_______________________________________________________
Details:
Here is a history of the issue...
Thanks Alex for shedding more light on this. We’ve been having problem off
and on and now it looks like the real underlying problem is the way XrdCnsd
uses the xrootd client to copy files into the redirector. One idea we’ve
been considering is to get rid of that part altogther and just let you
specify how the copy-in is to be done. That way, you can use you favorite
program (scp, xrdcp, cp to nfs, etc). This would solve the underlying
problem. Additionally, if you choose anything other than xrdcp you would not
need to run an extra xrootd on the redirector node.
Andy
From: Alex Bogert
Sent: Tuesday, April 26, 2011 11:32 AM
Subject: Xrootd CNSD bugged
Hi Everyone,
I recently found an error in our xrootd setup. The CNS daemons are no longer
reporting correctly to the redirector's inventory. When I noticed files
missing in the Inventory I rebooted the xrootd system, and saw some strange
behavior. Each of the worker nodes connect's to the redirector and begins
updating their inventory, but after about 100 - 200 updates, the local
XrdCNSd reports a trashed connection to the redirector. Also, If i disable
xrootdfs about 100 extra updates will get through before the connection is
trashed. I am stumped as to the cause of this problem. I've attached the logs
from the redirector and from one worker node after a restart.
I would appreciate any advice on how to resolve this.
Cheers,
Alex
Current xroot processes running on redirector:
$ ps aux | grep xroot
xrootd 8975 0.0 0.0 45424 4104 ? Sl Apr22 0:00
/opt/osg-v1.2/xrootd/bin//xrootd -l
/opt/osg-v1.2/xrootd/var/logs/xrdlog.atlas01.ucsc.edu -c
/opt/osg-v1.2/xrootd/etc/xrootd.cfg
xrootd 8976 0.0 0.0 41084 3856 ? Sl Apr22 0:00
/opt/osg-v1.2/xrootd/bin//xrootd -n cns -l
/opt/osg-v1.2/xrootd/var/logs/xrdlog.atlas01.ucsc.edu -c
/opt/osg-v1.2/xrootd/etc/xrootd.cfg
xrootd 9060 0.0 0.0 121032 4236 ? Sl Apr22 0:00
/opt/osg-v1.2/xrootd/bin//cmsd -l
/opt/osg-v1.2/xrootd/var/logs/cmslog.atlas01.ucsc.edu -c
/opt/osg-v1.2/xrootd/etc/xrootd.cfg
xrootd 9206 0.0 0.0 368432 12228 ? Ssl Apr22 0:01
/opt/osg-v1.2/xrootdfs/bin/xrootdfsd /xrootdfs/atlas -o
allow_other,fsname=xrootdfs,max_write=131072,attr_timeout=10,entry_timeout=10
Current xroot processing on worker (other workers look the same).
$ ps aux | grep xroot
1060 14469 0.0 0.0 61148 760 pts/1 S+ 11:19 0:00 grep xroot
xrootd 19697 0.0 0.0 49684 4412 ? Sl Apr22 0:00
/opt/osg-v1.2/xrootd/bin//xrootd -l
/opt/osg-v1.2/xrootd/var/logs/xrdlog.wrk2prv.ucsc.edu -c
/opt/osg-v1.2/xrootd/etc/xrootd.cfg
xrootd 19716 0.0 0.0 123740 12672 ? Sl Apr22 0:00
/opt/osg-v1.2/xrootd/bin/XrdCnsd -d -D 2 -i 90 -b
atlas01.ucsc.edu:1095:/atlas/inventory
xrootd 19778 0.0 0.0 46020 3864 ? Sl Apr22 0:00
/opt/osg-v1.2/xrootd/bin//cmsd -l
/opt/osg-v1.2/xrootd/var/logs/cmslog.wrk2prv.ucsc.edu -c
/opt/osg-v1.2/xrootd/etc/xrootd.cfg
_______________________________________________________
Reply to this item at:
<http://savannah.cern.ch/bugs/?81584>
_______________________________________________
Message sent via/by LCG Savannah
http://savannah.cern.ch/
