Hello,

I hope this is the right place for this issue. If not, could you please direct me to a more suitable address for my issue?

I am testing the xrootd network and the basics work fine but I am not able to list or locate data with xrdfs when using a clustered disc-caching-proxy in front of the xrootd data cluster.
When I use

xrdfs xroot://192.168.1.186:1094//xroot://192.168.1.103:1094/ ls /

it hangs and when I use

xrdfs xroot://192.168.1.186:1094//xroot://192.168.1.103:1094/ locate -r -d /testfile

I get

[ERROR] Server responded with an error: [3011] No servers have access to the file

xrdcp on the other hand works fine and I am able to copy the file. Also using the xrdfs ls and locate commands work fine when talking directly to my data redirector.

I am working on ubuntu and installed the packages by using this repository

wget -O - https://storage-ci.web.cern.ch/storage-ci/storageci.key | sudo apt-key add - 
sudo add-apt-repository 'deb https://storage-ci.web.cern.ch/storage-ci/debian/xrootd bionic master'
My configuration Cache redirector 
all.manager 192.168.1.186:3121
all.role manager
cms.dfs lookup distrib mdhold 20m redirect immed
if exec cmsd
  xrd.port 3121
fi
if exec xrootd
  xrd.port 1094
fi


cms.sched maxretries 0 nomultisrc space 98 fuzz 5 affinity weak
cms.fxhold noloc 15m 4h

set rootdir = /home/ubuntu/data
all.export / cache
oss.localroot $(rootdir)
xrd.network keepalive kaparms 10m,1m,5
xrd.timeout idle 60m
all.adminpath /var/spool/xrootd
all.pidpath /run/xrootd
xrd.trace all -debug
xrootd.trace all -debug
ofs.trace all -debug
cms.trace all -debug
oss.trace all -debug

Cache server:

set rootdir = /home/ubuntu/data
all.export / cache
all.manager 192.168.1.186:3121
oss.localroot $(rootdir)
all.adminpath /var/spool/xrootd
all.pidpath /run/xrootd
cms.dfs lookup distrib mdhold 20m redirect immed

if exec cmsd
    all.role server
fi
if exec xrootd
    all.role server
    set eBLOCKSIZE=1M
    set ePREFETCH=20
    set eRAMSIZE=2g
    set eSPACELOWWM=0.90
    set eSPACEHIGHWM=0.95


    pss.origin = xroot://192.168.1.103:1094

    ofs.osslib  libXrdPss.so
    pss.cachelib libXrdFileCache.so
    xrd.network keepalive kaparms 10m,1m,5
    xrd.timeout idle 60m
    pfc.blocksize $eBLOCKSIZE
    pfc.ram $eRAMSIZE
    pfc.prefetch $ePREFETCH
    pfc.diskusage $eSPACELOWWM $eSPACEHIGHWM files 15g 16g 17g purgeinterval 60s
fi
xrd.trace all -debug
xrootd.trace all -debug
ofs.trace all -debug
cms.trace all -debug
oss.trace all -debug

Data Redirector:

all.export / r/o nostage
all.manager 192.168.1.103:3121
all.role manager
cms.dfs lookup distrib mdhold 20m redirect immed
set rootdir = /home/ubuntu/data
oss.localroot $(rootdir)
if exec xrootd
  xrd.port 1094
fi
if exec cmsd
  xrd.port 3121
fi
xrd.trace all -debug
xrootd.trace all -debug
ofs.trace all -debug
cms.trace all -debug
oss.trace all -debug
all.adminpath /var/spool/xrootd
all.pidpath /run/xrootd

Data Server:

all.export /
set xrdr = 192.168.1.103
all.manager $(xrdr):3121
all.role server
cms.space min 2g 5g
all.adminpath /var/spool/xrootd
all.pidpath /run/xrootd
set rootdir = /vol
oss.localroot $(rootdir)
xrd.trace all -debug
xrootd.trace all -debug
ofs.trace all -debug
cms.trace all -debug
oss.trace all -debug
Log output Cache redirector 
211203 19:40:12 1200 Xrd_Sched: running underused thread monitor inq=0
211203 19:40:12 1200 Xrd_Sched: 3 threads; 1 idle
211203 19:40:12 1200 Xrd_Sched: scheduling underused thread monitor in 780 seconds
211203 19:40:36 1300 Starting on Linux 4.15.0-163-generic
211203 19:40:36 1300 /usr/bin/xrootd -b -l /var/log/xrootd/xrootd.log -c /etc/xrootd/xrootd-cache.cfg -k fifo -s /run/xrootd/xrootd-cache.pid -n cache
Copr.  2004-2012 Stanford University, xrd version v20211203-ffc4c6b67
Config warning: this hostname, cachered, is registered without a domain qualification.
++++++ xrootd cache@cachered initialization started.
Config using configuration file /etc/xrootd/xrootd-cache.cfg
=====> xrd.port 1094
=====> xrd.network keepalive kaparms 10m,1m,5
=====> xrd.timeout idle 60m
=====> all.adminpath /var/spool/xrootd
=====> all.pidpath /run/xrootd
=====> xrd.trace all -debug
Config maximum number of connections restricted to 1048576
211203 19:40:36 1300 Xrd_Config: sendfile enabled.
211203 19:40:36 1300 Xrd_Sched: scheduling underused thread monitor in 780 seconds
211203 19:40:36 1300 Xrd_Sched: Starting with 2 workers
211203 19:40:36 1300 Xrd_Sched: scheduling Idle link scan in 1200 seconds
211203 19:40:36 1300 Xrd_Poll: Starting poller 0
211203 19:40:36 1300 Xrd_Poll: Starting poller 1
211203 19:40:36 1300 Xrd_Poll: Starting poller 2
211203 19:40:36 1300 Xrd_Config: xroot:1094 wsz=131072
Copr.  2012 Stanford University, xroot protocol 5.1.0 version v20211203-ffc4c6b67
++++++ xroot protocol initialization started.
=====> all.export / cache
=====> xrootd.trace all -debug
Config exporting /
Config warning: 'xrootd.seclib' not specified; strong authentication disabled!
Config Routing for 192.168.1.186: local pub4 prv4 
Config Route all4: 192.168.1.186 Dest=[::192.168.1.186]:1094
++++++ File system initialization started.
=====> all.role manager
=====> ofs.trace all -debug
++++++ Storage system initialization started.
=====> all.export / cache
=====> oss.localroot /home/ubuntu/data 
=====> oss.trace all -debug
Config effective /etc/xrootd/xrootd-cache.cfg oss configuration:
       oss.alloc        0 0 0
       oss.spacescan    600
       oss.fdlimit      524288 1048576
       oss.maxsize      0
       oss.localroot /home/ubuntu/data
       oss.trace        7ff
       oss.xfr          1 deny 10800 keep 1200
       oss.memfile off  max 2068011008
       oss.defaults  r/w nocheck nodread nomig nopurge norcreate nostage
       oss.path / r/o cache
------ Storage system initialization completed.
++++++ Configuring manager role. . .
=====> all.manager 192.168.1.186:3121
=====> all.adminpath /var/spool/xrootd
=====> cms.trace all -debug
Config 1 manager(s) started.
Config effective /etc/xrootd/xrootd-cache.cfg ofs configuration:
       all.role manager
       ofs.maxdelay   60
       ofs.persist    manual hold 600 logdir /var/spool/xrootd/cache/.ofs/posc.log
       ofs.trace      7fff
------ File system manager initialization completed.
Config asynchronous I/O has been disabled!
211203 19:40:36 1300 Xrd_Sched: scheduling xroot protocol anchor in 3600 seconds
211203 19:40:36 1300 Xrd_Sched: scheduling transit protocol anchor in 3600 seconds
211203 19:40:36 1300  ofs_FAttr: FAttr req=info
------ xroot protocol initialization completed.
211203 19:40:36 1312 XrdOpen: Unable to connect socket to cachered; connection refused
------ xrootd cache@cachered:1094 initialization completed.
211203 19:40:36 1305 Xrd_Sched: running main accept inq=0
211203 19:40:40 1312 Xrd_Inet: Connected to cachered:3121
211203 19:40:40 1312 cms_ClientMan: Connected to suspended cachered v 3
211203 19:42:06 1312 cms_setStatus: Manager cachered resumed
211203 19:42:57 1304 Xrd_Sched: Now have 3 workers
211203 19:42:57 1304 Xrd_Sched: running main accept inq=0
211203 19:42:57 1305 Xrd_Inet: Accepted connection on port 1094 from 7@[::ffff:192.168.1.149]
211203 19:42:57 1305 anon.0:7@[::ffff:192.168.1.149] Xrd_Poll: FD 7 attached to poller 0; num=1
211203 19:42:57 1305 anon.0:7@[::ffff:192.168.1.149] Xrootd_Protocol: 0000 req=protocol dlen=0
211203 19:42:57 1305 anon.0:7@[::ffff:192.168.1.149] Xrootd_Response: 0000 sending 8 data bytes; status=0
211203 19:42:57 1305 anon.0:7@[::ffff:192.168.1.149] Xrootd_Protocol: 0000 req=login dlen=100
211203 19:42:57 1305 ubuntu.1219:7@[::ffff:192.168.1.149] Xrootd_Response: 0000 sending 16 data bytes
211203 19:42:57 1305 XrootdXeq: ubuntu.1219:7@[::ffff:192.168.1.149] pvt IPv4 login
211203 19:42:57 1305 ubuntu.1219:7@[::ffff:192.168.1.149] Xrootd_Protocol: 0100 req=stat dlen=1
211203 19:42:57 1305 ubuntu.1219:7@[::ffff:192.168.1.149] ofs_stat:  fn=/
211203 19:42:57 1312 cms_Decode: cachered redirects ubuntu.1219:7@[::ffff:192.168.1.149] to 192.168.1.54:1094 /
211203 19:42:57 1305 ubuntu.1219:7@[::ffff:192.168.1.149] Xrootd_Protocol: 0100 rc=-256 stat /
211203 19:42:57 1305 ubuntu.1219:7@[::ffff:192.168.1.149] Xrootd_Protocol: 0100 redirecting to 192.168.1.54:1094
211203 19:42:57 1305 ubuntu.1219:7@[::ffff:192.168.1.149] Xrootd_Response: 0100 sending 16 data bytes; status=4004
211203 19:42:57 1305 ubuntu.1219:7@[::ffff:192.168.1.149] Xrootd_Protocol: 0100 req=locate dlen=2
211203 19:42:57 1305 ubuntu.1219:7@[::ffff:192.168.1.149] Xrootd_Protocol: 0100 locate nuD */
211203 19:42:57 1305 ubuntu.1219:7@[::ffff:192.168.1.149] ofs_fsctl: locate args=*/
211203 19:42:57 1312 cms_Decode: cachered sent ubuntu.1219:7@[::ffff:192.168.1.149] 'Sr[::192.168.1.83]:1094 Sr[::192.168.1.54]:1094' */
211203 19:42:57 1305 ubuntu.1219:7@[::ffff:192.168.1.149] Xrootd_Protocol: 0100 rc=-1024 locate */
211203 19:42:57 1305 ubuntu.1219:7@[::ffff:192.168.1.149] Xrootd_Response: 0100 sending 48 data bytes
211203 19:43:00 1305 ubuntu.1219:7@[::ffff:192.168.1.149] Xrootd_Protocol: 0100 request timeout; read 0 of 24 bytes
211203 19:43:00 1305 Xrd_Poll: Poller 0 enabled ubuntu.1219:7@[::ffff:192.168.1.149]
211203 19:43:39 1306 ubuntu.1219:7@[::ffff:192.168.1.149] Xrd_Poll: Link 7 terminating: hangup
211203 19:43:39 1344 Xrd_Sched: running ubuntu.1219:7@[::ffff:192.168.1.149] inq=0
211203 19:43:39 1344 XrootdXeq: ubuntu.1219:7@[::ffff:192.168.1.149] disc 0:00:42
211203 19:43:39 1344 ubuntu.1219:7@[::ffff:192.168.1.149] Xrd_Poll: Poller 0 removing FD 7
211203 19:43:39 1344 ubuntu.1219:7@[::ffff:192.168.1.149] Xrd_Poll: FD 7 detached from poller 0; num=0
211203 19:43:52 1304 Xrd_Inet: Accepted connection on port 1094 from 22@[::ffff:192.168.1.149]
211203 19:43:52 1304 anon.0:22@[::ffff:192.168.1.149] Xrd_Poll: FD 22 attached to poller 0; num=1
211203 19:43:52 1304 anon.0:22@[::ffff:192.168.1.149] Xrootd_Protocol: 0000 req=protocol dlen=0
211203 19:43:52 1304 anon.0:22@[::ffff:192.168.1.149] Xrootd_Response: 0000 sending 8 data bytes; status=0
211203 19:43:52 1305 Xrd_Sched: running main accept inq=0
211203 19:43:52 1304 anon.0:22@[::ffff:192.168.1.149] Xrootd_Protocol: 0000 req=login dlen=100
211203 19:43:52 1304 ubuntu.1225:22@[::ffff:192.168.1.149] Xrootd_Response: 0000 sending 16 data bytes
211203 19:43:52 1304 XrootdXeq: ubuntu.1225:22@[::ffff:192.168.1.149] pvt IPv4 login
211203 19:43:52 1304 ubuntu.1225:22@[::ffff:192.168.1.149] Xrootd_Protocol: 0100 req=locate dlen=9
211203 19:43:52 1304 ubuntu.1225:22@[::ffff:192.168.1.149] Xrootd_Protocol: 0100 locate s /testfile
211203 19:43:52 1304 ubuntu.1225:22@[::ffff:192.168.1.149] ofs_fsctl: locate args=/testfile
211203 19:43:52 1312 cms_Decode: cachered delays ubuntu.1225:22@[::ffff:192.168.1.149] 5 /testfile
211203 19:43:52 1304 ubuntu.1225:22@[::ffff:192.168.1.149] Xrootd_Protocol: 0100 rc=5 locate /testfile
211203 19:43:52 1304 ubuntu.1225:22@[::ffff:192.168.1.149] Xrootd_Protocol: 0100 stalling client for 5 sec
211203 19:43:52 1304 ubuntu.1225:22@[::ffff:192.168.1.149] Xrootd_Response: 0100 sending 4 data bytes; status=4005
211203 19:43:55 1304 ubuntu.1225:22@[::ffff:192.168.1.149] Xrootd_Protocol: 0100 request timeout; read 0 of 24 bytes
211203 19:43:55 1304 Xrd_Poll: Poller 0 enabled ubuntu.1225:22@[::ffff:192.168.1.149]
211203 19:43:57 1344 Xrd_Sched: running ubuntu.1225:22@[::ffff:192.168.1.149] inq=0
211203 19:43:57 1344 ubuntu.1225:22@[::ffff:192.168.1.149] Xrootd_Protocol: 0100 req=locate dlen=9
211203 19:43:57 1344 ubuntu.1225:22@[::ffff:192.168.1.149] Xrootd_Protocol: 0100 locate  /testfile
211203 19:43:57 1344 ubuntu.1225:22@[::ffff:192.168.1.149] ofs_fsctl: locate args=/testfile
211203 19:43:57 1312 cms_Decode: cachered gave ubuntu.1225:22@[::ffff:192.168.1.149] err -2 'No servers have access to the file' /testfile
211203 19:43:57 1344 ubuntu.1225:22@[::ffff:192.168.1.149] Xrootd_Protocol: 0100 rc=-1 locate /testfile
211203 19:43:57 1344 ubuntu.1225:22@[::ffff:192.168.1.149] Xrootd_Response: 0100 sending err 3011: No servers have access to the file
211203 19:43:57 1344 XrootdXeq: ubuntu.1225:22@[::ffff:192.168.1.149] disc 0:00:05
211203 19:43:57 1344 ubuntu.1225:22@[::ffff:192.168.1.149] Xrd_Poll: Poller 0 removing FD 22
211203 19:43:57 1344 ubuntu.1225:22@[::ffff:192.168.1.149] Xrd_Poll: FD 22 detached from poller 0; num=0

Cache server one:

211203 19:40:37 1360 Starting on Linux 4.15.0-163-generic
211203 19:40:37 1360 /usr/bin/xrootd -b -l /var/log/xrootd/xrootd.log -c /etc/xrootd/xrootd-cache.cfg -k fifo -s /run/xrootd/xrootd-cache.pid -n cache
Copr.  2004-2012 Stanford University, xrd version v20211203-ffc4c6b67
Config warning: this hostname, xcache-f1c1f, is registered without a domain qualification.
++++++ xrootd cache@xcache-f1c1f initialization started.
Config using configuration file /etc/xrootd/xrootd-cache.cfg
=====> all.adminpath /var/spool/xrootd
=====> all.pidpath /run/xrootd
=====> xrd.network keepalive kaparms 10m,1m,5
=====> xrd.timeout idle 60m
=====> xrd.trace all -debug
Config maximum number of connections restricted to 1048576
211203 19:40:37 1360 Xrd_Config: sendfile enabled.
211203 19:40:37 1360 Xrd_Sched: scheduling underused thread monitor in 780 seconds
211203 19:40:37 1360 Xrd_Sched: Starting with 2 workers
211203 19:40:37 1360 Xrd_Sched: scheduling Idle link scan in 1200 seconds
211203 19:40:37 1360 Xrd_Poll: Starting poller 0
211203 19:40:37 1360 Xrd_Poll: Starting poller 1
211203 19:40:37 1360 Xrd_Poll: Starting poller 2
211203 19:40:37 1360 Xrd_Config: xroot:1094 wsz=131072
Copr.  2012 Stanford University, xroot protocol 5.1.0 version v20211203-ffc4c6b67
++++++ xroot protocol initialization started.
=====> all.export / cache
=====> xrootd.trace all -debug
Config exporting /
Config warning: 'xrootd.seclib' not specified; strong authentication disabled!
Config Routing for 192.168.1.54: local pub4 prv4 
Config Route all4: 192.168.1.54 Dest=[::192.168.1.54]:1094
++++++ File system initialization started.
=====> all.role server
=====> ofs.osslib libXrdPss.so 
=====> ofs.trace all -debug
Copr.  2019, Stanford University, Pss Version v20211203-ffc4c6b67
=====> all.export / cache
=====> pss.origin = xroot://192.168.1.160:1094
Config warning: 'libXrdFileCache' has been replaced by 'libXrdPfc'; for future compatibility specify 'default' instead!
=====> pss.cachelib libXrdFileCache.so 
Plugin loaded unreleased XrdPfc v20211203-ffc4c6b67 from cachelib libXrdPfc-5.so
++++++ Proxy file cache initialization started.
=====> pfc.blocksize 1M
=====> pfc.ram 2g
=====> pfc.prefetch 20
=====> pfc.diskusage 0.90 0.95 files 15g 16g 17g purgeinterval 60s
++++++ Storage system initialization started.
=====> all.export / cache
=====> oss.localroot /home/ubuntu/data 
=====> oss.trace all -debug
++++++ Configuring pfc mode . . .
Config effective /etc/xrootd/xrootd-cache.cfg oss configuration:
       oss.alloc        0 0 0
       oss.spacescan    600
       oss.fdlimit      524288 1048576
       oss.maxsize      0
       oss.localroot /home/ubuntu/data
       oss.trace        7ff
       oss.xfr          1 deny 10800 keep 1200
       oss.memfile off  max 2068011008
       oss.defaults  r/w nocheck nodread nomig nopurge norcreate nostage
       oss.path / r/o cache
------ Storage system initialization completed.
Config effective /etc/xrootd/xrootd-cache.cfg pfc configuration:
       pfc.cschk nocache net notls uvkeep lru
       pfc.blocksize 1048576
       pfc.prefetch 20
       pfc.ram 2g
       pfc.writequeue 16 4
       # Total available disk: 20629221376
       pfc.diskusage 18566299238 19597760307 files 16106127360 17179869184 18253611008 purgeinterval 60 purgecoldfiles -1
       pfc.spaces public public
       pfc.trace 2
       pfc.flush 2000
       pfc.acchistorysize 20

Config Proxy File Cache g-stream has NOT been configured via xrootd.monitor directive
------ Proxy File Cache configuration parsing completed
------ Proxy file cache initialization completed.
------ Proxy storage system initialization completed.
++++++ Configuring server role. . .
=====> all.manager 192.168.1.186:3121
=====> all.adminpath /var/spool/xrootd
=====> cms.trace all -debug
Config POSC has been disabled by the osslib plugin.
Config effective /etc/xrootd/xrootd-cache.cfg ofs configuration:
       all.role server
       ofs.maxdelay   60
       ofs.persist    off hold 600
       ofs.trace      7fff
       ofs.osslib libXrdPss-5.so 
------ File system server initialization completed.
Config sendfile has been disabled by file system plugin.
211203 19:40:37 1360 Xrd_Sched: scheduling xroot protocol anchor in 3600 seconds
211203 19:40:37 1360 Xrd_Sched: scheduling transit protocol anchor in 3600 seconds
211203 19:40:37 1360  ofs_FAttr: FAttr req=info
------ xroot protocol initialization completed.
------ xrootd cache@xcache-f1c1f:1094 initialization completed.
211203 19:40:37 1364 Xrd_Sched: running main accept inq=0
211203 19:40:37 1380 cms_Open: Unable to connect socket to /var/spool/xrootd/cache/.olb/olbd.admin; connection refused
211203 19:40:38 1379 ubuntu oss_Opendir: lcl path /home/ubuntu/data/ (/)
211203 19:40:47 1380 cms_Finder: Connected to cmsd via /var/spool/xrootd/cache/.olb/olbd.admin
211203 19:42:57 1365 Xrd_Sched: Now have 3 workers
211203 19:42:57 1365 Xrd_Sched: running main accept inq=0
211203 19:42:57 1364 Xrd_Inet: Accepted connection on port 1094 from 7@[::ffff:192.168.1.149]
211203 19:42:57 1364 anon.0:7@[::ffff:192.168.1.149] Xrd_Poll: FD 7 attached to poller 0; num=1
211203 19:42:57 1364 anon.0:7@[::ffff:192.168.1.149] Xrootd_Protocol: 0000 req=protocol dlen=0
211203 19:42:57 1364 anon.0:7@[::ffff:192.168.1.149] Xrootd_Response: 0000 sending 8 data bytes; status=0
211203 19:42:57 1364 anon.0:7@[::ffff:192.168.1.149] Xrootd_Protocol: 0000 req=login dlen=100
211203 19:42:57 1364 ubuntu.1219:7@[::ffff:192.168.1.149] Xrootd_Response: 0000 sending 16 data bytes
211203 19:42:57 1364 XrootdXeq: ubuntu.1219:7@[::ffff:192.168.1.149] pvt IPv4 login
211203 19:42:57 1364 ubuntu.1219:7@[::ffff:192.168.1.149] Xrootd_Protocol: 0100 req=stat dlen=1
211203 19:42:57 1364 ubuntu.1219:7@[::ffff:192.168.1.149] ofs_stat:  fn=/
211203 19:42:57 1364 ubuntu.1219:7@[::ffff:192.168.1.149] Xrootd_Protocol: 0100 rc=0 stat /
211203 19:42:57 1364 ubuntu.1219:7@[::ffff:192.168.1.149] Xrootd_Response: 0100 sending 77 data bytes
211203 19:43:00 1364 ubuntu.1219:7@[::ffff:192.168.1.149] Xrootd_Protocol: 0100 request timeout; read 0 of 24 bytes
211203 19:43:00 1364 Xrd_Poll: Poller 0 enabled ubuntu.1219:7@[::ffff:192.168.1.149]
211203 19:43:39 1366 ubuntu.1219:7@[::ffff:192.168.1.149] Xrd_Poll: Link 7 terminating: hangup
211203 19:43:39 1407 Xrd_Sched: running ubuntu.1219:7@[::ffff:192.168.1.149] inq=0
211203 19:43:39 1407 XrootdXeq: ubuntu.1219:7@[::ffff:192.168.1.149] disc 0:00:42
211203 19:43:39 1407 ubuntu.1219:7@[::ffff:192.168.1.149] Xrd_Poll: Poller 0 removing FD 7
211203 19:43:39 1407 ubuntu.1219:7@[::ffff:192.168.1.149] Xrd_Poll: FD 7 detached from poller 0; num=0

Cache server two:

211203 19:40:37 1447 Starting on Linux 4.15.0-163-generic
211203 19:40:37 1447 /usr/bin/xrootd -b -l /var/log/xrootd/xrootd.log -c /etc/xrootd/xrootd-cache.cfg -k fifo -s /run/xrootd/xrootd-cache.pid -n cache
Copr.  2004-2012 Stanford University, xrd version v20211203-ffc4c6b67
Config warning: this hostname, xcache2-9e50f, is registered without a domain qualification.
++++++ xrootd cache@xcache2-9e50f initialization started.
Config using configuration file /etc/xrootd/xrootd-cache.cfg
=====> all.adminpath /var/spool/xrootd
=====> all.pidpath /run/xrootd
=====> xrd.network keepalive kaparms 10m,1m,5
=====> xrd.timeout idle 60m
=====> xrd.trace all -debug
Config maximum number of connections restricted to 1048576
211203 19:40:37 1447 Xrd_Config: sendfile enabled.
211203 19:40:37 1447 Xrd_Sched: scheduling underused thread monitor in 780 seconds
211203 19:40:37 1447 Xrd_Sched: Starting with 2 workers
211203 19:40:37 1447 Xrd_Sched: scheduling Idle link scan in 1200 seconds
211203 19:40:37 1447 Xrd_Poll: Starting poller 0
211203 19:40:37 1447 Xrd_Poll: Starting poller 1
211203 19:40:37 1447 Xrd_Poll: Starting poller 2
211203 19:40:37 1447 Xrd_Config: xroot:1094 wsz=131072
Copr.  2012 Stanford University, xroot protocol 5.1.0 version v20211203-ffc4c6b67
++++++ xroot protocol initialization started.
=====> all.export / cache
=====> xrootd.trace all -debug
Config exporting /
Config warning: 'xrootd.seclib' not specified; strong authentication disabled!
Config Routing for 192.168.1.83: local pub4 prv4 
Config Route all4: 192.168.1.83 Dest=[::192.168.1.83]:1094
++++++ File system initialization started.
=====> all.role server
=====> ofs.osslib libXrdPss.so 
=====> ofs.trace all -debug
Copr.  2019, Stanford University, Pss Version v20211203-ffc4c6b67
=====> all.export / cache
=====> pss.origin = xroot://192.168.1.160:1094
Config warning: 'libXrdFileCache' has been replaced by 'libXrdPfc'; for future compatibility specify 'default' instead!
=====> pss.cachelib libXrdFileCache.so 
Plugin loaded unreleased XrdPfc v20211203-ffc4c6b67 from cachelib libXrdPfc-5.so
++++++ Proxy file cache initialization started.
=====> pfc.blocksize 1M
=====> pfc.ram 2g
=====> pfc.prefetch 20
=====> pfc.diskusage 0.90 0.95 files 15g 16g 17g purgeinterval 60s
++++++ Storage system initialization started.
=====> all.export / cache
=====> oss.localroot /home/ubuntu/data 
=====> oss.trace all -debug
++++++ Configuring pfc mode . . .
Config effective /etc/xrootd/xrootd-cache.cfg oss configuration:
       oss.alloc        0 0 0
       oss.spacescan    600
       oss.fdlimit      524288 1048576
       oss.maxsize      0
       oss.localroot /home/ubuntu/data
       oss.trace        7ff
       oss.xfr          1 deny 10800 keep 1200
       oss.memfile off  max 2068011008
       oss.defaults  r/w nocheck nodread nomig nopurge norcreate nostage
       oss.path / r/o cache
------ Storage system initialization completed.
Config effective /etc/xrootd/xrootd-cache.cfg pfc configuration:
       pfc.cschk nocache net notls uvkeep lru
       pfc.blocksize 1048576
       pfc.prefetch 20
       pfc.ram 2g
       pfc.writequeue 16 4
       # Total available disk: 20629221376
       pfc.diskusage 18566299238 19597760307 files 16106127360 17179869184 18253611008 purgeinterval 60 purgecoldfiles -1
       pfc.spaces public public
       pfc.trace 2
       pfc.flush 2000
       pfc.acchistorysize 20

Config Proxy File Cache g-stream has NOT been configured via xrootd.monitor directive
------ Proxy File Cache configuration parsing completed
------ Proxy file cache initialization completed.
------ Proxy storage system initialization completed.
++++++ Configuring server role. . .
=====> all.manager 192.168.1.186:3121
=====> all.adminpath /var/spool/xrootd
=====> cms.trace all -debug
Config POSC has been disabled by the osslib plugin.
Config effective /etc/xrootd/xrootd-cache.cfg ofs configuration:
       all.role server
       ofs.maxdelay   60
       ofs.persist    off hold 600
       ofs.trace      7fff
       ofs.osslib libXrdPss-5.so 
------ File system server initialization completed.
Config sendfile has been disabled by file system plugin.
211203 19:40:38 1447 Xrd_Sched: scheduling xroot protocol anchor in 3600 seconds
211203 19:40:38 1447 Xrd_Sched: scheduling transit protocol anchor in 3600 seconds
211203 19:40:38 1447  ofs_FAttr: FAttr req=info
------ xroot protocol initialization completed.
------ xrootd cache@xcache2-9e50f:1094 initialization completed.
211203 19:40:38 1451 Xrd_Sched: running main accept inq=0
211203 19:40:38 1467 cms_Open: Unable to connect socket to /var/spool/xrootd/cache/.olb/olbd.admin; connection refused
211203 19:40:38 1466 ubuntu oss_Opendir: lcl path /home/ubuntu/data/ (/)
211203 19:40:48 1467 cms_Finder: Connected to cmsd via /var/spool/xrootd/cache/.olb/olbd.admin
211203 19:42:57 1452 Xrd_Sched: Now have 3 workers
211203 19:42:57 1452 Xrd_Sched: running main accept inq=0
211203 19:42:57 1451 Xrd_Inet: Accepted connection on port 1094 from 7@[::ffff:192.168.1.149]
211203 19:42:57 1451 anon.0:7@[::ffff:192.168.1.149] Xrd_Poll: FD 7 attached to poller 0; num=1
211203 19:42:57 1451 anon.0:7@[::ffff:192.168.1.149] Xrootd_Protocol: 0000 req=protocol dlen=0
211203 19:42:57 1451 anon.0:7@[::ffff:192.168.1.149] Xrootd_Response: 0000 sending 8 data bytes; status=0
211203 19:42:57 1451 anon.0:7@[::ffff:192.168.1.149] Xrootd_Protocol: 0000 req=login dlen=100
211203 19:42:57 1451 ubuntu.1219:7@[::ffff:192.168.1.149] Xrootd_Response: 0000 sending 16 data bytes
211203 19:42:57 1451 XrootdXeq: ubuntu.1219:7@[::ffff:192.168.1.149] pvt IPv4 login
211203 19:42:57 1451 ubuntu.1219:7@[::ffff:192.168.1.149] Xrootd_Protocol: 0100 req=dirlist dlen=1
211203 19:42:57 1451 ubuntu.1219:7@[::ffff:192.168.1.149] ofs_opendir:  fn=/

Data redirector:

211203 19:40:44 1313 Starting on Linux 4.15.0-163-generic
211203 19:40:44 1313 /usr/bin/xrootd -l /var/log/xrootd/xrootd.log -b -c /etc/xrootd/xrootd-clustered.cfg -k fifo -s /run/xrootd/xrootd-clustered.pid -n clustered
Copr.  2004-2012 Stanford University, xrd version v20211203-ffc4c6b67
Config warning: this hostname, xred-41f06, is registered without a domain qualification.
++++++ xrootd clustered@xred-41f06 initialization started.
Config using configuration file /etc/xrootd/xrootd-clustered.cfg
=====> xrd.port 1094
=====> xrd.trace all -debug
=====> all.adminpath /var/spool/xrootd
=====> all.pidpath /run/xrootd
Config maximum number of connections restricted to 1048576
211203 19:40:44 1313 Xrd_Config: sendfile enabled.
211203 19:40:44 1313 Xrd_Sched: scheduling underused thread monitor in 780 seconds
211203 19:40:44 1313 Xrd_Sched: Starting with 2 workers
211203 19:40:44 1313 Xrd_Poll: Starting poller 0
211203 19:40:44 1313 Xrd_Poll: Starting poller 1
211203 19:40:44 1313 Xrd_Poll: Starting poller 2
211203 19:40:44 1313 Xrd_Config: xroot:1094 wsz=131072
Copr.  2012 Stanford University, xroot protocol 5.1.0 version v20211203-ffc4c6b67
++++++ xroot protocol initialization started.
=====> all.export / r/o
=====> xrootd.trace all -debug
Config exporting /
Config warning: 'xrootd.seclib' not specified; strong authentication disabled!
Config Routing for 192.168.1.103: local pub4 prv4 
Config Route all4: 192.168.1.103 Dest=[::192.168.1.103]:1094
++++++ File system initialization started.
=====> all.role manager
=====> ofs.trace all -debug
++++++ Storage system initialization started.
=====> all.export / r/o nostage
=====> oss.localroot /home/ubuntu/data 
=====> oss.trace all -debug
Config effective /etc/xrootd/xrootd-clustered.cfg oss configuration:
       oss.alloc        0 0 0
       oss.spacescan    600
       oss.fdlimit      524288 1048576
       oss.maxsize      0
       oss.localroot /home/ubuntu/data
       oss.trace        7ff
       oss.xfr          1 deny 10800 keep 1200
       oss.memfile off  max 2068015104
       oss.defaults  r/w nocheck nodread nomig nopurge norcreate nostage
       oss.path / r/o nocheck nodread nomig nopurge norcreate nostage
------ Storage system initialization completed.
++++++ Configuring manager role. . .
=====> all.manager 192.168.1.103:3121
=====> cms.trace all -debug
=====> all.adminpath /var/spool/xrootd
Config 1 manager(s) started.
Config effective /etc/xrootd/xrootd-clustered.cfg ofs configuration:
       all.role manager
       ofs.maxdelay   60
       ofs.persist    manual hold 600 logdir /var/spool/xrootd/clustered/.ofs/posc.log
       ofs.trace      7fff
------ File system manager initialization completed.
Config asynchronous I/O has been disabled!
211203 19:40:44 1313 Xrd_Sched: scheduling xroot protocol anchor in 3600 seconds
211203 19:40:44 1313 Xrd_Sched: scheduling transit protocol anchor in 3600 seconds
211203 19:40:44 1313  ofs_FAttr: FAttr req=info
------ xroot protocol initialization completed.
------ xrootd clustered@xred-41f06:1094 initialization completed.
211203 19:40:44 1318 Xrd_Sched: running main accept inq=0
211203 19:40:44 1325 XrdOpen: Unable to connect socket to xred-41f06; connection refused
211203 19:40:48 1325 Xrd_Inet: Connected to xred-41f06:3121
211203 19:40:48 1325 cms_ClientMan: Connected to suspended xred-41f06 v 3
211203 19:42:14 1325 cms_setStatus: Manager xred-41f06 resumed

Data server:

211203 19:40:45 1389 Starting on Linux 4.15.0-163-generic
211203 19:40:45 1389 /usr/bin/xrootd -l /var/log/xrootd/xrootd.log -b -c /etc/xrootd/xrootd-clustered.cfg -k fifo -s /run/xrootd/xrootd-clustered.pid -n clustered
Copr.  2004-2012 Stanford University, xrd version v20211203-ffc4c6b67
Config warning: this hostname, xdata1-af575, is registered without a domain qualification.
++++++ xrootd clustered@xdata1-af575 initialization started.
Config using configuration file /etc/xrootd/xrootd-clustered.cfg
=====> all.adminpath /var/spool/xrootd
=====> all.pidpath /run/xrootd
=====> xrd.trace all -debug
Config maximum number of connections restricted to 1048576
211203 19:40:45 1389 Xrd_Config: sendfile enabled.
211203 19:40:45 1389 Xrd_Sched: scheduling underused thread monitor in 780 seconds
211203 19:40:45 1389 Xrd_Sched: Starting with 2 workers
211203 19:40:45 1389 Xrd_Poll: Starting poller 0
211203 19:40:45 1389 Xrd_Poll: Starting poller 1
211203 19:40:45 1389 Xrd_Poll: Starting poller 2
211203 19:40:45 1389 Xrd_Config: xroot:1094 wsz=131072
Copr.  2012 Stanford University, xroot protocol 5.1.0 version v20211203-ffc4c6b67
++++++ xroot protocol initialization started.
=====> all.export /
=====> xrootd.trace all -debug
Config exporting /
Config warning: 'xrootd.seclib' not specified; strong authentication disabled!
Config Routing for 192.168.1.231: local pub4 prv4 
Config Route all4: 192.168.1.231 Dest=[::192.168.1.231]:1094
++++++ File system initialization started.
=====> all.role server
=====> ofs.trace all -debug
++++++ Storage system initialization started.
=====> all.export /
=====> oss.localroot /vol 
=====> oss.trace all -debug
Config effective /etc/xrootd/xrootd-clustered.cfg oss configuration:
       oss.alloc        0 0 0
       oss.spacescan    600
       oss.fdlimit      524288 1048576
       oss.maxsize      0
       oss.localroot /vol
       oss.trace        7ff
       oss.xfr          1 deny 10800 keep 1200
       oss.memfile off  max 2068011008
       oss.defaults  r/w nocheck nodread nomig nopurge norcreate nostage
       oss.path / r/w nocheck nodread nomig nopurge norcreate nostage
------ Storage system initialization completed.
++++++ Configuring server role. . .
=====> all.manager 192.168.1.103:3121
=====> all.adminpath /var/spool/xrootd
=====> cms.trace all -debug
++++++ Checkpoint initialization started.
++++++ Checkpoint initialization completed.
Config effective /etc/xrootd/xrootd-clustered.cfg ofs configuration:
       all.role server
       ofs.maxdelay   60
       ofs.persist    manual hold 600 logdir /var/spool/xrootd/clustered/.ofs/posc.log
       ofs.trace      7fff
------ File system server initialization completed.
Config asynchronous I/O has been disabled!
211203 19:40:45 1389 Xrd_Sched: scheduling xroot protocol anchor in 3600 seconds
211203 19:40:45 1389 Xrd_Sched: scheduling transit protocol anchor in 3600 seconds
211203 19:40:45 1401 cms_Open: Unable to connect socket to /var/spool/xrootd/clustered/.olb/olbd.admin; connection refused
211203 19:40:45 1389  ofs_FAttr: FAttr req=info
------ xroot protocol initialization completed.
------ xrootd clustered@xdata1-af575:1094 initialization completed.
211203 19:40:45 1394 Xrd_Sched: running main accept inq=0
211203 19:40:55 1401 cms_Finder: Connected to cmsd via /var/spool/xrootd/clustered/.olb/olbd.admin

Thanks in advance and kind regards,
Ewgenij

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

[ { "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": "https://github.com/xrootd/xrootd/issues/1565", "url": "https://github.com/xrootd/xrootd/issues/1565", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

Use REPLY-ALL to reply to list

To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1