Also make sure you have enough space to hold a core dump. It can something be 10GB+
--
Wei Yang | [log in to unmask] | 650-926-3338(O)
-----Original Message-----
From: <[log in to unmask]> on behalf of Horst Severini <[log in to unmask]>
Date: Saturday, March 21, 2020 at 1:30 PM
To: xrootd-dev <[log in to unmask]>, "[log in to unmask]" <[log in to unmask]>
Subject: Re: XrootD smoke test report for 2020-03-21 10:01:46 GMT
Thanks Wei,
I'll send you the next one I get! :)
Cheers,
Horst
On 3/21/20 2:55 PM, Yang, Wei wrote:
> Indeed a core dump is usually the thing we need.
>
> --
> Wei Yang [log in to unmask] | 650-926-3338(O)
>
> On 3/21/20, 12:19 PM,"[log in to unmask] on behalf of Horst Severini" <[log in to unmask] on behalf of [log in to unmask]> wrote:
>
> We're running 4.11.1 here at OU.
>
> Cheers,
>
> Horst
>
> Albert Rossi<[log in to unmask]> wrote:
>
> > Hi Horst,
> >
> > actually, if you notice, all endpoints failed on today's test. So it was not just OU.
> >
> > The Stanford developers may want you to run a few commands over the core file from gdb once you have it in hand.
> >
> > What version of xrootd are you running, just out of curiosity? Is it bleeding-edge, or a stable release?
> >
> > Cheers, Al
> >
> > ________________________________________________
> > Albert L. Rossi
> > Application Developer & Systems Analyst III
> > Scientific Computing Division, Data Movement Development
> > FCC 229A
> > Mail Station 369 (FCC 2W)
> > Fermi National Accelerator Laboratory
> > Batavia, IL 60510
> > (630) 840-3023
> > ________________________________
> > From: Horst Severini<[log in to unmask]>
> > Sent: Saturday, March 21, 2020 1:18 PM
> > To:[log in to unmask] <[log in to unmask]>;[log in to unmask] <[log in to unmask]>; Albert Rossi<[log in to unmask]>
> > Subject: Re: XrootD smoke test report for 2020-03-21 10:01:46 GMT
> >
> > Hi Al,
> >
> > thanks, good idea. I'll save the next core file.
> >
> > I'm pretty sure the authentication failures simply came because
> > that partition was full and no new proxies or what not could be
> > created, so I wouldn't worry about that.
> >
> > Thanks,
> >
> > Horst
> >
> > Albert Rossi<[log in to unmask]> wrote:
> >
> > > Hi Horst,
> > >
> > > I would definitely report this [log in to unmask]
> > >
> > > As for why the massive authentication failure, I've seen this before, it might have to do with CA cert issues.
> > >
> > > Cheers, Al
> > >
> > > ________________________________________________
> > > Albert L. Rossi
> > > Application Developer & Systems Analyst III
> > > Scientific Computing Division, Data Movement Development
> > > FCC 229A
> > > Mail Station 369 (FCC 2W)
> > > Fermi National Accelerator Laboratory
> > > Batavia, IL 60510
> > > (630) 840-3023
> > > ________________________________
> > > From: Horst Severini<[log in to unmask]>
> > > Sent: Saturday, March 21, 2020 11:19 AM
> > > To:[log in to unmask] <[log in to unmask]>
> > > Subject: Re: XrootD smoke test report for 2020-03-21 10:01:46 GMT
> > >
> > > Hi all,
> > >
> > > our/var/ partition had filled up because of too many xrootd core dumps.
> > > I cleared those up and restarted xrootd, and things look better again now.
> > >
> > > Not sure why we keep getting core dumps, though.
> > >
> > > Cheers,
> > >
> > > Horst
> > >
> > > On 3/21/20 5:01 AM,[log in to unmask] wrote:
> > > > XROOTD SMOKE TEST SUMMARY
> > > > 2020-03-21 10:01:46 GMT
> > > >
> > > > Client: bogus6.fnal.gov
> > > >
> > > > XrootD version: v4.11.2
> > > >
> > > > Reference server: CERN-TRUNK
> > > >
> > > > Credential delegation: ON
> > > >
> > > > Checksum: -C adler32
> > > >
> > > > Total number of round-trip tests: 21
> > > >
> > > > --------------------------------SOUND ENDPOINTS---------------------------------
> > > >
> > > > SCORE ENDPT TYPE UP SRC DST DN
> > > > --------------------------------------------------------------------------------
> > > >
> > > > -----------------------------PROBLEMATIC ENDPOINTS------------------------------
> > > >
> > > > SCORE ENDPT TYPE UP SRC DST DN
> > > > --------------------------------------------------------------------------------
> > > > 19 BRUSSELS dCache F - F F 0/4
> > > > 19 CERN-EOS EOS F - F F 0/4
> > > > 19 CERN-TRUNK DPM - F - F 0/2
> > > > 19 DESY-PROM dCache F - F F 0/4
> > > > 19 FNAL dCache F - F F 0/4
> > > > 19 IN2P3-DOMA xrootd F - F F 0/4
> > > > 19 PRAGUE DPM F - F F 0/4
> > > > 19 RAL-CEPH CEPH F - F F 0/4
> > > > 19 RAL-LCG2 Echo F - F F 0/4
> > > > 19 SLAC XrootD F - F F 0/4
> > > > 19 TRIUMF dCache F - F F 0/4
> > > > 19 UKI-BRUNEL DPM F - F F 0/4
> > > > 19 UKI-LANC DPM F - F F 0/4
> > > > 19 UKI-MAN1 DPM F - F F 0/4
> > > > 19 UKI-MAN2 DPM F - F F 0/4
> > > > 19 UNI-BONN CephFS F - F F 0/4
> > > > 18 OU XrootD F - F F 0/4
> > > > 13 BNL dCache F - F F 0/4
> > > > 7 AGLT2 dCache F - F F 0/4
> > > > 0 CALTECH HDFS F - F F 0/4
> > > > 0 TRIUMF-PROD dCache F - F F 0/4
> > > >
> > > > --------------------------------ERROR DETAILS-----------------------------------
> > > >
> > > > [1] BRUSSELS
> > > > UPLOAD (round-trip-15-upload): 000186 : [2020-03-21 05:01:06.216990 -0500][Error ][XRootD ] [maite.iihe.ac.be:1094] Unable to send the message kXR_open (file: pnfs/iihe//dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?oss.asize=1048576, mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat ): [FATAL] Auth failed
> > > > TPC_DST_D (round-trip-15-tpc-dst-d): 000195 : [2020-03-21 05:01:06.719178 -0500][Error ][Utility ] Unable to open target root://maite.iihe.ac.be:1094/pnfs/iihe//dteam/from-CERN-TRUNK-21d897f6-51dd-4cc6-9535-58a414108e04-D?oss.asize=0&tpc.dlg=maite.iihe.ac.be:1094&tpc.dlgon=1&tpc.key=247f6a10a6b5a9305e75e5e2&tpc.lfn=pnfs/iihe//dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04&tpc.scgi=tpc.stage=placement&tpc.spr=root&tpc.src=maite.iihe.ac.be:1094&tpc.stage=copy&tpc.str=1&tpc.tpr=root: [FATAL] Auth failed
> > > > DOWNLOAD (round-trip-15-download): 000110 : [2020-03-21 05:01:07.460848 -0500][Error ][XRootD ] [maite.iihe.ac.be:1094] Impossible to send message kXR_open (file: pnfs/iihe//dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ). Trying to recover.
> > > >
> > > > [2] CERN-EOS
> > > > UPLOAD (round-trip-10-upload): 000218 : Run: [ERROR] Server responded with an error: [3010] Unable to open file /eos/opstest/tpc/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04; Operation not permitted
> > > > TPC_DST_D (round-trip-10-tpc-dst-d): 000000 : [ERROR] Server responded with an error: [3011] Unable to remove /eos/opstest/tpc/from-CERN-TRUNK-21d897f6-51dd-4cc6-9535-58a414108e04-D; No such file or directory
> > > > DOWNLOAD (round-trip-10-download): 000129 : Run: [ERROR] Server responded with an error: [3011] Unable to open file /eos/opstest/tpc/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04; No such file or directory
> > > >
> > > > [3] CERN-TRUNK
> > > > TPC_SRC_D (round-trip-8-tpc-src-d): 000000 : [ERROR] Server responded with an error: [3010] Unable to remove /dpm/cern.ch/home/dteam/from-CERN-TRUNK-21d897f6-51dd-4cc6-9535-58a414108e04-D; permission denied
> > > > DOWNLOAD (round-trip-8-download): 000127 : Run: [ERROR] Server responded with an error: [3010] Permission denied
> > > >
> > > > [4] DESY-PROM
> > > > UPLOAD (round-trip-9-upload): 000186 : [2020-03-21 05:00:47.833821 -0500][Error ][XRootD ] [prometheus.desy.de:1095] Unable to send the message kXR_open (file: /VOs/dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?oss.asize=1048576, mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat ): [FATAL] Auth failed
> > > > TPC_DST_D (round-trip-9-tpc-dst-d): 000195 : [2020-03-21 05:00:48.337679 -0500][Error ][Utility ] Unable to open target root://prometheus.desy.de:1095//VOs/dteam/from-CERN-TRUNK-21d897f6-51dd-4cc6-9535-58a414108e04-D?oss.asize=0&tpc.dlg=prometheus.desy.de:1095&tpc.dlgon=1&tpc.key=140c2808a6b5a81d5e75e5d0&tpc.lfn=/VOs/dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04&tpc.scgi=tpc.stage=placement&tpc.spr=root&tpc.src=prometheus.desy.de:1095&tpc.stage=copy&tpc.str=1&tpc.tpr=root: [FATAL] Auth failed
> > > > DOWNLOAD (round-trip-9-download): 000110 : [2020-03-21 05:00:49.246017 -0500][Error ][XRootD ] [prometheus.desy.de:1095] Impossible to send message kXR_open (file: /VOs/dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ). Trying to recover.
> > > >
> > > > [5] FNAL
> > > > UPLOAD (round-trip-14-upload): 000186 : [2020-03-21 05:01:05.136590 -0500][Error ][XRootD ] [stkendca06a.fnal.gov:1094] Unable to send the message kXR_open (file: /pnfs/fnal.gov/VOs/dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?oss.asize=1048576, mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat ): [FATAL] Auth failed
> > > > TPC_DST_D (round-trip-14-tpc-dst-d): 000195 : [2020-03-21 05:01:05.235583 -0500][Error ][Utility ] Unable to open target root://stkendca06a.fnal.gov:1094//pnfs/fnal.gov/VOs/dteam/from-CERN-TRUNK-21d897f6-51dd-4cc6-9535-58a414108e04-D?oss.asize=0&tpc.dlg=stkendca06a.fnal.gov:1094&tpc.dlgon=1&tpc.key=0de3b5b8a6b5a9045e75e5e1&tpc.lfn=/pnfs/fnal.gov/VOs/dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04&tpc.scgi=tpc.stage=placement&tpc.spr=root&tpc.src=stkendca06a.fnal.gov:1094&tpc.stage=copy&tpc.str=1&tpc.tpr=root: [FATAL] Auth failed
> > > > DOWNLOAD (round-trip-14-download): 000110 : [2020-03-21 05:01:05.444704 -0500][Error ][XRootD ] [stkendca06a.fnal.gov:1094] Impossible to send message kXR_open (file: /pnfs/fnal.gov/VOs/dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ). Trying to recover.
> > > >
> > > > [6] IN2P3-DOMA
> > > > UPLOAD (round-trip-20-upload): 000186 : [2020-03-21 05:01:42.067389 -0500][Error ][XRootD ] [ccxrootddoma.in2p3.fr:1094] Unable to send the message kXR_open (file: /xrootd/dteam/doma/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?oss.asize=1048576, mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat ): [FATAL] Auth failed
> > > > TPC_DST_D (round-trip-20-tpc-dst-d): 000195 : [2020-03-21 05:01:42.585936 -0500][Error ][Utility ] Unable to open target root://ccxrootddoma.in2p3.fr:1094//xrootd/dteam/doma/from-CERN-TRUNK-21d897f6-51dd-4cc6-9535-58a414108e04-D?oss.asize=0&tpc.dlg=ccxrootddoma.in2p3.fr:1094&tpc.dlgon=1&tpc.key=1c56a380a6b5aa0c5e75e606&tpc.lfn=/xrootd/dteam/doma/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04&tpc.scgi=tpc.stage=placement&tpc.spr=root&tpc.src=ccosvms0124.in2p3.fr:1094&tpc.stage=copy&tpc.str=1&tpc.tpr=root: [FATAL] Auth failed
> > > > DOWNLOAD (round-trip-20-download): 000110 : [2020-03-21 05:01:43.407723 -0500][Error ][XRootD ] [ccxrootddoma.in2p3.fr:1094] Impossible to send message kXR_open (file: /xrootd/dteam/doma/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ). Trying to recover.
> > > >
> > > > [7] PRAGUE
> > > > UPLOAD (round-trip-3-upload): 000214 : Run: [ERROR] Server responded with an error: [3010] Permission denied
> > > > TPC_DST_D (round-trip-3-tpc-dst-d): 000000 : [ERROR] Server responded with an error: [3010] Unable to remove /dpm/farm.particle.cz/home/dteam/tpc-test/from-CERN-TRUNK-21d897f6-51dd-4cc6-9535-58a414108e04-D; permission denied
> > > > DOWNLOAD (round-trip-3-download): 000127 : Run: [ERROR] Server responded with an error: [3010] Permission denied
> > > >
> > > > [8] RAL-CEPH
> > > > UPLOAD (round-trip-2-upload): 000186 : [2020-03-21 05:00:18.701154 -0500][Error ][XRootD ] [ceph-test-gw683.gridpp.rl.ac.uk:1094] Unable to send the message kXR_open (file: /dteam:test/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?oss.asize=1048576, mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat ): [FATAL] Auth failed
> > > > TPC_DST_D (round-trip-2-tpc-dst-d): 000195 : [2020-03-21 05:00:19.190035 -0500][Error ][Utility ] Unable to open target root://ceph-test-gw683.gridpp.rl.ac.uk:1094//dteam:test/from-CERN-TRUNK-21d897f6-51dd-4cc6-9535-58a414108e04-D?oss.asize=0&tpc.dlg=ceph-test-gw683.gridpp.rl.ac.uk:1094&tpc.dlgon=1&tpc.key=05805ef8a6b5a6f85e75e5b3&tpc.lfn=/dteam:test/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04&tpc.scgi=tpc.stage=placement&tpc.spr=root&tpc.src=ceph-test-gw683.gridpp.rl.ac.uk:1094&tpc.stage=copy&tpc.str=1&tpc.tpr=root: [FATAL] Auth failed
> > > > DOWNLOAD (round-trip-2-download): 000110 : [2020-03-21 05:00:19.969919 -0500][Error ][XRootD ] [ceph-test-gw683.gridpp.rl.ac.uk:1094] Impossible to send message kXR_open (file: /dteam:test/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ). Trying to recover.
> > > >
> > > > [9] RAL-LCG2
> > > > UPLOAD (round-trip-1-upload): 000356 : [2020-03-21 05:00:05.333273 -0500][Error ][XRootD ] [echo.stfc.ac.uk:1094] Unable to send the message kXR_open (file: /dteam:test/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?oss.asize=1048576, mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat ): [FATAL] Auth failed
> > > > TPC_DST_D (round-trip-1-tpc-dst-d): 000365 : [2020-03-21 05:00:07.998168 -0500][Error ][Utility ] Unable to open target root://echo.stfc.ac.uk:1094//dteam:test/from-CERN-TRUNK-21d897f6-51dd-4cc6-9535-58a414108e04-D?oss.asize=0&tpc.dlg=echo.stfc.ac.uk:1094&tpc.dlgon=1&tpc.key=35a6b208a6b5a6cb5e75e5a7&tpc.lfn=/dteam:test/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04&tpc.scgi=tpc.stage=placement&tpc.spr=root&tpc.src=echo.stfc.ac.uk:1094&tpc.stage=copy&tpc.str=1&tpc.tpr=root: [FATAL] Auth failed
> > > > DOWNLOAD (round-trip-1-download): 000280 : [2020-03-21 05:00:13.049330 -0500][Error ][XRootD ] [echo.stfc.ac.uk:1094] Impossible to send message kXR_open (file: /dteam:test/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ). Trying to recover.
> > > >
> > > > [10] SLAC
> > > > UPLOAD (round-trip-11-upload): 000186 : [2020-03-21 05:00:58.184767 -0500][Error ][XRootD ] [griddev03.slac.stanford.edu:2094] Unable to send the message kXR_open (file: /xrootd/atlas/tpctest/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?oss.asize=1048576, mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat ): [FATAL] Auth failed
> > > > TPC_DST_D (round-trip-11-tpc-dst-d): 000195 : [2020-03-21 05:00:58.501336 -0500][Error ][Utility ] Unable to open target root://griddev03.slac.stanford.edu:2094//xrootd/atlas/tpctest/from-CERN-TRUNK-21d897f6-51dd-4cc6-9535-58a414108e04-D?oss.asize=0&tpc.dlg=griddev03.slac.stanford.edu:2094&tpc.dlgon=1&tpc.key=1ad90c00a6b5a8755e75e5da&tpc.lfn=/xrootd/atlas/tpctest/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04&tpc.scgi=tpc.stage=placement&tpc.spr=root&tpc.src=griddev03.slac.stanford.edu:2094&tpc.stage=copy&tpc.str=1&tpc.tpr=root: [FATAL] Auth failed
> > > > DOWNLOAD (round-trip-11-download): 000110 : [2020-03-21 05:00:58.987351 -0500][Error ][XRootD ] [griddev03.slac.stanford.edu:2094] Impossible to send message kXR_open (file: /xrootd/atlas/tpctest/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ). Trying to recover.
> > > >
> > > > [11] TRIUMF
> > > > UPLOAD (round-trip-16-upload): 000186 : [2020-03-21 05:01:08.742011 -0500][Error ][XRootD ] [ppshead.lcg.triumf.ca:1094] Unable to send the message kXR_open (file: /dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?oss.asize=1048576, mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat ): [FATAL] Auth failed
> > > > TPC_DST_D (round-trip-16-tpc-dst-d): 000195 : [2020-03-21 05:01:09.056302 -0500][Error ][Utility ] Unable to open target root://ppshead.lcg.triumf.ca:1094//dteam/from-CERN-TRUNK-21d897f6-51dd-4cc6-9535-58a414108e04-D?oss.asize=0&tpc.dlg=ppshead.lcg.triumf.ca:1094&tpc.dlgon=1&tpc.key=000fa3e8a6b5a95c5e75e5e5&tpc.lfn=/dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04&tpc.scgi=tpc.stage=placement&tpc.spr=root&tpc.src=ppshead.lcg.triumf.ca:1094&tpc.stage=copy&tpc.str=1&tpc.tpr=root: [FATAL] Auth failed
> > > > DOWNLOAD (round-trip-16-download): 000110 : [2020-03-21 05:01:09.513921 -0500][Error ][XRootD ] [ppshead.lcg.triumf.ca:1094] Impossible to send message kXR_open (file: /dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ). Trying to recover.
> > > >
> > > > [12] UKI-BRUNEL
> > > > UPLOAD (round-trip-4-upload): 000214 : Run: [ERROR] Server responded with an error: [3010] Permission denied
> > > > TPC_DST_D (round-trip-4-tpc-dst-d): 000000 : [ERROR] Server responded with an error: [3010] Unable to remove /dpm/brunel.ac.uk/home/dteam/wlcg-tpc/from-CERN-TRUNK-21d897f6-51dd-4cc6-9535-58a414108e04-D; permission denied
> > > > DOWNLOAD (round-trip-4-download): 000127 : Run: [ERROR] Server responded with an error: [3010] Permission denied
> > > >
> > > > [13] UKI-LANC
> > > > UPLOAD (round-trip-7-upload): 000214 : Run: [ERROR] Server responded with an error: [3010] Permission denied
> > > > TPC_DST_D (round-trip-7-tpc-dst-d): 000000 : [ERROR] Server responded with an error: [3010] Unable to remove /dpm/lancs.ac.uk/home/dteam/from-CERN-TRUNK-21d897f6-51dd-4cc6-9535-58a414108e04-D; permission denied
> > > > DOWNLOAD (round-trip-7-download): 000127 : Run: [ERROR] Server responded with an error: [3010] Permission denied
> > > >
> > > > [14] UKI-MAN1
> > > > UPLOAD (round-trip-5-upload): 000214 : Run: [ERROR] Server responded with an error: [3010] Permission denied
> > > > TPC_DST_D (round-trip-5-tpc-dst-d): 000000 : [ERROR] Server responded with an error: [3010] Unable to remove /dpm/tier2.hep.manchester.ac.uk/home/dteam/from-CERN-TRUNK-21d897f6-51dd-4cc6-9535-58a414108e04-D; permission denied
> > > > DOWNLOAD (round-trip-5-download): 000127 : Run: [ERROR] Server responded with an error: [3010] Permission denied
> > > >
> > > > [15] UKI-MAN2
> > > > UPLOAD (round-trip-6-upload): 000214 : Run: [ERROR] Server responded with an error: [3010] Permission denied
> > > > TPC_DST_D (round-trip-6-tpc-dst-d): 000000 : [ERROR] Server responded with an error: [3010] Unable to remove /dpm/tier2.hep.manchester.ac.uk/home/dteam/from-CERN-TRUNK-21d897f6-51dd-4cc6-9535-58a414108e04-D; permission denied
> > > > DOWNLOAD (round-trip-6-download): 000127 : Run: [ERROR] Server responded with an error: [3010] Permission denied
> > > >
> > > > [16] UNI-BONN
> > > > UPLOAD (round-trip-13-upload): 000186 : [2020-03-21 05:01:01.114829 -0500][Error ][XRootD ] [xrootd.physik.uni-bonn.de:1094] Unable to send the message kXR_open (file: /cephfs/grid/dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?oss.asize=1048576, mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat ): [FATAL] Auth failed
> > > > TPC_DST_D (round-trip-13-tpc-dst-d): 000197 : [2020-03-21 05:01:03.253425 -0500][Error ][Utility ] Unable to open target root://xrootd.physik.uni-bonn.de:1094//cephfs/grid/dteam/from-CERN-TRUNK-21d897f6-51dd-4cc6-9535-58a414108e04-D?oss.asize=0&tpc.dlg=xrootd.physik.uni-bonn.de:1094&tpc.dlgon=1&tpc.key=1ffebc20a6b5a8cd5e75e5dd&tpc.lfn=/cephfs/grid/dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04&tpc.scgi=tpc.stage=placement&tpc.spr=root&tpc.src=xrootd.physik.uni-bonn.de:1094&tpc.stage=copy&tpc.str=1&tpc.tpr=root: [FATAL] Auth failed
> > > > DOWNLOAD (round-trip-13-download): 000110 : [2020-03-21 05:01:04.113281 -0500][Error ][XRootD ] [xrootd.physik.uni-bonn.de:1094] Impossible to send message kXR_open (file: /cephfs/grid/dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ). Trying to recover.
> > > >
> > > > [17] OU
> > > > UPLOAD (round-trip-12-upload): 000186 : [2020-03-21 05:00:59.696352 -0500][Error ][XRootD ] [se1.oscer.ou.edu:1094] Unable to send the message kXR_open (file: /xrd/dteam/doma/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?oss.asize=1048576, mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat ): [FATAL] Auth failed
> > > > TPC_DST_D (round-trip-12-tpc-dst-d): 000195 : [2020-03-21 05:00:59.855213 -0500][Error ][Utility ] Unable to open target root://se1.oscer.ou.edu:1094//xrd/dteam/doma/from-CERN-TRUNK-21d897f6-51dd-4cc6-9535-58a414108e04-D?oss.asize=0&tpc.dlg=se1.oscer.ou.edu:1094&tpc.dlgon=1&tpc.key=31996728a6b5a8a15e75e5db&tpc.lfn=/xrd/dteam/doma/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04&tpc.scgi=tpc.stage=placement&tpc.spr=root&tpc.src=se1.oscer.ou.edu:1094&tpc.stage=copy&tpc.str=1&tpc.tpr=root: [FATAL] Auth failed
> > > > DOWNLOAD (round-trip-12-download): 000110 : [2020-03-21 05:01:00.140248 -0500][Error ][XRootD ] [se1.oscer.ou.edu:1094] Impossible to send message kXR_open (file: /xrd/dteam/doma/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ). Trying to recover.
> > > >
> > > > [18] BNL
> > > > UPLOAD (round-trip-19-upload): 000186 : [2020-03-21 05:01:39.336785 -0500][Error ][XRootD ] [dcqos001.usatlas.bnl.gov:1096] Unable to send the message kXR_open (file: /pnfs/usatlas.bnl.gov/data/dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?oss.asize=1048576, mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat ): [FATAL] Auth failed
> > > > TPC_DST_D (round-trip-19-tpc-dst-d): 000195 : [2020-03-21 05:01:39.772977 -0500][Error ][Utility ] Unable to open target root://dcqos001.usatlas.bnl.gov:1096//pnfs/usatlas.bnl.gov/data/dteam/from-CERN-TRUNK-21d897f6-51dd-4cc6-9535-58a414108e04-D?oss.asize=0&tpc.dlg=dcqos001.usatlas.bnl.gov:1096&tpc.dlgon=1&tpc.key=2dc112e0a6b5a9e05e75e603&tpc.lfn=/pnfs/usatlas.bnl.gov/data/dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04&tpc.scgi=tpc.stage=placement&tpc.spr=root&tpc.src=dcqos001.usatlas.bnl.gov:1096&tpc.stage=copy&tpc.str=1&tpc.tpr=root: [FATAL] Auth failed
> > > > DOWNLOAD (round-trip-19-download): 000110 : [2020-03-21 05:01:40.554008 -0500][Error ][XRootD ] [dcqos001.usatlas.bnl.gov:1096] Impossible to send message kXR_open (file: /pnfs/usatlas.bnl.gov/data/dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ). Trying to recover.
> > > >
> > > > [19] AGLT2
> > > > UPLOAD (round-trip-21-upload): 000214 : [2020-03-21 05:01:44.550200 -0500][Error ][XRootD ] [xrootd.aglt2.org:1094] Unable to send the message kXR_open (file: pnfs/aglt2.org/dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?oss.asize=1048576, mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat ): [FATAL] Auth failed
> > > > TPC_DST_D (round-trip-21-tpc-dst-d): 000223 : [2020-03-21 05:01:44.731532 -0500][Error ][Utility ] Unable to open target root://xrootd.aglt2.org:1094/pnfs/aglt2.org/dteam/from-CERN-TRUNK-21d897f6-51dd-4cc6-9535-58a414108e04-D?oss.asize=0&tpc.dlg=xrootd.aglt2.org:1094&tpc.dlgon=1&tpc.key=2af6e2d8a6b5aa385e75e608&tpc.lfn=pnfs/aglt2.org/dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04&tpc.scgi=tpc.stage=placement&tpc.spr=root&tpc.src=xrootd.aglt2.org:1094&tpc.stage=copy&tpc.str=1&tpc.tpr=root: [FATAL] Auth failed
> > > > DOWNLOAD (round-trip-21-download): 000138 : [2020-03-21 05:01:45.086137 -0500][Error ][XRootD ] [xrootd.aglt2.org:1094] Impossible to send message kXR_open (file: pnfs/aglt2.org/dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ). Trying to recover.
> > > >
> > > > [20] CALTECH
> > > > UPLOAD (round-trip-18-upload): 000186 : [2020-03-21 05:01:37.383371 -0500][Error ][XRootD ] [transfer-1.ultralight.org:1094] Unable to send the message kXR_open (file: /store/user/dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?oss.asize=1048576, mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat ): [FATAL] Auth failed
> > > > TPC_DST_D (round-trip-18-tpc-dst-d): 000195 : [2020-03-21 05:01:37.711002 -0500][Error ][Utility ] Unable to open target root://transfer-1.ultralight.org:1094//store/user/dteam/from-CERN-TRUNK-21d897f6-51dd-4cc6-9535-58a414108e04-D?oss.asize=0&tpc.dlg=transfer-1.ultralight.org:1094&tpc.dlgon=1&tpc.key=26e7de90a6b5a9b45e75e601&tpc.lfn=/store/user/dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04&tpc.scgi=tpc.stage=placement&tpc.spr=root&tpc.src=transfer-1.ultralight.org:1094&tpc.stage=copy&tpc.str=1&tpc.tpr=root: [FATAL] Auth failed
> > > > DOWNLOAD (round-trip-18-download): 000110 : [2020-03-21 05:01:38.345042 -0500][Error ][XRootD ] [transfer-1.ultralight.org:1094] Impossible to send message kXR_open (file: /store/user/dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ). Trying to recover.
> > > >
> > > > [21] TRIUMF-PROD
> > > > UPLOAD (round-trip-17-upload): 000720 : [2020-03-21 05:01:14.574303 -0500][Error ][XRootD ] [xrootd.lcg.triumf.ca:1094] Unable to send the message kXR_open (file: /dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?oss.asize=1048576, mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat ): [FATAL] Auth failed
> > > > TPC_DST_D (round-trip-17-tpc-dst-d): 000729 : [2020-03-21 05:01:19.059769 -0500][Error ][Utility ] Unable to open target root://xrootd.lcg.triumf.ca:1094//dteam/from-CERN-TRUNK-21d897f6-51dd-4cc6-9535-58a414108e04-D?oss.asize=0&tpc.dlg=xrootd.lcg.triumf.ca:1094&tpc.dlgon=1&tpc.key=00011558a6b5a9885e75e5ef&tpc.lfn=/dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04&tpc.scgi=tpc.stage=placement&tpc.spr=root&tpc.src=xrootd.lcg.triumf.ca:1094&tpc.stage=copy&tpc.str=1&tpc.tpr=root: [FATAL] Auth failed
> > > > DOWNLOAD (round-trip-17-download): 000644 : [2020-03-21 05:01:28.033733 -0500][Error ][XRootD ] [xrootd.lcg.triumf.ca:1094] Impossible to send message kXR_open (file: /dteam/xrootd-test-data-21d897f6-51dd-4cc6-9535-58a414108e04?, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ). Trying to recover.
> > > >
>
> ########################################################################
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the XROOTD-DEV list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>
>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
|