In the case of a redirector setup, XrdCl will swallow the last error it got from a data server and respond with a generic no servers available. Here's a truncated output:

$ BEARER_TOKEN=...TOKEN_OMITTED... xrdcp -d 2 -f root://redirector.osgstorage.org//user/ligo/test_access/access_ligo /dev/null
[2023-02-18 11:05:22.910848 -0600][Debug  ][Utility           ] CopyProcess: 2 jobs to prepare
... snip ...
[2023-02-18 11:05:22.912312 -0600][Debug  ][Utility           ] Opening root://redirector.osgstorage.org:1094//user/ligo/test_access/access_ligo for reading
[2023-02-18 11:05:22.912435 -0600][Debug  ][File              ] [0xca0a80@root://redirector.osgstorage.org:1094//user/ligo/test_access/access_ligo?xrdcl.requuid=c242fdbf-a2e6-4b1b-9ec6-97c48063a2c6] Sending an open command
[2023-02-18 11:05:22.912501 -0600][Debug  ][ExDbgMsg          ] [redirector.osgstorage.org:1094] MsgHandler created: 0xca6850 (message: kXR_open (file: /user/ligo/test_access/access_ligo?, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) ).
[2023-02-18 11:05:22.912584 -0600][Debug  ][PostMaster        ] Creating new channel to: root://redirector.osgstorage.org:1094
... snip ...
[2023-02-18 11:05:22.913477 -0600][Debug  ][AsyncSock         ] [redirector.osgstorage.org:1094.0] Attempting connection to [::ffff:128.104.103.164]:1094
... snip ...
[2023-02-18 11:05:22.961574 -0600][Debug  ][XRootDTransport   ] [redirector.osgstorage.org:1094.0] Sending out kXR_login request, username: bbockelm, cgi: ?xrd.cc=us&xrd.tz=-6&xrd.appname=xrdcp&xrd.info=&xrd.hostname=example.com&xrd.rn=v5.2.0, dual-stack: true, private IPv4: false, private IPv6: false
... snip ...
[2023-02-18 11:05:22.985035 -0600][Debug  ][XRootDTransport   ] [redirector.osgstorage.org:1094.0] Logged in, session: 3eaf0500b06e0000790000001fba0500
... snip ...
[2023-02-18 11:05:23.063394 -0600][Debug  ][ExDbgMsg          ] [xrootd-local.unl.edu:1094] Retry at server MsgHandler: 0xca6850 (message: kXR_open (file: /user/ligo/test_access/access_ligo?, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) ).
[2023-02-18 11:05:23.063460 -0600][Debug  ][PostMaster        ] Creating new channel to: root://xrootd-local.unl.edu:1094
... snip ...
[2023-02-18 11:05:23.064456 -0600][Debug  ][AsyncSock         ] [xrootd-local.unl.edu:1094.0] Attempting connection to [2600:900:6:1101:5054:ff:fe65:7c48]:1094
... snip ...
[2023-02-18 11:05:23.066080 -0600][Debug  ][XRootDTransport   ] [xrootd-local.unl.edu:1094.0] Sending out kXR_login request, username: bbockelm, cgi: ?xrd.cc=us&xrd.tz=-6&xrd.appname=xrdcp&xrd.info=&xrd.hostname=example.com&xrd.rn=v5.2.0, dual-stack: true, private IPv4: false, private IPv6: false
... snip ...
[2023-02-18 11:05:23.070075 -0600][Debug  ][XRootDTransport   ] [xrootd-local.unl.edu:1094.0] Logged in, session: 5b9e020031180000f0000000c9270300
[2023-02-18 11:05:23.070097 -0600][Debug  ][XRootDTransport   ] [xrootd-local.unl.edu:1094.0] Authentication is required: &P=gsi,v:10600,c:ssl,ca:ba240aa8.0|f5f0dfc2.0&P=ztn,0:4096:
... snip ...
[2023-02-18 11:05:23.089334 -0600][Debug  ][XRootDTransport   ] [xrootd-local.unl.edu:1094.0] Authenticated with ztn.
... snip ...
[2023-02-18 11:05:23.090208 -0600][Debug  ][ExDbgMsg          ] [red-xfer2.unl.edu:1094] Retry at server MsgHandler: 0xca6850 (message: kXR_open (file: /user/ligo/test_access/access_ligo?, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) ).
[2023-02-18 11:05:23.090268 -0600][Debug  ][PostMaster        ] Creating new channel to: root://red-xfer2.unl.edu:1094
... snip ...
[2023-02-18 11:05:23.091116 -0600][Debug  ][AsyncSock         ] [red-xfer2.unl.edu:1094.0] Attempting connection to [2600:900:6:1101:20f:53ff:fe31:d6a0]:1094
... snip ...
[2023-02-18 11:05:23.092932 -0600][Debug  ][XRootDTransport   ] [red-xfer2.unl.edu:1094.0] Sending out kXR_login request, username: bbockelm, cgi: ?xrd.cc=us&xrd.tz=-6&xrd.appname=xrdcp&xrd.info=&xrd.hostname=example.com&xrd.rn=v5.2.0, dual-stack: true, private IPv4: false, private IPv6: false
... snip ...
[2023-02-18 11:05:23.096605 -0600][Debug  ][XRootDTransport   ] [red-xfer2.unl.edu:1094.0] Authentication is required: &P=gsi,v:10600,c:ssl,ca:ba240aa8.0|f5f0dfc2.0&P=ztn,0:4096:
... snip ...
[2023-02-18 11:05:23.466519 -0600][Debug  ][XRootDTransport   ] [red-xfer2.unl.edu:1094.0] Authenticated with ztn.
... snip ...
[2023-02-18 11:05:23.466623 -0600][Debug  ][ExDbgMsg          ] [red-xfer2.unl.edu:1094] Moving MsgHandler: 0xca6850 (message: kXR_open (file: /user/ligo/test_access/access_ligo?, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) ) from out-queu to in-queue.
[2023-02-18 11:05:23.488628 -0600][Debug  ][XRootD            ] [red-xfer2.unl.edu:1094] Handling error while processing kXR_open (file: /user/ligo/test_access/access_ligo?, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ): [ERROR] Error response: permission denied.
[2023-02-18 11:05:23.488797 -0600][Debug  ][ExDbgMsg          ] [redirector.osgstorage.org:1094] Retry at server MsgHandler: 0xca6850 (message: kXR_open (file: /user/ligo/test_access/access_ligo?tried=red-xfer2.unl.edu,xrootd-local.unl.edu, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) ).
[2023-02-18 11:05:23.489062 -0600][Debug  ][ExDbgMsg          ] [redirector.osgstorage.org:1094] Moving MsgHandler: 0xca6850 (message: kXR_open (file: /user/ligo/test_access/access_ligo?tried=red-xfer2.unl.edu,xrootd-local.unl.edu, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) ) from out-queu to in-queue.
[2023-02-18 11:05:23.530035 -0600][Debug  ][XRootD            ] [redirector.osgstorage.org:1094] Handling error while processing kXR_open (file: /user/ligo/test_access/access_ligo?tried=red-xfer2.unl.edu,xrootd-local.unl.edu, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ): [ERROR] Error response: no such file or directory.
[2023-02-18 11:05:23.530101 -0600][Debug  ][ExDbgMsg          ] [redirector.osgstorage.org:1094] Calling MsgHandler: 0xca6850 (message: kXR_open (file: /user/ligo/test_access/access_ligo?tried=red-xfer2.unl.edu,xrootd-local.unl.edu, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) ) with status: [ERROR] Error response: no such file or directory.
[2023-02-18 11:05:23.530171 -0600][Debug  ][File              ] [0xca0a80@root://redirector.osgstorage.org:1094//user/ligo/test_access/access_ligo?xrdcl.requuid=c242fdbf-a2e6-4b1b-9ec6-97c48063a2c6] Open has returned with status [ERROR] Server responded with an error: [3011] No servers are available to read the file.
[2023-02-18 11:05:23.530187 -0600][Debug  ][File              ] [0xca0a80@root://redirector.osgstorage.org:1094//user/ligo/test_access/access_ligo?xrdcl.requuid=c242fdbf-a2e6-4b1b-9ec6-97c48063a2c6] Error while opening at redirector.osgstorage.org:1094: [ERROR] Server responded with an error: [3011] No servers are available to read the file.
[2023-02-18 11:05:23.530248 -0600][Debug  ][XRootD            ] Redirect trace-back:
[2023-02-18 11:05:23.530248 -0600][Debug  ][XRootD            ] 	0. Redirected from: root://redirector.osgstorage.org:1094//user/ligo/test_access/access_ligo to: root://xrootd-local.unl.edu:1094/
[2023-02-18 11:05:23.530248 -0600][Debug  ][XRootD            ] 	1. Redirected from: root://xrootd-local.unl.edu:1094/ to: root://red-xfer2.unl.edu:1094/
[2023-02-18 11:05:23.530248 -0600][Debug  ][XRootD            ] 	2. Retrying: root://redirector.osgstorage.org:1094//user/ligo/test_access/access_ligo
[2023-02-18 11:05:23.530285 -0600][Debug  ][ExDbgMsg          ] [redirector.osgstorage.org:1094] Destroying MsgHandler: 0xca6850.
[0B/0B][100%][==================================================][0B/s]  
Run: [ERROR] Server responded with an error: [3011] No servers are available to read the file. (source)
.. snip ....

To summarize:

Client was redirected twice, ending up at an actual data server.
Data server gave client a "permission denied"
Client returned to the first host (redirector), indicating which servers it had already retried.
Client open resulted in a [ERROR] Server responded with an error: [3011] No servers are available to read the file.

I would suggest the following logic:

If a client connects to a data server and receives a specific error message, and
The client returns to the redirector excluding the last data server, and
The client gets a "No servers are available" from the redirector, then
The client should return the error message from last data server.

So the above sequence would result in a:

[3010] Permission Denied.

Alternately, the client could generate:

[3010] Permission Denied.  Last redirector response was [3011] No servers are available to read the file.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.

[ { "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": "https://github.com/xrootd/xrootd/issues/1914", "url": "https://github.com/xrootd/xrootd/issues/1914", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

Use REPLY-ALL to reply to list

To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1