URL:
<http://savannah.cern.ch/bugs/?93794>
Summary: Infinite loop in talking to redirector
Project: XROOTD
Submitted by: bbockelm
Submitted on: 2012-04-18 03:03
Severity: 4 - Important
Priority: 5 - Normal
Status: None
Privacy: Public
Assigned to: None
Originator Email:
Open/Closed: Open
Discussion Lock: Any
Fixed by commit(s):
_______________________________________________________
Details:
This is seen with the Xrootd version that ships with ROOT 5.32 (3.1.0, if
memory recalls).
We have seen a couple of cases where Xrootd fails a few redirects due to
authentication issues, then goes into a loop in talking to the redirector.
See the stdout copy/pasted below. This is all the information I have at this
point: I haven't tried to reproduce.
On the redirector, I see logins and disconnects from the same clients,
over-and-over.
I am travelling and will not be able to give a good attempt at reproduction
until next week. Matevz might be able to help. I'm filing this bug in the
meantime in case if the issue is "obvious" to anyone.
Brian
[bbockelm@brian-test ~]$ head -n 100 /tmp/ucsd_issue
15-Apr-2012 19:45:50 PDT Initiating request to open file
root://xrootd.unl.edu//store/data/Run2012A/Photon/AOD/PromptReco-v1/000/191/090/F6B097F0-0B87-E111-8017-001D09F291D2.root
120415 19:45:51 001 Xrd: GoToAnotherServer: Going to:
cithep160.ultralight.org:1094
120415 19:45:58 001 Xrd: XrdClientSock::RecvRaw: Error reading from socket:
Connection reset by peer
120415 19:45:58 001 Xrd: DoHandShake: Failed to read 4 bytes. Retrying ...
120415 19:45:58 001 Xrd: GetAccessToSrv: HandShake failed with server
[cithep160.ultralight.org:1094]
120415 19:45:58 001 Xrd: GoToAnotherServer: Error handshaking to
[cithep160.ultralight.org:1094]
120415 19:46:04 001 Xrd: GoToAnotherServer: Going to: xrootd.unl.edu:1094
120415 19:46:04 001 Xrd: GoToAnotherServer: Going to:
cithep251.ultralight.org:1094
120415 19:46:14 001 Xrd: XrdClientSock::RecvRaw: Error reading from socket:
Connection reset by peer
120415 19:46:14 001 Xrd: DoHandShake: Failed to read 4 bytes. Retrying ...
120415 19:46:14 001 Xrd: GetAccessToSrv: HandShake failed with server
[cithep251.ultralight.org:1094]
120415 19:46:14 001 Xrd: GoToAnotherServer: Error handshaking to
[cithep251.ultralight.org:1094]
120415 19:46:19 001 Xrd: GoToAnotherServer: Going to: xrootd.unl.edu:1094
120415 19:46:19 001 Xrd: GoToAnotherServer: Going to:
gridftp-16-23.ultralight.org:1094
120415 19:46:23 001 Xrd: XrdClientSock::RecvRaw: Error reading from socket:
Connection reset by peer
120415 19:46:23 001 Xrd: DoHandShake: Failed to read 4 bytes. Retrying ...
120415 19:46:23 001 Xrd: GetAccessToSrv: HandShake failed with server
[gridftp-16-23.ultralight.org:1094]
120415 19:46:23 001 Xrd: GoToAnotherServer: Error handshaking to
[gridftp-16-23.ultralight.org:1094]
120415 19:46:28 001 Xrd: GoToAnotherServer: Going to: xrootd.unl.edu:1094
120415 19:46:28 001 Xrd: GoToAnotherServer: Going to:
cithep160.ultralight.org:1094
120415 19:47:28 001 Xrd: CheckErrorStatus: Server [xrootd.unl.edu:1094]
declared: (error code: 3010)
120415 19:47:28 001 Xrd: DoAuthentication:
XrdSec: No authentication protocols are available.
120415 19:47:28 001 Xrd: GoToAnotherServer: Error handshaking to
[cithep160.ultralight.org:1094]
120415 19:47:28 001 Xrd: Open: Error in handling a redirection.
120415 19:47:28 001 Xrd: CheckErrorStatus: Error while being redirected for
request Open
120415 19:47:28 001 Xrd: GoToAnotherServer: Going to: xrootd.unl.edu:1094
120415 19:47:29 5955 Xrd: XrdClientMessage::ReadRaw: Failed to read header (8
bytes).
120415 19:47:33 001 Xrd: GoToAnotherServer: Going to:
cithep251.ultralight.org:1094
120415 19:47:43 001 Xrd: XrdClientSock::RecvRaw: Error reading from socket:
Connection reset by peer
120415 19:47:43 001 Xrd: DoHandShake: Failed to read 4 bytes. Retrying ...
120415 19:47:43 001 Xrd: GetAccessToSrv: HandShake failed with server
[cithep251.ultralight.org:1094]
120415 19:47:43 001 Xrd: GoToAnotherServer: Error handshaking to
[cithep251.ultralight.org:1094]
120415 19:47:43 001 Xrd: Open: Error in handling a redirection.
120415 19:47:43 001 Xrd: CheckErrorStatus: Error while being redirected for
request Open
120415 19:47:43 001 Xrd: GoToAnotherServer: Going to: xrootd.unl.edu:1094
120415 19:47:49 001 Xrd: GoToAnotherServer: Going to:
gridftp-16-23.ultralight.org:1094
120415 19:48:49 001 Xrd: CheckErrorStatus: Server [xrootd.unl.edu:1094]
declared: (error code: 3010)
120415 19:48:49 001 Xrd: DoAuthentication:
XrdSec: No authentication protocols are available.
120415 19:48:49 001 Xrd: GoToAnotherServer: Error handshaking to
[gridftp-16-23.ultralight.org:1094]
120415 19:48:49 001 Xrd: Open: Error in handling a redirection.
120415 19:48:49 001 Xrd: CheckErrorStatus: Error while being redirected for
request Open
120415 19:48:49 001 Xrd: GoToAnotherServer: Going to: xrootd.unl.edu:1094
120415 19:48:50 5955 Xrd: XrdClientMessage::ReadRaw: Failed to read header (8
bytes).
120415 19:48:54 001 Xrd: GoToAnotherServer: Going to:
cithep172.ultralight.org:1094
120415 19:49:04 001 Xrd: XrdClientSock::RecvRaw: Error reading from socket:
Connection reset by peer
120415 19:49:04 001 Xrd: DoHandShake: Failed to read 4 bytes. Retrying ...
120415 19:49:04 001 Xrd: GetAccessToSrv: HandShake failed with server
[cithep172.ultralight.org:1094]
120415 19:49:04 001 Xrd: GoToAnotherServer: Error handshaking to
[cithep172.ultralight.org:1094]
120415 19:49:04 001 Xrd: Open: Error in handling a redirection.
120415 19:49:04 001 Xrd: CheckErrorStatus: Error while being redirected for
request Open
120415 19:49:04 001 Xrd: GoToAnotherServer: Going to: xrootd.unl.edu:1094
120415 19:49:09 001 Xrd: GoToAnotherServer: Going to:
cithep230.ultralight.org:1094
120415 19:49:49 5955 Xrd: XrdClientMessage::ReadRaw: Failed to read header (8
bytes).
120415 19:54:29 001 Xrd: ReadPartialAnswer: Failed to read msg from connmgr
(server [cithep230.ultralight.org:1094]). Retrying ...
120415 19:54:29 001 Xrd: GoToAnotherServer: Going to: xrootd.unl.edu:1094
120415 19:54:29 001 Xrd: CheckErrorStatus: Server
[cithep230.ultralight.org:1094] declared: Invalid request code(error code:
3006)
120415 19:54:29 001 Xrd: DoAuthentication: Invalid request code
XrdSec: No authentication protocols are available.
120415 19:54:29 001 Xrd: GoToAnotherServer: Error handshaking to
[cithep230.ultralight.org:1094]
120415 19:54:29 001 Xrd: Open: Error in handling a redirection.
120415 19:54:29 001 Xrd: CheckErrorStatus: Error while being redirected for
request Open
120415 19:54:29 001 Xrd: GoToAnotherServer: Going to: xrootd.unl.edu:1094
120415 19:54:34 5955 Xrd: XrdClientMessage::ReadRaw: Failed to read header (8
bytes).
120415 19:54:34 001 Xrd: GoToAnotherServer: Going to:
cithep230.ultralight.org:1094
120415 19:58:42 001 Xrd: CheckErrorStatus: Server [xrootd.unl.edu:1094]
declared: (error code: 3010)
120415 19:58:42 001 Xrd: DoAuthentication:
XrdSec: No authentication protocols are available.
120415 19:58:42 001 Xrd: GoToAnotherServer: Error handshaking to
[cithep230.ultralight.org:1094]
120415 19:58:42 001 Xrd: Open: Error in handling a redirection.
120415 19:58:42 001 Xrd: CheckErrorStatus: Error while being redirected for
request Open
120415 19:58:42 001 Xrd: GoToAnotherServer: Going to: xrootd.unl.edu:1094
120415 19:58:43 5955 Xrd: XrdClientMessage::ReadRaw: Failed to read header (8
bytes).
120415 19:58:47 001 Xrd: GoToAnotherServer: Going to: cmssrv32.fnal.gov:1094
120415 19:58:48 001 Xrd: GoToAnotherServer: Going to:
cmsstor158.fnal.gov:1094
120415 19:58:48 001 Xrd: SendGenCommand: Too many redirections for request
kXR_open. Aborting command.
120415 19:58:48 001 Xrd: GoToAnotherServer: Going to: xrootd.unl.edu:1094
120415 19:58:48 001 Xrd: SendGenCommand: Too many redirections for request
kXR_open. Aborting command.
120415 19:58:48 001 Xrd: GoToAnotherServer: Going to: xrootd.unl.edu:1094
120415 19:58:48 001 Xrd: SendGenCommand: Too many redirections for request
kXR_login. Aborting command.
120415 19:58:48 001 Xrd: GoToAnotherServer: Error handshaking to
[xrootd.unl.edu:1094]
120415 19:58:48 001 Xrd: GoToAnotherServer: Going to: xrootd.unl.edu:1094
120415 19:58:49 001 Xrd: SendGenCommand: Too many redirections for request
kXR_login. Aborting command.
120415 19:58:49 001 Xrd: GoToAnotherServer: Error handshaking to
[xrootd.unl.edu:1094]
120415 19:58:49 001 Xrd: GoToAnotherServer: Going to: xrootd.unl.edu:1094
120415 19:58:49 001 Xrd: SendGenCommand: Too many redirections for request
kXR_login. Aborting command.
120415 19:58:49 001 Xrd: GoToAnotherServer: Error handshaking to
[xrootd.unl.edu:1094]
120415 19:58:49 001 Xrd: GoToAnotherServer: Going to: xrootd.unl.edu:1094
120415 19:58:49 001 Xrd: SendGenCommand: Too many redirections for request
kXR_login. Aborting command.
120415 19:58:49 001 Xrd: GoToAnotherServer: Error handshaking to
[xrootd.unl.edu:1094]
120415 19:58:49 001 Xrd: GoToAnotherServer: Going to: xrootd.unl.edu:1094
120415 19:58:49 001 Xrd: SendGenCommand: Too many redirections for request
kXR_login. Aborting command.
120415 19:58:49 001 Xrd: GoToAnotherServer: Error handshaking to
[xrootd.unl.edu:1094]
120415 19:58:49 001 Xrd: GoToAnotherServer: Going to: xrootd.unl.edu:1094
120415 19:58:49 5955 Xrd: XrdClientMessage::ReadRaw: Failed to read header (8
bytes).
120415 19:58:49 001 Xrd: SendGenCommand: Too many redirections for request
kXR_login. Aborting command.
120415 19:58:49 001 Xrd: GoToAnotherServer: Error handshaking to
[xrootd.unl.edu:1094]
120415 19:58:49 001 Xrd: GoToAnotherServer: Going to: xrootd.unl.edu:1094
120415 19:58:50 001 Xrd: SendGenCommand: Too many redirections for request
kXR_login. Aborting command.
_______________________________________________________
Reply to this item at:
<http://savannah.cern.ch/bugs/?93794>
_______________________________________________
Message sent via/by LCG Savannah
http://savannah.cern.ch/
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
