Hi,
While running the xrootd-hdfs-fallback at UCSD where we rely on
tried=xrootd.t2.ucsd.edu to get files served from elsewhere (as we know they are
corrupt at ucsd) we observed that sometimes the proxy still gets redirected back
to UCSD.
I managed to reproduce this behaviour with xrdcp, two logs attached. The first
one is simpler and should be enough, the second one just has an extra
redirection to CIEAMT where the authentication fails.
It seems tried= gets lost right at the start when the client is passed from
xrootd.unl.edu to cms-xrd-global and is set to
[tried=+1213cmsxrootd1.fnal.gov1213xrootd.unl.edu]
Are we (CMS) doing something wrong here? Are redirectors really allowed to drop
opaque parameters?
But given that the tried= generated by the redirector looks rather fishy we
might be in luck and this is just a bug somewhere :)
Matevz
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
matevz@desire tmp> xrdcp --debug 1 --force 'root://xrootd.unl.edu:1094///store/mc/Summer12_DR53X/TTJets_HadronicMGDecays_8TeV-madgraph/AODSIM/PU_S10_START53_V7A_ext-v1/00000/861247D0-E125-E211-97A8-485B39800C0F.root?hdfs_block_size=134217728&tried=xrootd.t2.ucsd.edu' /dev/null
140930 14:44:58 14188 Xrd: main: (C) 2004-2011 by the XRootD collaboration. Version: v3.3.6
140930 14:44:58 14188 Xrd: Create: (C) 2004-2010 by the Xrootd group. XrdClient $Revision$ - Xrootd version: v3.3.6
140930 14:44:58 14188 Xrd: ShowUrls: The converted URLs count is 1
140930 14:44:58 14188 Xrd: ShowUrls: URL n.1: root://xrootd.unl.edu:1094///store/mc/Summer12_DR53X/TTJets_HadronicMGDecays_8TeV-madgraph/AODSIM/PU_S10_START53_V7A_ext-v1/00000/861247D0-E125-E211-97A8-485B39800C0F.root?hdfs_block_size=134217728&tried=xrootd.t2.ucsd.edu.
140930 14:44:58 14188 Xrd: ShowUrls: The converted URLs count is 1
140930 14:44:58 14188 Xrd: ShowUrls: URL n.1: root://xrootd.unl.edu:1094///store/mc/Summer12_DR53X/TTJets_HadronicMGDecays_8TeV-madgraph/AODSIM/PU_S10_START53_V7A_ext-v1/00000/861247D0-E125-E211-97A8-485B39800C0F.root?hdfs_block_size=134217728&tried=xrootd.t2.ucsd.edu.
140930 14:44:58 14188 Xrd: Open: Access to server granted.
140930 14:44:58 14188 Xrd: Open: Opening the remote file //store/mc/Summer12_DR53X/TTJets_HadronicMGDecays_8TeV-madgraph/AODSIM/PU_S10_START53_V7A_ext-v1/00000/861247D0-E125-E211-97A8-485B39800C0F.root?hdfs_block_size=134217728&tried=xrootd.t2.ucsd.edu
140930 14:44:58 14188 Xrd: Open: File open in progress.
140930 14:44:58 14191 Xrd: CheckErrorStatus: Server [xrootd.unl.edu:1094] requested 1 seconds of wait
140930 14:44:59 14191 Xrd: HandleServerError: Received redirection to [cms-xrd-global.cern.ch:1094]. Token=[]]. Opaque=[tried=+1213cmsxrootd1.fnal.gov1213xrootd.unl.edu].
140930 14:45:00 14191 Xrd: CheckErrorStatus: Server [cms-xrd-global.cern.ch:1094] requested 5 seconds of wait
140930 14:45:05 14191 Xrd: HandleServerError: Received redirection to [xrootd.unl.edu:1094]. Token=[]]. Opaque=[].
140930 14:45:05 14191 Xrd: HandleServerError: Received redirection to [cms-xrd-global.cern.ch:1094]. Token=[]]. Opaque=[tried=+1213cmsxrootd1.fnal.gov1213xrootd.unl.edu].
140930 14:45:05 14191 Xrd: HandleServerError: Received redirection to [cmsxrootd1.fnal.gov:1094]. Token=[]]. Opaque=[].
140930 14:45:06 14191 Xrd: HandleServerError: Received redirection to [xrootd.t2.ucsd.edu:1094]. Token=[]]. Opaque=[].
sec_Client: protocol request for host xrootd.t2.ucsd.edu token='&P=gsi,v:10300,c:ssl,ca:82c2b224.0|c7a717ce.0'
sec_PM: Loading gsi protocol object from libXrdSecgsi.so
140930 14:45:06 14191 secgsi_InitOpts: *** ------------------------------------------------------------ ***
140930 14:45:06 14191 secgsi_InitOpts: Mode: client
140930 14:45:06 14191 secgsi_InitOpts: Debug: 1
140930 14:45:06 14191 secgsi_InitOpts: CA dir: /etc/grid-security/certificates
140930 14:45:06 14191 secgsi_InitOpts: CA verification level: 1
140930 14:45:06 14191 secgsi_InitOpts: CRL dir: /etc/grid-security/certificates
140930 14:45:06 14191 secgsi_InitOpts: CRL extension: .r0
140930 14:45:06 14191 secgsi_InitOpts: CRL check level: 1
140930 14:45:06 14191 secgsi_InitOpts: CRL refresh time: 86400
140930 14:45:06 14191 secgsi_InitOpts: Certificate: /home/matevz/.globus/usercert.pem
140930 14:45:06 14191 secgsi_InitOpts: Key: /home/matevz/.globus/userkey.pem
140930 14:45:06 14191 secgsi_InitOpts: Proxy file: /tmp/x509up_u411
140930 14:45:06 14191 secgsi_InitOpts: Proxy validity: 12:00
140930 14:45:06 14191 secgsi_InitOpts: Proxy dep length: 0
140930 14:45:06 14191 secgsi_InitOpts: Proxy bits: 512
140930 14:45:06 14191 secgsi_InitOpts: Proxy sign option: 1
140930 14:45:06 14191 secgsi_InitOpts: Proxy delegation option: 0
140930 14:45:06 14191 secgsi_InitOpts: Allowed server names: [*/]<target host name>[/*]
140930 14:45:06 14191 secgsi_InitOpts: Crypto modules: ssl
140930 14:45:06 14191 secgsi_InitOpts: Ciphers: aes-128-cbc:bf-cbc:des-ede3-cbc
140930 14:45:06 14191 secgsi_InitOpts: MDigests: sha1:md5
140930 14:45:06 14191 secgsi_InitOpts: *** ------------------------------------------------------------ ***
sec_PM: Using gsi protocol, args='v:10300,c:ssl,ca:82c2b224.0|c7a717ce.0'
140930 14:45:06 14191 cryptossl_X509::IsCA: certificate has 6 extensions
140930 14:45:06 14191 secgsi_VerifyCA: Warning: CA certificate not self-signed and integrity not checked: assuming OK (82c2b224.0)
140930 14:45:06 14191 cryptossl_X509::IsCA: certificate has 6 extensions
140930 14:45:06 14191 cryptossl_X509::IsCA: certificate has 3 extensions
140930 14:45:06 14191 cryptossl_X509::IsCA: certificate has 11 extensions
140930 14:45:06 14191 cryptossl_X509::IsCA: certificate has 9 extensions
140930 14:45:06 14191 Xrd: HandleServerError: Received redirection to [cabinet-8-8-13.t2.ucsd.edu:1094]. Token=[]]. Opaque=[].
sec_Client: protocol request for host cabinet-8-8-13.t2.ucsd.edu token='&P=gsi,v:10300,c:ssl,ca:82c2b224.0|c7a717ce.0'
sec_PM: Using gsi protocol, args='v:10300,c:ssl,ca:82c2b224.0|c7a717ce.0'
140930 14:45:06 14191 cryptossl_X509::IsCA: certificate has 9 extensions
140930 14:45:06 14188 Xrd: main: root://xrootd.unl.edu:1094///store/mc/Summer12_DR53X/TTJets_HadronicMGDecays_8TeV-madgraph/AODSIM/PU_S10_START53_V7A_ext-v1/00000/861247D0-E125-E211-97A8-485B39800C0F.root?hdfs_block_size=134217728&tried=xrootd.t2.ucsd.edu --> /dev/null
140930 14:45:06 14199 Xrd: Read: Hole in the cache: offs=0, len=8388608
^Crootd] Total 4025.52 MB |=>..................| 8.55 % [55.6 MB/s]
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
matevz@desire tmp> xrdcp --debug 1 --force 'root://xrootd.unl.edu:1094//store/mc/Summer12_DR53X/TTJets_HadronicMGDecays_8TeV-madgraph/AODSIM/PU_S10_START53_V7A_ext-v1/00002/72D53160-3427-E211-8E1F-90E6BA0D09AD.root?hdfs_block_size=134217728&tried=xrootd.t2.ucsd.edu' /dev/null
140930 14:39:04 13987 Xrd: main: (C) 2004-2011 by the XRootD collaboration. Version: v3.3.6
140930 14:39:04 13987 Xrd: Create: (C) 2004-2010 by the Xrootd group. XrdClient $Revision$ - Xrootd version: v3.3.6
140930 14:39:04 13987 Xrd: ShowUrls: The converted URLs count is 1
140930 14:39:04 13987 Xrd: ShowUrls: URL n.1: root://xrootd.unl.edu:1094//store/mc/Summer12_DR53X/TTJets_HadronicMGDecays_8TeV-madgraph/AODSIM/PU_S10_START53_V7A_ext-v1/00002/72D53160-3427-E211-8E1F-90E6BA0D09AD.root?hdfs_block_size=134217728&tried=xrootd.t2.ucsd.edu.
140930 14:39:04 13987 Xrd: ShowUrls: The converted URLs count is 1
140930 14:39:04 13987 Xrd: ShowUrls: URL n.1: root://xrootd.unl.edu:1094//store/mc/Summer12_DR53X/TTJets_HadronicMGDecays_8TeV-madgraph/AODSIM/PU_S10_START53_V7A_ext-v1/00002/72D53160-3427-E211-8E1F-90E6BA0D09AD.root?hdfs_block_size=134217728&tried=xrootd.t2.ucsd.edu.
140930 14:39:04 13987 Xrd: Open: Access to server granted.
140930 14:39:04 13987 Xrd: Open: Opening the remote file /store/mc/Summer12_DR53X/TTJets_HadronicMGDecays_8TeV-madgraph/AODSIM/PU_S10_START53_V7A_ext-v1/00002/72D53160-3427-E211-8E1F-90E6BA0D09AD.root?hdfs_block_size=134217728&tried=xrootd.t2.ucsd.edu
140930 14:39:04 13987 Xrd: Open: File open in progress.
140930 14:39:04 13990 Xrd: CheckErrorStatus: Server [xrootd.unl.edu:1094] requested 1 seconds of wait
140930 14:39:05 13990 Xrd: HandleServerError: Received redirection to [cms-xrd-global.cern.ch:1094]. Token=[]]. Opaque=[tried=+1213cmsxrootd1.fnal.gov1213xrootd.unl.edu].
140930 14:39:06 13990 Xrd: HandleServerError: Received redirection to [xrootd-redic.pi.infn.it:1094]. Token=[]]. Opaque=[].
140930 14:39:07 13990 Xrd: HandleServerError: Received redirection to [gaexroot01.ciemat.es:1095]. Token=[]]. Opaque=[].
140930 14:39:11 13990 Xrd: HandleServerError: Received redirection to [gaeds011.ciemat.es:1094]. Token=[]]. Opaque=[].
sec_Client: protocol request for host gaeds011.ciemat.es token='&P=gsi,v:10200,c:ssl,ca:9dd23746'
sec_PM: Loading gsi protocol object from libXrdSecgsi.so
140930 14:39:12 13990 secgsi_InitOpts: *** ------------------------------------------------------------ ***
140930 14:39:12 13990 secgsi_InitOpts: Mode: client
140930 14:39:12 13990 secgsi_InitOpts: Debug: 1
140930 14:39:12 13990 secgsi_InitOpts: CA dir: /etc/grid-security/certificates
140930 14:39:12 13990 secgsi_InitOpts: CA verification level: 1
140930 14:39:12 13990 secgsi_InitOpts: CRL dir: /etc/grid-security/certificates
140930 14:39:12 13990 secgsi_InitOpts: CRL extension: .r0
140930 14:39:12 13990 secgsi_InitOpts: CRL check level: 1
140930 14:39:12 13990 secgsi_InitOpts: CRL refresh time: 86400
140930 14:39:12 13990 secgsi_InitOpts: Certificate: /home/matevz/.globus/usercert.pem
140930 14:39:12 13990 secgsi_InitOpts: Key: /home/matevz/.globus/userkey.pem
140930 14:39:12 13990 secgsi_InitOpts: Proxy file: /tmp/x509up_u411
140930 14:39:12 13990 secgsi_InitOpts: Proxy validity: 12:00
140930 14:39:12 13990 secgsi_InitOpts: Proxy dep length: 0
140930 14:39:12 13990 secgsi_InitOpts: Proxy bits: 512
140930 14:39:12 13990 secgsi_InitOpts: Proxy sign option: 1
140930 14:39:12 13990 secgsi_InitOpts: Proxy delegation option: 0
140930 14:39:12 13990 secgsi_InitOpts: Allowed server names: [*/]<target host name>[/*]
140930 14:39:12 13990 secgsi_InitOpts: Crypto modules: ssl
140930 14:39:12 13990 secgsi_InitOpts: Ciphers: aes-128-cbc:bf-cbc:des-ede3-cbc
140930 14:39:12 13990 secgsi_InitOpts: MDigests: sha1:md5
140930 14:39:12 13990 secgsi_InitOpts: *** ------------------------------------------------------------ ***
sec_PM: Using gsi protocol, args='v:10200,c:ssl,ca:9dd23746'
140930 14:39:12 13990 cryptossl_X509::IsCA: certificate has 10 extensions
140930 14:39:12 13990 cryptossl_X509::IsCA: certificate has 3 extensions
140930 14:39:12 13990 cryptossl_X509::IsCA: certificate has 11 extensions
140930 14:39:12 13990 cryptossl_X509::IsCA: certificate has 9 extensions
140930 14:39:12 13990 Xrd: CheckErrorStatus: Server [gaexroot01.ciemat.es:1095] declared: login failed(error code: 3010)
140930 14:39:12 13990 Xrd: DoAuthentication: login failed
sec_Client: protocol request for host gaeds011.ciemat.es token=''
XrdSec: No authentication protocols are available.
140930 14:39:12 13990 Xrd: GoToAnotherServer: Error handshaking to [gaeds011.ciemat.es:1094]
140930 14:39:12 13990 Xrd: Open: Error in handling a redirection.
140930 14:39:12 13990 Xrd: CheckErrorStatus: Error while being redirected for request Open
140930 14:39:12 13990 Xrd: Open: Back to cms-xrd-global.cern.ch 1094. Refreshing cache. Opaque info: &tried=xrootd.unl.edu,gaeds011.ciemat.es,
140930 14:39:13 13990 Xrd: CheckErrorStatus: Server [cms-xrd-global.cern.ch:1094] requested 5 seconds of wait
140930 14:39:14 14005 Xrd: XrdClientMessage::ReadRaw: Failed to read header (8 bytes).
140930 14:39:18 13990 Xrd: HandleServerError: Received redirection to [cmsxrootd1.fnal.gov:1094]. Token=[]]. Opaque=[].
140930 14:39:19 13990 Xrd: HandleServerError: Received redirection to [xrootd.t2.ucsd.edu:1094]. Token=[]]. Opaque=[].
sec_Client: protocol request for host xrootd.t2.ucsd.edu token='&P=gsi,v:10300,c:ssl,ca:82c2b224.0|c7a717ce.0'
sec_PM: Using gsi protocol, args='v:10300,c:ssl,ca:82c2b224.0|c7a717ce.0'
140930 14:39:19 13990 cryptossl_X509::IsCA: certificate has 6 extensions
140930 14:39:19 13990 secgsi_VerifyCA: Warning: CA certificate not self-signed and integrity not checked: assuming OK (82c2b224.0)
140930 14:39:19 13990 cryptossl_X509::IsCA: certificate has 6 extensions
140930 14:39:19 13990 cryptossl_X509::IsCA: certificate has 9 extensions
140930 14:39:19 13990 Xrd: HandleServerError: Received redirection to [uaf-4.t2.ucsd.edu:1094]. Token=[]]. Opaque=[].
sec_Client: protocol request for host uaf-4.t2.ucsd.edu token='&P=gsi,v:10300,c:ssl,ca:82c2b224.0|c7a717ce.0'
sec_PM: Using gsi protocol, args='v:10300,c:ssl,ca:82c2b224.0|c7a717ce.0'
140930 14:39:19 13990 cryptossl_X509::IsCA: certificate has 9 extensions
140930 14:39:19 13987 Xrd: main: root://xrootd.unl.edu:1094//store/mc/Summer12_DR53X/TTJets_HadronicMGDecays_8TeV-madgraph/AODSIM/PU_S10_START53_V7A_ext-v1/00002/72D53160-3427-E211-8E1F-90E6BA0D09AD.root?hdfs_block_size=134217728&tried=xrootd.t2.ucsd.edu --> /dev/null
140930 14:39:19 14012 Xrd: Read: Hole in the cache: offs=0, len=8388608
140930 14:39:19 14012 Xrd: Cache: Cache Status --------------------------
140930 14:39:19 14012 Xrd: Cache blk: 0Placeholder 0->8388607
140930 14:39:19 14012 Xrd: Cache blk: 1Data block 8388608->8650751
140930 14:39:19 14012 Xrd: Cache blk: 2Placeholder 8650752->12582911
140930 14:39:19 14012 Xrd: Cache blk: 3Placeholder 12582912->16777215
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
|