 |
 |
Hi Matevz, Thanks again for your help on this. FNAL files are in either in dCache or EOS. The behavior happened to both storage types. Also, the ufl.edu are behind luster(? Bockjoo may correct me). I tried with various environment variables you suggested. It seems XRD_NETWORKSTACK doesn't make any difference. But XRD_NOTLSOK made the real difference: bash-4.2$ export XRD_NOTLSOK=1 bash-4.2$ export XRD_NETWORKSTACK=IPv4 bash-4.2$ xrdfs cmsxrootd2.fnal.gov:1094 locate -d /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root [2023-09-20 05:18:03.245157 +0200][Info ][Utility ] Env: Importing from shell XRD_NOTLSOK=1 as notlsok [2023-09-20 05:18:03.245480 +0200][Info ][Utility ] Env: Importing from shell XRD_NETWORKSTACK=IPv4 as networkstack [::131.225.188.39]:1093 Server Read cmsxrootd-site3.fnal.gov:1093 Server Read cmsxrootd-site1.fnal.gov:1093 Server Read cmsxrootd-site2.fnal.gov:1093 Server Read cmsio5.rc.ufl.edu:1094 Server ReadWrite cmsio7.rc.ufl.edu:1094 Server ReadWrite cmsio9.rc.ufl.edu:1094 Server ReadWrite cmsio4.rc.ufl.edu:1094 Server ReadWrite cmsio3.rc.ufl.edu:1094 Server ReadWrite cmsio8.rc.ufl.edu:1094 Server ReadWrite cmsio6.rc.ufl.edu:1094 Server ReadWrite ----------- bash-4.2$ export XRD_NOTLSOK=0 bash-4.2$ xrdfs cmsxrootd2.fnal.gov:1094 locate -d /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root [2023-09-20 05:18:45.344122 +0200][Info ][Utility ] Env: Importing from shell XRD_NOTLSOK=0 as notlsok [2023-09-20 05:18:45.344248 +0200][Info ][Utility ] Env: Importing from shell XRD_NETWORKSTACK=IPv4 as networkstack [2023-09-20 05:18:46.106895 +0200][Error ][TlsMsg ] [] TLS error rc=-1 ec=1 (error_ssl) errno=0. [2023-09-20 05:18:46.106963 +0200][Error ][TlsMsg ] Failed to do TLS connect: Unable to connect to 131.225.188.52; error_ssl [2023-09-20 05:18:46.106976 +0200][Error ][AsyncSock ] [131.225.188.52:1094.0] Socket error while handshaking: [FATAL] TLS error: resource temporarily unavailable [2023-09-20 05:18:46.107068 +0200][Error ][PostMaster ] [131.225.188.52:1094] elapsed = 1, pConnectionWindow = 120 seconds. [2023-09-20 05:18:46.107088 +0200][Error ][PostMaster ] [131.225.188.52:1094] Unable to recover: [FATAL] TLS error: resource temporarily unavailable. [2023-09-20 05:18:46.107097 +0200][Error ][XRootD ] [131.225.188.52:1094] Impossible to send message kXR_locate (path: /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root, flags: none). Trying to recover. [2023-09-20 05:18:46.107112 +0200][Info ][XRootD ] [131.225.188.52:1094] Retrying request: kXR_locate (path: /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root, flags: none). [2023-09-20 05:18:46.107201 +0200][Info ][XRootD ] [131.225.188.52:1094] Retrying request: kXR_locate (path: /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root?tried=131.225.188.52&triedrc=srverr, flags: none). [2023-09-20 05:18:46.107271 +0200][Info ][XRootD ] [131.225.188.52:1094] Retrying request: kXR_locate (path: /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root?tried=131.225.188.52,131.225.188.52&triedrc=srverr,srverr, flags: none). [2023-09-20 05:18:46.112588 +0200][Error ][TlsMsg ] [] TLS error rc=-1 ec=1 (error_ssl) errno=0. [2023-09-20 05:18:46.112635 +0200][Error ][TlsMsg ] Failed to do TLS connect: Unable to connect to 131.225.205.75; error_ssl [2023-09-20 05:18:46.112649 +0200][Error ][AsyncSock ] [131.225.205.75:1094.0] Socket error while handshaking: [FATAL] TLS error: resource temporarily unavailable [2023-09-20 05:18:46.112655 +0200][Error ][TlsMsg ] [] TLS error rc=-1 ec=1 (error_ssl) errno=0. [2023-09-20 05:18:46.112693 +0200][Error ][TlsMsg ] Failed to do TLS connect: Unable to connect to 131.225.205.239; error_ssl [2023-09-20 05:18:46.112703 +0200][Error ][AsyncSock ] [131.225.205.239:1094.0] Socket error while handshaking: [FATAL] TLS error: resource temporarily unavailable [2023-09-20 05:18:46.112719 +0200][Error ][PostMaster ] [131.225.205.75:1094] elapsed = 1, pConnectionWindow = 120 seconds. [2023-09-20 05:18:46.112731 +0200][Error ][PostMaster ] [131.225.205.75:1094] Unable to recover: [FATAL] TLS error: resource temporarily unavailable. [2023-09-20 05:18:46.112739 +0200][Error ][XRootD ] [131.225.205.75:1094] Impossible to send message kXR_locate (path: /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root, flags: none). Trying to recover. [2023-09-20 05:18:46.112749 +0200][Error ][PostMaster ] [131.225.205.239:1094] elapsed = 1, pConnectionWindow = 120 seconds. [2023-09-20 05:18:46.112768 +0200][Error ][PostMaster ] [131.225.205.239:1094] Unable to recover: [FATAL] TLS error: resource temporarily unavailable. [2023-09-20 05:18:46.112779 +0200][Error ][XRootD ] [131.225.205.239:1094] Impossible to send message kXR_locate (path: /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root, flags: none). Trying to recover. [2023-09-20 05:18:46.112795 +0200][Info ][XRootD ] [131.225.205.239:1094] Retrying request: kXR_locate (path: /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root, flags: none). [2023-09-20 05:18:46.112849 +0200][Info ][XRootD ] [131.225.205.239:1094] Retrying request: kXR_locate (path: /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root?tried=131.225.205.239&triedrc=srverr, flags: none). [2023-09-20 05:18:46.112890 +0200][Info ][XRootD ] [131.225.205.239:1094] Retrying request: kXR_locate (path: /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root?tried=131.225.205.239,131.225.205.239&triedrc=srverr,srverr, flags: none). [2023-09-20 05:18:46.112754 +0200][Info ][XRootD ] [131.225.205.75:1094] Retrying request: kXR_locate (path: /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root, flags: none). [2023-09-20 05:18:46.112989 +0200][Info ][XRootD ] [131.225.205.75:1094] Retrying request: kXR_locate (path: /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root?tried=131.225.205.75&triedrc=srverr, flags: none). [2023-09-20 05:18:46.113038 +0200][Info ][XRootD ] [131.225.205.75:1094] Retrying request: kXR_locate (path: /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root?tried=131.225.205.75,131.225.205.75&triedrc=srverr,srverr, flags: none). [2023-09-20 05:18:46.139300 +0200][Error ][TlsMsg ] [] TLS error rc=-1 ec=1 (error_ssl) errno=0. [2023-09-20 05:18:46.139350 +0200][Error ][TlsMsg ] Failed to do TLS connect: Unable to connect to 128.227.221.219; error_ssl [2023-09-20 05:18:46.139361 +0200][Error ][AsyncSock ] [128.227.221.219:1094.0] Socket error while handshaking: [FATAL] TLS error: resource temporarily unavailable [2023-09-20 05:18:46.139425 +0200][Error ][PostMaster ] [128.227.221.219:1094] elapsed = 1, pConnectionWindow = 120 seconds. [2023-09-20 05:18:46.139433 +0200][Error ][PostMaster ] [128.227.221.219:1094] Unable to recover: [FATAL] TLS error: resource temporarily unavailable. [2023-09-20 05:18:46.139441 +0200][Error ][XRootD ] [128.227.221.219:1094] Impossible to send message kXR_locate (path: /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root, flags: none). Trying to recover. [2023-09-20 05:18:46.139455 +0200][Info ][XRootD ] [128.227.221.219:1094] Retrying request: kXR_locate (path: /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root, flags: none). [2023-09-20 05:18:46.139521 +0200][Info ][XRootD ] [128.227.221.219:1094] Retrying request: kXR_locate (path: /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root?tried=128.227.221.219&triedrc=srverr, flags: none). [2023-09-20 05:18:46.139565 +0200][Info ][XRootD ] [128.227.221.219:1094] Retrying request: kXR_locate (path: /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root?tried=128.227.221.219,128.227.221.219&triedrc=srverr,srverr, flags: none). [!] Some of the requests failed. The result may be incomplete. [::131.225.188.39]:1093 Server Read For the 131.225.188.39 server, it is a test server --- I commented out the authentication section in the redirector level and rely on the underlying XRootD door level authentication(maybe we are having too many layers of authentications). ---------------- This is the same for an EOS file: bash-4.2$ export XRD_NOTLSOK=1 bash-4.2$ xrdfs cmsxrootd2.fnal.gov:1094 locate -d /store/test/xrootd/redirector/test-file [2023-09-20 05:24:00.294558 +0200][Info ][Utility ] Env: Importing from shell XRD_NOTLSOK=1 as notlsok [2023-09-20 05:24:00.294661 +0200][Info ][Utility ] Env: Importing from shell XRD_NETWORKSTACK=IPv4 as networkstack [::131.225.188.39]:1092 Server Read cmsxrootd-site3.fnal.gov:1092 Server Read cmsxrootd-site1.fnal.gov:1096 Server Read bash-4.2$ export XRD_NOTLSOK=0 bash-4.2$ xrdfs cmsxrootd2.fnal.gov:1094 locate -d /store/test/xrootd/redirector/test-file [2023-09-20 05:23:48.903971 +0200][Info ][Utility ] Env: Importing from shell XRD_NOTLSOK=0 as notlsok [2023-09-20 05:23:48.904109 +0200][Info ][Utility ] Env: Importing from shell XRD_NETWORKSTACK=IPv4 as networkstack [2023-09-20 05:23:51.336095 +0200][Error ][TlsMsg ] [] TLS error rc=-1 ec=1 (error_ssl) errno=0. [2023-09-20 05:23:51.336167 +0200][Error ][TlsMsg ] Failed to do TLS connect: Unable to connect to 131.225.205.75; error_ssl [2023-09-20 05:23:51.336179 +0200][Error ][AsyncSock ] [131.225.205.75:1094.0] Socket error while handshaking: [FATAL] TLS error: resource temporarily unavailable [2023-09-20 05:23:51.336289 +0200][Error ][PostMaster ] [131.225.205.75:1094] elapsed = 0, pConnectionWindow = 120 seconds. [2023-09-20 05:23:51.336300 +0200][Error ][PostMaster ] [131.225.205.75:1094] Unable to recover: [FATAL] TLS error: resource temporarily unavailable. [2023-09-20 05:23:51.336308 +0200][Error ][XRootD ] [131.225.205.75:1094] Impossible to send message kXR_locate (path: /store/test/xrootd/redirector/test-file, flags: none). Trying to recover. [2023-09-20 05:23:51.336319 +0200][Info ][XRootD ] [131.225.205.75:1094] Retrying request: kXR_locate (path: /store/test/xrootd/redirector/test-file, flags: none). [2023-09-20 05:23:51.336406 +0200][Info ][XRootD ] [131.225.205.75:1094] Retrying request: kXR_locate (path: /store/test/xrootd/redirector/test-file?tried=131.225.205.75&triedrc=srverr, flags: none). [2023-09-20 05:23:51.336453 +0200][Info ][XRootD ] [131.225.205.75:1094] Retrying request: kXR_locate (path: /store/test/xrootd/redirector/test-file?tried=131.225.205.75,131.225.205.75&triedrc=srverr,srverr, flags: none). [2023-09-20 05:23:51.345510 +0200][Error ][TlsMsg ] [] TLS error rc=-1 ec=1 (error_ssl) errno=0. [2023-09-20 05:23:51.345576 +0200][Error ][TlsMsg ] Failed to do TLS connect: Unable to connect to 131.225.188.52; error_ssl [2023-09-20 05:23:51.345589 +0200][Error ][AsyncSock ] [131.225.188.52:1094.0] Socket error while handshaking: [FATAL] TLS error: resource temporarily unavailable [2023-09-20 05:23:51.345658 +0200][Error ][PostMaster ] [131.225.188.52:1094] elapsed = 0, pConnectionWindow = 120 seconds. [2023-09-20 05:23:51.345667 +0200][Error ][PostMaster ] [131.225.188.52:1094] Unable to recover: [FATAL] TLS error: resource temporarily unavailable. [2023-09-20 05:23:51.345675 +0200][Error ][XRootD ] [131.225.188.52:1094] Impossible to send message kXR_locate (path: /store/test/xrootd/redirector/test-file, flags: none). Trying to recover. [2023-09-20 05:23:51.345688 +0200][Info ][XRootD ] [131.225.188.52:1094] Retrying request: kXR_locate (path: /store/test/xrootd/redirector/test-file, flags: none). [2023-09-20 05:23:51.345751 +0200][Info ][XRootD ] [131.225.188.52:1094] Retrying request: kXR_locate (path: /store/test/xrootd/redirector/test-file?tried=131.225.188.52&triedrc=srverr, flags: none). [2023-09-20 05:23:51.345797 +0200][Info ][XRootD ] [131.225.188.52:1094] Retrying request: kXR_locate (path: /store/test/xrootd/redirector/test-file?tried=131.225.188.52,131.225.188.52&triedrc=srverr,srverr, flags: none). [!] Some of the requests failed. The result may be incomplete. [::131.225.188.39]:1092 Server Read Please just let me know if you need any other info or want me to run additional tests. Regards, Yujun ________________________________ From: Matevz Tadel <[log in to unmask]> Sent: Tuesday, September 19, 2023 6:52 PM To: Yujun Wu <[log in to unmask]>; Bockjoo Kim <[log in to unmask]>; Yujun Wu <[log in to unmask]>; Andrew Hanushevsky <[log in to unmask]> Cc: xrootd-l <[log in to unmask]>; Bockjoo Kim <[log in to unmask]> Subject: Re: Help with a xrdfs query puzzle with "xrdfs locate -d" Hi Yujun, On 9/19/23 07:16, Yujun Wu wrote: > Hi Matevz and Bockjoo, > > Thanks for your help with the investigation. Those cmsxrootd-site[1-3].fnal.gov > are connected with our dCache and EOS XROOTD doors. Is fnal eos still the 4.12 based one? Maybe there was a bug with -m support there. And maybe the same is true for the dCache doors? I remember 4.12 has some serious trouble parsing certs with DNs containing ',' ... and maybe some other characters (took me a long while t ofigure out why UCSD xcache wasn't able to auth to fnal eos :) ). I'd try to figure out the lowest level where the -m/-d problem appears. Is it on cmsxrootd-site[123].fnal.gov:1094 redirs? What versions are those? But if there is a problem with the underlying servers (dCache & EOS doors) this should really be fixed there. XRD_LOGLEVEL=Info xrdfs cmsxrootd2.fnal.gov:1094 locate -d /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root You can also try XRD_LOGLEVEL=Debug (and see other XrdCl env vars, esp. XRD_TLSDBGLVL and timeouts, as descibed in man xrdcp). Have you tried doing ipv4 vs. 6 test yet? XRD_NETWORKSTACK=... eg: matevz@black src> XRD_NOTLSOK=1 XRD_NETWORKSTACK=IPv4 XRD_LOGLEVEL=Info xrdfs cmsxrootd2.fnal.gov:1094 locate -d /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root [2023-09-19 16:50:17.116271 -0700][Info ][Utility ] Env: Importing from shell XRD_NOTLSOK=1 as notlsok [2023-09-19 16:50:17.116359 -0700][Info ][Utility ] Env: Importing from shell XRD_NETWORKSTACK=IPv4 as networkstack [::131.225.188.39]:1093 Server Read cmsxrootd-site3.fnal.gov:1093 Server Read cmsxrootd-site2.fnal.gov:1093 Server Read cmsxrootd-site1.fnal.gov:1093 Server Read cmsio5.rc.ufl.edu:1094 Server ReadWrite cmsio7.rc.ufl.edu:1094 Server ReadWrite cmsio9.rc.ufl.edu:1094 Server ReadWrite cmsio4.rc.ufl.edu:1094 Server ReadWrite cmsio3.rc.ufl.edu:1094 Server ReadWrite cmsio8.rc.ufl.edu:1094 Server ReadWrite cmsio6.rc.ufl.edu:1094 Server ReadWrite Cheers, Matevz > It looks like the following work fine: > > > 1. Without both "-d" and "-m" option: > > bash-4.2$ time xrdfs cmsxrootd2.fnal.gov:1094 locate > /store/test/xrootd/T1_US_FNAL/store/mc/SAM/GenericTTbar/AODSIM/CMSSW_9_2_6_91X_mcRun1_realistic_v2-v1/00000/AE237916-5D76-E711-A48C-FA163EEEBFED.root > > [2620:6a:0:8420::f8]:1094 Manager Read > > [2620:6a:0:8421::243]:1094 Manager Read > > [2620:6a:0:8420::f9]:1094 Manager Read > > > real0m0.440s > > user0m0.006s > > sys0m0.007s > > > 2.Only with "-m" option: > > bash-4.2$ time xrdfs cmsxrootd2.fnal.gov:1094 locate -m > /store/test/xrootd/T1_US_FNAL/store/mc/SAM/GenericTTbar/AODSIM/CMSSW_9_2_6_91X_mcRun1_realistic_v2-v1/00000/AE237916-5D76-E711-A48C-FA163EEEBFED.root > > cmsxrootd-site1.fnal.gov:1094 Manager Read > > cmsxrootd-site3.fnal.gov:1094 Manager Read > > cmsxrootd-site2.fnal.gov:1094 Manager Read > > > real0m0.442s > > user0m0.005s > > sys0m0.008s > > > > 3. With both "-d" and "-m" option: > > bash-4.2$ time xrdfs cmsxrootd2.fnal.gov:1094 locate -d -m > /store/test/xrootd/T1_US_FNAL/store/mc/SAM/GenericTTbar/AODSIM/CMSSW_9_2_6_91X_mcRun1_realistic_v2-v1/00000/AE237916-5D76-E711-A48C-FA163EEEBFED.root > > cmsxrootd-site3.fnal.gov:1093 Server Read > > cmsxrootd-site2.fnal.gov:1093 Server Read > > cmsxrootd-site1.fnal.gov:1093 Server Read > > > real1m16.579s > > user0m0.199s > > sys0m0.010s > > > (Try again) > > bash-4.2$time xrdfs cmsxrootd2.fnal.gov:1094 locate -d -m > /store/test/xrootd/T1_US_FNAL/store/mc/SAM/GenericTTbar/AODSIM/CMSSW_9_2_6_91X_mcRun1_realistic_v2-v1/00000/AE237916-5D76-E711-A48C-FA163EEEBFED.root > > cmsxrootd-site3.fnal.gov:1093 Server Read > > cmsxrootd-site1.fnal.gov:1093 Server Read > > cmsxrootd-site2.fnal.gov:1093 Server Read > > > real0m1.859s > > user0m0.182s > > sys0m0.013s > > > ------------- > > It does NOT work with only "-d" option: > > bash-4.2$ time xrdfs cmsxrootd2.fnal.gov:1094 locate -d > /store/test/xrootd/T1_US_FNAL/store/mc/SAM/GenericTTbar/AODSIM/CMSSW_9_2_6_91X_mcRun1_realistic_v2-v1/00000/AE237916-5D76-E711-A48C-FA163EEEBFED.root > > [ERROR] Server responded with an error: [3011] No valid location found > > > > real0m0.763s > > user0m0.009s > > sys0m0.007s > > > ------------ > > It seems "-d" is a pretty expensive operation. It goes deep to all levels of > XRootD redirectors and only stop after reaching all data servers. As you can > see, the first time my query with "-d" and "-m" options took more than over 76 > seconds to do the requests and the 2^nd time the query returned after 1.86 > seconds as the results were still in cache. > > I have a test setup. It seems "-d" option could work if I reduce one layer of > authentication at cmsxrootd-site level, but not sure if this will violate the > CMS metadata access policy. I will put this in our test server and see how it goes. > > Anyway, it will be good to have xrdfs locate with ONLY "-d" option work. On the > other hand, I just looked at your monitoring table. Could you please tell if it > is possible to get the info without "-d" option? Also, personally I would use > "-m" option to make sure to get the hostnames instead of IP addresses. > > > Regards, > Yujun > > > > > 2. > > > > > > -------------------------------------------------------------------------------- > *From:* Bockjoo Kim <[log in to unmask]> > *Sent:* Tuesday, September 19, 2023 3:25 AM > *To:* Matevz Tadel <[log in to unmask]>; Yujun Wu > <[log in to unmask]>; Andrew Hanushevsky > <[log in to unmask]> > *Cc:* xrootd-l <[log in to unmask]>; Bockjoo Kim <[log in to unmask]> > *Subject:* Re: Help with a xrdfs query puzzle with "xrdfs locate -d" > > 1093 port is served by the dCache door. > > Because file locations became imbalanced according to the top level > redirector: > > https://bockjoo.web.cern.ch/aaaRedirectorMonitoring.txt-20230912 > <https://urldefense.com/v3/__https://bockjoo.web.cern.ch/aaaRedirectorMonitoring.txt-20230912__;!!Mih3wA!EOUiA0ZCGIAwKRfijYGjhakArUROAghVShw4MbKscFcBfdJpEJpfLTicmkRIHa6jlaFe6iiDiCh0jYH8w9Rt$> > > where one can see the jobs at American sites are redirected to the EU > regional redirectors > > instead of the US regional redirector. > > xrdfs should work with or without '-d' option, no? > > Thanks, > > Bockjoo > > On 9/19/23 01:58, Matevz Tadel wrote: >> xrdmapc cmsxrootd2.fnal.gov:1094 >> 1 Man cmsxrootd-site2.fnal.gov:1094 >> Srv cmsxrootd-site2.fnal.gov:1093 >> 1 Man cmsxrootd-site3.fnal.gov:1094 >> Srv cmsxrootd-site3.fnal.gov:1093 >> Srv cmsxrootd-site3.fnal.gov:1092 >> 1 Man cmsxrootd-site1.fnal.gov:1094 >> Srv cmsxrootd-site1.fnal.gov:1096 >> Srv cmsxrootd-site1.fnal.gov:1093 >> .... >> >> Are those servers dCache doors? >> >> Things also seem to work better without the -d flag. Why are you using -d? >> >> Matevz >> >> On 9/18/23 09:46, Yujun Wu wrote: >>> Hi Andrew, >>> >>> Good morning. >>> >>> Thanks again for your help. All the client and server hosts have both IPv4 and >>> IPv6 addresses. When querying the cmsxrootd-site1.fnal.gov redirector directly. >>> I get: >>> >>> bash-4.2$ date; /usr/bin/xrdfs cmsxrootd-site1.fnal.gov:1094 locate -d >>> /store/test/xrootd/T1_US_FNAL/store/mc/SAM/GenericTTbar/AODSIM/CMSSW_9_2_6_91X_mcRun1_realistic_v2-v1/00000/AE237916-5D76-E711-A48C-FA163EEEBFED.root >>> Mon Sep 18 18:09:30 CEST 2023 >>> cmsxrootd-site1.fnal.gov:1093 Server Read >>> bash-4.2$ date; /usr/bin/xrdfs cmsxrootd-site1.fnal.gov:1094 locate -d -m >>> /store/test/xrootd/T1_US_FNAL/store/mc/SAM/GenericTTbar/AODSIM/CMSSW_9_2_6_91X_mcRun1_realistic_v2-v1/00000/AE237916-5D76-E711-A48C-FA163EEEBFED.root >>> Mon Sep 18 18:09:40 CEST 2023 >>> cmsxrootd-site1.fnal.gov:1093 Server Read >>> Both queries with/without "-m" option return hostname. >>> >>> ******** >>> Also, for your info. Our cmsxrootd-site1.fnal.gov redirector at Port 1094 is >>> basically a manager for both cmsd and xrootd with two redirectors registered >>> with it: >>> one at Port 1093 and other at Port 1092 (their roles are: all.role server). If I >>> query them directly, I get: >>> >>> bash-4.2$ date; /usr/bin/xrdfs cmsxrootd-site1.fnal.gov:1093 locate -d >>> /store/test/xrootd/T1_US_FNAL/store/mc/SAM/GenericTTbar/AODSIM/CMSSW_9_2_6_91X_mcRun1_realistic_v2-v1/00000/AE237916-5D76-E711-A48C-FA163EEEBFED.root >>> Mon Sep 18 18:09:57 CEST 2023 >>> [::131.225.205.75]:1093 Server ReadWrite >>> >>> bash-4.2$ date; /usr/bin/xrdfs cmsxrootd-site1.fnal.gov:1093 locate -d -m >>> /store/test/xrootd/T1_US_FNAL/store/mc/SAM/GenericTTbar/AODSIM/CMSSW_9_2_6_91X_mcRun1_realistic_v2-v1/00000/AE237916-5D76-E711-A48C-FA163EEEBFED.root >>> Mon Sep 18 18:10:07 CEST 2023 >>> cmsxrootd-site1.fnal.gov:1093 Server ReadWrite >>> The first one returns IP address without "-m" optioin and 2^nd one returns >>> hostname. >>> --------------------------------------------------------------------------------------- >>> Then, cmsxrootd-site1.fnal.gov manager registers with cmsxrootd.fnal.gov (An >>> alias for xrootd.unl.edu and cmsxrootd2.fnal.gov): >>> all.manager cmsxrootd-site1.fnal.gov:1213 >>> all.manager meta all cmsxrootd.fnal.gov+:1213 >>> >>> bash-4.2$ host cmsxrootd.fnal.gov >>> cmsxrootd.fnal.gov has address 131.225.184.17 (cmsxrootd2.fnal.gov) >>> cmsxrootd.fnal.gov has address 129.93.239.131(xrootd.unl.edu) >>> cmsxrootd.fnal.gov has IPv6 address >>> 2600:900:6:1101:5054:ff:fe00:70cb(xrootd.unl.edu) >>> cmsxrootd.fnal.gov has IPv6 address 2620:6a:0:187::184:17(cmsxrootd2.fnal.gov >>> >>> >>> bash-4.2$ date; /usr/bin/xrdfs cmsxrootd2.fnal.gov:1094 locate -d -m >>> /store/test/xrootd/T1_US_FNAL/store/mc/SAM/GenericTTbar/AODSIM/CMSSW_9_2_6_91X_mcRun1_realistic_v2-v1/00000/AE237916-5D76-E711-A48C-FA163EEEBFED.root >>> Mon Sep 18 18:35:26 CEST 2023 >>> cmsxrootd-site2.fnal.gov:1093 Server Read >>> cmsxrootd-site1.fnal.gov:1093 Server Read >>> cmsxrootd-site3.fnal.gov:1093 Server Read >>> bash-4.2$ date; /usr/bin/xrdfs cmsxrootd2.fnal.gov:1094 locate -d >>> /store/test/xrootd/T1_US_FNAL/store/mc/SAM/GenericTTbar/AODSIM/CMSSW_9_2_6_91X_mcRun1_realistic_v2-v1/00000/AE237916-5D76-E711-A48C-FA163EEEBFED.root >>> Mon Sep 18 18:35:36 CEST 2023 >>> [ERROR] Server responded with an error: [3011] No valid location found >>> >>> The weirdest part is: xrootd.unl.edu returns the same result with and without "-m": >>> bash-4.2$ date; /usr/bin/xrdfs xrootd.unl.edu:1094 locate -d >>> /store/test/xrootd/T1_US_FNAL/store/mc/SAM/GenericTTbar/AODSIM/CMSSW_9_2_6_91X_mcRun1_realistic_v2-v1/00000/AE237916-5D76-E711-A48C-FA163EEEBFED.root >>> Mon Sep 18 18:40:25 CEST 2023 >>> cmsxrootd-site1.fnal.gov:1093 Server Read >>> cmsxrootd-site2.fnal.gov:1093 Server Read >>> cmsxrootd-site3.fnal.gov:1093 Server Read >>> bash-4.2$ date; /usr/bin/xrdfs xrootd.unl.edu:1094 locate -d -m >>> /store/test/xrootd/T1_US_FNAL/store/mc/SAM/GenericTTbar/AODSIM/CMSSW_9_2_6_91X_mcRun1_realistic_v2-v1/00000/AE237916-5D76-E711-A48C-FA163EEEBFED.root >>> Mon Sep 18 18:40:35 CEST 2023 >>> cmsxrootd-site2.fnal.gov:1093 Server Read >>> cmsxrootd-site1.fnal.gov:1093 Server Read >>> cmsxrootd-site3.fnal.gov:1093 Server Read >>> ------------------------------------------------------------------ >>> >>> This happens even after I put xrootd.unl.edu XRootD configuration file on >>> cmsxrootd2.fnal.gov. >>> >>> Could you please advise what could be the cause on this? Or, if there is an >>> option at server side that could force "-m" behavior? Thanks again for your help. >>> >>> >>> Regards, >>> Yujun >>> >>> >>> -------------------------------------------------------------------------------- >>> *From:* Andrew Hanushevsky <[log in to unmask]> >>> *Sent:* Friday, September 15, 2023 11:53 PM >>> *To:* Yujun Wu <[log in to unmask]> >>> *Subject:* Re: Help with a xrdfs query puzzle with "xrdfs locate -d" >>> >>> One more thing try doing a locate against cmsxrootd-site1.fnal.gov:1094 >>> which in theory has the file. >>> >>> On Sat, 16 Sep 2023, Yujun Wu wrote: >>> >>>> Hi Andrew, >>>> >>>> Thank you for your help. Currently, CMS XRootD redirectors only allow access with voms proxy from /cms group. >>>> >>>> It seems all the files I tried have this issue. The following are two examples: >>>> >>>> >>>> [enstore@fndca2b ~]$ date; xrdfs cmsxrootd2.fnal.gov:1094 locate -d -m /store/test/xrootd/T1_US_FNAL/store/mc/SAM/GenericTTbar/AODSIM/CMSSW_9_2_6_91X_mcRun1_realistic_v2-v1/00000/AE237916-5D76-E711-A48C-FA163EEEBFED.root >>>> >>>> Fri Sep 15 22:41:40 CDT 2023 >>>> >>>> cmsxrootd-site3.fnal.gov:1093 Server Read >>>> >>>> cmsxrootd-site1.fnal.gov:1093 Server Read >>>> >>>> cmsxrootd-site2.fnal.gov:1093 Server Read >>>> >>>> [enstore@fndca2b ~]$ date; xrdfs cmsxrootd2.fnal.gov:1094 locate -d /store/test/xrootd/T1_US_FNAL/store/mc/SAM/GenericTTbar/AODSIM/CMSSW_9_2_6_91X_mcRun1_realistic_v2-v1/00000/AE237916-5D76-E711-A48C-FA163EEEBFED.root >>>> >>>> Fri Sep 15 22:41:56 CDT 2023 >>>> >>>> [ERROR] Server responded with an error: [3011] No valid location found >>>> >>>> >>>> ------------- >>>> >>>> [enstore@fndca2b ~]$ date; xrdfs cmsxrootd2.fnal.gov:1094 locate -d -m /store/test/xrootd/redirector/test-file >>>> >>>> Fri Sep 15 22:42:34 CDT 2023 >>>> >>>> cmsxrootd-site3.fnal.gov:1092 Server Read >>>> >>>> cmsxrootd-site1.fnal.gov:1096 Server Read >>>> >>>> [enstore@fndca2b ~]$ date; xrdfs cmsxrootd2.fnal.gov:1094 locate -d /store/test/xrootd/redirector/test-file >>>> >>>> Fri Sep 15 22:42:45 CDT 2023 >>>> >>>> [ERROR] Server responded with an error: [3011] No valid location found >>>> >>>> ------------- >>>> For the 2nd file, the log files showed: >>>> ###cmsxrootd2### >>>> >>>> [root@cmsxrootd2 tmp]# tail -n 100000 cmsd.log|grep test-file >>>> >>>> [root@cmsxrootd2 tmp]# tail -n 100000 xrootd.log|grep test-file >>>> >>>> 230915 22:42:34 17939 acc_Audit: enstore.233531:159@fndca2b grant host *@fndca2b.fnal.gov stat /store/test/xrootd/redirector/test-file >>>> >>>> 230915 22:42:34 17913 cms_Decode: cmsxrootd2 delays enstore.233531:159@fndca2b 1 /store/test/xrootd/redirector/test-file >>>> >>>> 230915 22:42:35 17939 acc_Audit: enstore.233531:159@fndca2b grant host *@fndca2b.fnal.gov stat /store/test/xrootd/redirector/test-file >>>> >>>> 230915 22:42:35 17913 cms_Decode: cmsxrootd2 sent enstore.233531:159@fndca2b 'Mrcmsxrootd-site1.fnal.gov:1094 Mrcmsxrootd-site3.fnal.gov:1094' /store/test/xrootd/redirector/test-file >>>> >>>> 230915 22:42:45 17869 acc_Audit: enstore.233544:463@fndca2b grant host *@fndca2b.fnal.gov stat /store/test/xrootd/redirector/test-file >>>> >>>> 230915 22:42:45 17913 cms_Decode: cmsxrootd2 sent enstore.233544:463@fndca2b 'Mr[2620:6a:0:8420::f8]:1094 Mr[2620:6a:0:8421::243]:1094' /store/test/xrootd/redirector/test-file >>>> >>>> >>>> ###cmsxrootd-site1### >>>> >>>> >>>> >>>> [root@cmsxrootd-site1 site]# tail -n 1000000 cmsd.log|grep test-file >>>> >>>> 230915 22:26:02 1955 manager.0:[log in to unmask] cms_do_State: /store/test/xrootd/T2_BR_SPRACE/store/user/jmsilva/1MiB-subscription-test-file.raw >>>> >>>> 230915 22:42:34 1953 manager.0:27@cmsxrootd2 cms_do_State: /store/test/xrootd/redirector/test-file >>>> >>>> 230915 22:42:34 13496 server.1580:30@cmsxrootd-site1:1096 cms_do_Have: /store/test/xrootd/redirector/test-file >>>> >>>> 230915 22:42:36 20735 enstore.233531:3840@fndca2b cms_do_Locate: /store/test/xrootd/redirector/test-file >>>> >>>> [root@cmsxrootd-site1 site]# tail -n 1000000 xrootd.log|grep test-file >>>> >>>> 230915 22:42:36 31818 acc_Audit: enstore.233531:3840@fndca2b grant gsi [log in to unmask] stat /store/test/xrootd/redirector/test-file >>>> >>>> 230915 22:42:36 1944 cms_Decode: cmsxrootd-site1 sent enstore.233531:3840@fndca2b 'Srcmsxrootd-site1.fnal.gov:1096' /store/test/xrootd/redirector/test-file >>>> >>>> ###cmsxrootd-site3### >>>> >>>> [root@cmsxrootd-site3 site]# tail -n 1000000 cmsd.log|grep test-file >>>> >>>> 230915 22:26:02 33739 manager.0:[log in to unmask] cms_do_State: /store/test/xrootd/T2_BR_SPRACE/store/user/jmsilva/1MiB-subscription-test-file.raw >>>> >>>> 230915 22:42:34 33738 manager.0:31@cmsxrootd2 cms_do_State: /store/test/xrootd/redirector/test-file >>>> >>>> 230915 22:42:34 34190 server.33696:33@cmsxrootd-site3:1092 cms_do_Have: /store/test/xrootd/redirector/test-file >>>> >>>> 230915 22:42:36 34194 enstore.233531:49@fndca2b cms_do_Locate: /store/test/xrootd/redirector/test-file >>>> >>>> [root@cmsxrootd-site3 site]# tail -n 1000000 xrootd.log|grep test-file >>>> >>>> 230915 22:42:36 33982 acc_Audit: enstore.233531:49@fndca2b grant gsi [log in to unmask] stat /store/test/xrootd/redirector/test-file >>>> >>>> 230915 22:42:36 33731 cms_Decode: cmsxrootd-site3 sent enstore.233531:49@fndca2b 'Srcmsxrootd-site3.fnal.gov:1092' /store/test/xrootd/redirector/test-file >>>> >>>> ------------ >>>> >>>> So, the one without "-m" option didn't showed up in the cmxrootd-site1/3 redirectors. Could it be possible this was due to IP addresses were sent instead of host names when not using "-m" option? >>>> >>>> Thanks again for your help on this. >>>> >>>> >>>> Regards, >>>> Yujun >>>> ________________________________ >>>> From: Andrew Hanushevsky <[log in to unmask]> >>>> Sent: Friday, September 15, 2023 7:12 PM >>>> To: Yujun Wu <[log in to unmask]> >>>> Subject: Re: Help with a xrdfs query puzzle with "xrdfs locate -d" >>>> >>>> Well, that's strange because when I try it I always get >>>> >>>> Server responded with an error: [3011] No valid location found >>>> >>>> regardless of whether -m is there or not. Could you provide another file >>>> that should be present? >>>> >>>> On Fri, 15 Sep 2023, Yujun Wu wrote: >>>> >>>>> Good afternoon. >>>>> >>>>> Could XRootD developers please help with a xrdfs puzzle? >>>>> >>>>> Bockjoo first noticed. When we query our XRootD regional redirector cmsxrootd2.fnal.gov, we were able to get locate results using "-m" option, but no results without "-m" option: >>>>> >>>>> ### xrdfs locate -d with "-m" option >>>>> >>>>> >>>>> bash-4.2$ date; xrdfs cmsxrootd2.fnal.gov:1094 locate -d -m /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root >>>>> >>>>> Fri Sep 15 21:22:52 CEST 2023 >>>>> >>>>> cmsxrootd-site3.fnal.gov:1093 Server Read >>>>> >>>>> cmsxrootd-site2.fnal.gov:1093 Server Read >>>>> >>>>> cmsxrootd-site1.fnal.gov:1093 Server Read >>>>> >>>>> cmsio5.rc.ufl.edu:1094 Server ReadWrite >>>>> >>>>> cmsio7.rc.ufl.edu:1094 Server ReadWrite >>>>> >>>>> cmsio9.rc.ufl.edu:1094 Server ReadWrite >>>>> >>>>> cmsio4.rc.ufl.edu:1094 Server ReadWrite >>>>> >>>>> cmsio3.rc.ufl.edu:1094 Server ReadWrite >>>>> >>>>> cmsio8.rc.ufl.edu:1094 Server ReadWrite >>>>> >>>>> cmsio6.rc.ufl.edu:1094 Server ReadWrite >>>>> >>>>> >>>>> ------------------- >>>>> >>>>> ### xrdfs locate -d without "-m" option >>>>> >>>>> >>>>> bash-4.2$ date; xrdfs cmsxrootd2.fnal.gov:1094 locate -d /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root >>>>> >>>>> Fri Sep 15 21:27:13 CEST 2023 >>>>> >>>>> [ERROR] Server responded with an error: [3011] No valid location found >>>>> >>>>> >>>>> -------------------- >>>>> >>>>> On the server side, the first one showed: >>>>> >>>>> 230915 14:22:53 18227 acc_Audit: yujun.12664:[log in to unmask] grant host *@lxplus734.cern.ch stat /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root >>>>> >>>>> 230915 14:22:53 17913 cms_Receive: cmsxrootd2 125 bytes on 727534553 >>>>> >>>>> 230915 14:22:53 17913 cms_Decode: cmsxrootd2 sent yujun.12664:[log in to unmask] 'Mwcmsio2.rc.ufl.edu:1094 Mrcmsxrootd-site1.fnal.gov:1094 Mrcmsxrootd-site3.fnal.gov:1094 Mrcmsxrootd-site2.fnal.gov:1094' /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root >>>>> >>>>> 230915 14:22:53 17913 cms_Receive: cmsxrootd2 32 bytes on 727535577 >>>>> >>>>> >>>>> >>>>> While the 2nd one without "-m" option showed: >>>>> >>>>> >>>>> 230915 14:27:13 27864 acc_Audit: yujun.9149:[log in to unmask] grant host *@lxplus734.cern.ch stat /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root >>>>> >>>>> 230915 14:27:13 17913 cms_Receive: cmsxrootd2 135 bytes on 743171027 >>>>> >>>>> 230915 14:27:13 17913 cms_Decode: cmsxrootd2 sent yujun.9149:[log in to unmask] 'Mw[2620:104:1f:1000:faf2:1eff:fed9:dd80]:1094 Mr[2620:6a:0:8420::f8]:1094 Mr[2620:6a:0:8421::243]:1094 Mr[2620:6a:0:8420::f9]:1094' /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root >>>>> >>>>> 230915 14:27:13 17913 cms_Receive: cmsxrootd2 4 bytes on 743172051 >>>>> >>>>> >>>>> =================== >>>>> >>>>> >>>>> As you can see, the successful one sent multiple hostnames while the failed one sent multiple IP addresses: >>>>> >>>>> 230915 14:27:13 27864 acc_Audit: yujun.9149:[log in to unmask] grant host *@lxplus734.cern.ch stat /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root >>>>> >>>>> 230915 14:27:13 17913 cms_Receive: cmsxrootd2 135 bytes on 743171027 >>>>> >>>>> 230915 14:27:13 17913 cms_Decode: cmsxrootd2 sent yujun.9149:[log in to unmask] 'Mw[2620:104:1f:1000:faf2:1eff:fed9:dd80]:1094 Mr[2620:6a:0:8420::f8]:1094 Mr[2620:6a:0:8421::243]:1094 Mr[2620:6a:0:8420::f9]:1094' /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root >>>>> >>>>> 230915 14:27:13 17913 cms_Receive: cmsxrootd2 4 bytes on 743172051 >>>>> >>>>> >>>>> ------------------------------------ >>>>> >>>>> >>>>> In contrast, another USCMS regional redirector xrootd.unl.edu could return results with and without "-m" option: >>>>> >>>>> >>>>> bash-4.2$ date; xrdfs xrootd.unl.edu:1094 locate -d /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root >>>>> >>>>> Fri Sep 15 21:38:17 CEST 2023 >>>>> >>>>> cmsxrootd-site3.fnal.gov:1093 Server Read >>>>> >>>>> cmsio5.rc.ufl.edu:1094 Server ReadWrite >>>>> >>>>> cmsio7.rc.ufl.edu:1094 Server ReadWrite >>>>> >>>>> cmsio9.rc.ufl.edu:1094 Server ReadWrite >>>>> >>>>> cmsio4.rc.ufl.edu:1094 Server ReadWrite >>>>> >>>>> cmsio3.rc.ufl.edu:1094 Server ReadWrite >>>>> >>>>> cmsio8.rc.ufl.edu:1094 Server ReadWrite >>>>> >>>>> cmsio6.rc.ufl.edu:1094 Server ReadWrite >>>>> >>>>> cmsxrootd-site1.fnal.gov:1093 Server Read >>>>> >>>>> cmsxrootd-site2.fnal.gov:1093 Server Read >>>>> >>>>> >>>>> I was told their log file showed hostnames were sent even without "-m" option. >>>>> >>>>> >>>>> >>>>> For cmsxrootd2.fnal.gov XRootD redirector, I added >>>>> >>>>> >>>>> xrd.network dnr >>>>> >>>>> >>>>> option in our server side xrootd.cfg but it seems IP addresses still were sent. Could you please tell how I could force hostnames instead of IP addresses to be sent >>>>> >>>>> when running "xrdfs cmsxrootd2.fnal.gov:1094 locate -d" command? >>>>> >>>>> >>>>> Thanks in advance for any advice on this. >>>>> >>>>> >>>>> >>>>> Regards, >>>>> >>>>> Yujun >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> ######################################################################## >>>>> Use REPLY-ALL to reply to list >>>>> >>>>> To unsubscribe from the XROOTD-L list, click the following link: >>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__listserv.slac.stanford.edu_cgi-2Dbin_wa-3FSUBED1-3DXROOTD-2DL-26A-3D1&d=DwIBAg&c=gRgGjJ3BkIsb5y6s49QqsA&r=bFk3iAgykb1xfdm5362Q9w&m=u4ZeuNN02gPONHd_hstIHONjzndQrVW3s6axSEhW_-qx4IqYlkKlkgW5N3y0N0HY&s=HTkD3kSVa8SHsVmhFbg5SevVFe2CM4HmftLiX3yPCbg&e= <https://urldefense.proofpoint.com/v2/url?u=https-3A__listserv.slac.stanford.edu_cgi-2Dbin_wa-3FSUBED1-3DXROOTD-2DL-26A-3D1&d=DwIBAg&c=gRgGjJ3BkIsb5y6s49QqsA&r=bFk3iAgykb1xfdm5362Q9w&m=u4ZeuNN02gPONHd_hstIHONjzndQrVW3s6axSEhW_-qx4IqYlkKlkgW5N3y0N0HY&s=HTkD3kSVa8SHsVmhFbg5SevVFe2CM4HmftLiX3yPCbg&e=> <https://urldefense.proofpoint.com/v2/url?u=https-3A__listserv.slac.stanford.edu_cgi-2Dbin_wa-3FSUBED1-3DXROOTD-2DL-26A-3D1&d=DwIBAg&c=gRgGjJ3BkIsb5y6s49QqsA&r=bFk3iAgykb1xfdm5362Q9w&m=u4ZeuNN02gPONHd_hstIHONjzndQrVW3s6axSEhW_-qx4IqYlkKlkgW5N3y0N0HY&s=HTkD3kSVa8SHsVmhFbg5SevVFe2CM4HmftLiX3yPCbg&e= <https://urldefense.proofpoint.com/v2/url?u=https-3A__listserv.slac.stanford.edu_cgi-2Dbin_wa-3FSUBED1-3DXROOTD-2DL-26A-3D1&d=DwIBAg&c=gRgGjJ3BkIsb5y6s49QqsA&r=bFk3iAgykb1xfdm5362Q9w&m=u4ZeuNN02gPONHd_hstIHONjzndQrVW3s6axSEhW_-qx4IqYlkKlkgW5N3y0N0HY&s=HTkD3kSVa8SHsVmhFbg5SevVFe2CM4HmftLiX3yPCbg&e=>> >>>>> >>> -------------------------------------------------------------------------------- >>> >>> Use REPLY-ALL to reply to list >>> >>> To unsubscribe from the XROOTD-L list, click the following link: >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__listserv.slac.stanford.edu_cgi-2Dbin_wa-3FSUBED1-3DXROOTD-2DL-26A-3D1&d=DwIDaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=bFk3iAgykb1xfdm5362Q9w&m=8DWcaSpkMu8Mk-iLymsl2CLja72X84_QpbAai4GwqH1aR8ZMjmsvEg59qsQ3wOb7&s=KEre7NbD46BvBgfpo7nEWYdPDGVfF9G1Y3VE2IN6YYs&e= <https://urldefense.proofpoint.com/v2/url?u=https-3A__listserv.slac.stanford.edu_cgi-2Dbin_wa-3FSUBED1-3DXROOTD-2DL-26A-3D1&d=DwIDaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=bFk3iAgykb1xfdm5362Q9w&m=8DWcaSpkMu8Mk-iLymsl2CLja72X84_QpbAai4GwqH1aR8ZMjmsvEg59qsQ3wOb7&s=KEre7NbD46BvBgfpo7nEWYdPDGVfF9G1Y3VE2IN6YYs&e=> >>> <https://urldefense.com/v3/__https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1__;!!Mih3wA!FSW9Tek3rz3nD0Ff-wL6cRUEPKE83ZXADszSIAkXo3sLVs_pQ8ziG7k59rdA2THK2UmsqQnv0uiBTZggmZOs-EfTLl0w3tZWBnohgSwj68vI8L5j$ <https://urldefense.com/v3/__https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1__;!!Mih3wA!FSW9Tek3rz3nD0Ff-wL6cRUEPKE83ZXADszSIAkXo3sLVs_pQ8ziG7k59rdA2THK2UmsqQnv0uiBTZggmZOs-EfTLl0w3tZWBnohgSwj68vI8L5j$>> >>> >> ######################################################################## >> Use REPLY-ALL to reply to list >> >> To unsubscribe from the XROOTD-L list, click the following link: >> https://urldefense.proofpoint.com/v2/url?u=https-3A__listserv.slac.stanford.edu_cgi-2Dbin_wa-3FSUBED1-3DXROOTD-2DL-26A-3D1&d=DwIDaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=bFk3iAgykb1xfdm5362Q9w&m=8DWcaSpkMu8Mk-iLymsl2CLja72X84_QpbAai4GwqH1aR8ZMjmsvEg59qsQ3wOb7&s=KEre7NbD46BvBgfpo7nEWYdPDGVfF9G1Y3VE2IN6YYs&e= <https://urldefense.proofpoint.com/v2/url?u=https-3A__listserv.slac.stanford.edu_cgi-2Dbin_wa-3FSUBED1-3DXROOTD-2DL-26A-3D1&d=DwIDaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=bFk3iAgykb1xfdm5362Q9w&m=8DWcaSpkMu8Mk-iLymsl2CLja72X84_QpbAai4GwqH1aR8ZMjmsvEg59qsQ3wOb7&s=KEre7NbD46BvBgfpo7nEWYdPDGVfF9G1Y3VE2IN6YYs&e=> ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1