One more thing try doing a locate against cmsxrootd-site1.fnal.gov:1094
which in theory has the file.
On Sat, 16 Sep 2023, Yujun Wu wrote:
> Hi Andrew,
>
> Thank you for your help. Currently, CMS XRootD redirectors only allow access with voms proxy from /cms group.
>
> It seems all the files I tried have this issue. The following are two examples:
>
>
> [enstore@fndca2b ~]$ date; xrdfs cmsxrootd2.fnal.gov:1094 locate -d -m /store/test/xrootd/T1_US_FNAL/store/mc/SAM/GenericTTbar/AODSIM/CMSSW_9_2_6_91X_mcRun1_realistic_v2-v1/00000/AE237916-5D76-E711-A48C-FA163EEEBFED.root
>
> Fri Sep 15 22:41:40 CDT 2023
>
> cmsxrootd-site3.fnal.gov:1093 Server Read
>
> cmsxrootd-site1.fnal.gov:1093 Server Read
>
> cmsxrootd-site2.fnal.gov:1093 Server Read
>
> [enstore@fndca2b ~]$ date; xrdfs cmsxrootd2.fnal.gov:1094 locate -d /store/test/xrootd/T1_US_FNAL/store/mc/SAM/GenericTTbar/AODSIM/CMSSW_9_2_6_91X_mcRun1_realistic_v2-v1/00000/AE237916-5D76-E711-A48C-FA163EEEBFED.root
>
> Fri Sep 15 22:41:56 CDT 2023
>
> [ERROR] Server responded with an error: [3011] No valid location found
>
>
> -------------
>
> [enstore@fndca2b ~]$ date; xrdfs cmsxrootd2.fnal.gov:1094 locate -d -m /store/test/xrootd/redirector/test-file
>
> Fri Sep 15 22:42:34 CDT 2023
>
> cmsxrootd-site3.fnal.gov:1092 Server Read
>
> cmsxrootd-site1.fnal.gov:1096 Server Read
>
> [enstore@fndca2b ~]$ date; xrdfs cmsxrootd2.fnal.gov:1094 locate -d /store/test/xrootd/redirector/test-file
>
> Fri Sep 15 22:42:45 CDT 2023
>
> [ERROR] Server responded with an error: [3011] No valid location found
>
> -------------
> For the 2nd file, the log files showed:
> ###cmsxrootd2###
>
> [root@cmsxrootd2 tmp]# tail -n 100000 cmsd.log|grep test-file
>
> [root@cmsxrootd2 tmp]# tail -n 100000 xrootd.log|grep test-file
>
> 230915 22:42:34 17939 acc_Audit: enstore.233531:159@fndca2b grant host *@fndca2b.fnal.gov stat /store/test/xrootd/redirector/test-file
>
> 230915 22:42:34 17913 cms_Decode: cmsxrootd2 delays enstore.233531:159@fndca2b 1 /store/test/xrootd/redirector/test-file
>
> 230915 22:42:35 17939 acc_Audit: enstore.233531:159@fndca2b grant host *@fndca2b.fnal.gov stat /store/test/xrootd/redirector/test-file
>
> 230915 22:42:35 17913 cms_Decode: cmsxrootd2 sent enstore.233531:159@fndca2b 'Mrcmsxrootd-site1.fnal.gov:1094 Mrcmsxrootd-site3.fnal.gov:1094' /store/test/xrootd/redirector/test-file
>
> 230915 22:42:45 17869 acc_Audit: enstore.233544:463@fndca2b grant host *@fndca2b.fnal.gov stat /store/test/xrootd/redirector/test-file
>
> 230915 22:42:45 17913 cms_Decode: cmsxrootd2 sent enstore.233544:463@fndca2b 'Mr[2620:6a:0:8420::f8]:1094 Mr[2620:6a:0:8421::243]:1094' /store/test/xrootd/redirector/test-file
>
>
> ###cmsxrootd-site1###
>
>
>
> [root@cmsxrootd-site1 site]# tail -n 1000000 cmsd.log|grep test-file
>
> 230915 22:26:02 1955 manager.0:
[log in to unmask] cms_do_State: /store/test/xrootd/T2_BR_SPRACE/store/user/jmsilva/1MiB-subscription-test-file.raw
>
> 230915 22:42:34 1953 manager.0:27@cmsxrootd2 cms_do_State: /store/test/xrootd/redirector/test-file
>
> 230915 22:42:34 13496 server.1580:30@cmsxrootd-site1:1096 cms_do_Have: /store/test/xrootd/redirector/test-file
>
> 230915 22:42:36 20735 enstore.233531:3840@fndca2b cms_do_Locate: /store/test/xrootd/redirector/test-file
>
> [root@cmsxrootd-site1 site]# tail -n 1000000 xrootd.log|grep test-file
>
> 230915 22:42:36 31818 acc_Audit: enstore.233531:3840@fndca2b grant gsi
[log in to unmask] stat /store/test/xrootd/redirector/test-file
>
> 230915 22:42:36 1944 cms_Decode: cmsxrootd-site1 sent enstore.233531:3840@fndca2b 'Srcmsxrootd-site1.fnal.gov:1096' /store/test/xrootd/redirector/test-file
>
> ###cmsxrootd-site3###
>
> [root@cmsxrootd-site3 site]# tail -n 1000000 cmsd.log|grep test-file
>
> 230915 22:26:02 33739 manager.0:
[log in to unmask] cms_do_State: /store/test/xrootd/T2_BR_SPRACE/store/user/jmsilva/1MiB-subscription-test-file.raw
>
> 230915 22:42:34 33738 manager.0:31@cmsxrootd2 cms_do_State: /store/test/xrootd/redirector/test-file
>
> 230915 22:42:34 34190 server.33696:33@cmsxrootd-site3:1092 cms_do_Have: /store/test/xrootd/redirector/test-file
>
> 230915 22:42:36 34194 enstore.233531:49@fndca2b cms_do_Locate: /store/test/xrootd/redirector/test-file
>
> [root@cmsxrootd-site3 site]# tail -n 1000000 xrootd.log|grep test-file
>
> 230915 22:42:36 33982 acc_Audit: enstore.233531:49@fndca2b grant gsi
[log in to unmask] stat /store/test/xrootd/redirector/test-file
>
> 230915 22:42:36 33731 cms_Decode: cmsxrootd-site3 sent enstore.233531:49@fndca2b 'Srcmsxrootd-site3.fnal.gov:1092' /store/test/xrootd/redirector/test-file
>
> ------------
>
> So, the one without "-m" option didn't showed up in the cmxrootd-site1/3 redirectors. Could it be possible this was due to IP addresses were sent instead of host names when not using "-m" option?
>
> Thanks again for your help on this.
>
>
> Regards,
> Yujun
> ________________________________
> From: Andrew Hanushevsky <
[log in to unmask]>
> Sent: Friday, September 15, 2023 7:12 PM
> To: Yujun Wu <
[log in to unmask]>
> Subject: Re: Help with a xrdfs query puzzle with "xrdfs locate -d"
>
> Well, that's strange because when I try it I always get
>
> Server responded with an error: [3011] No valid location found
>
> regardless of whether -m is there or not. Could you provide another file
> that should be present?
>
> On Fri, 15 Sep 2023, Yujun Wu wrote:
>
>> Good afternoon.
>>
>> Could XRootD developers please help with a xrdfs puzzle?
>>
>> Bockjoo first noticed. When we query our XRootD regional redirector cmsxrootd2.fnal.gov, we were able to get locate results using "-m" option, but no results without "-m" option:
>>
>> ### xrdfs locate -d with "-m" option
>>
>>
>> bash-4.2$ date; xrdfs cmsxrootd2.fnal.gov:1094 locate -d -m /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root
>>
>> Fri Sep 15 21:22:52 CEST 2023
>>
>> cmsxrootd-site3.fnal.gov:1093 Server Read
>>
>> cmsxrootd-site2.fnal.gov:1093 Server Read
>>
>> cmsxrootd-site1.fnal.gov:1093 Server Read
>>
>> cmsio5.rc.ufl.edu:1094 Server ReadWrite
>>
>> cmsio7.rc.ufl.edu:1094 Server ReadWrite
>>
>> cmsio9.rc.ufl.edu:1094 Server ReadWrite
>>
>> cmsio4.rc.ufl.edu:1094 Server ReadWrite
>>
>> cmsio3.rc.ufl.edu:1094 Server ReadWrite
>>
>> cmsio8.rc.ufl.edu:1094 Server ReadWrite
>>
>> cmsio6.rc.ufl.edu:1094 Server ReadWrite
>>
>>
>> -------------------
>>
>> ### xrdfs locate -d without "-m" option
>>
>>
>> bash-4.2$ date; xrdfs cmsxrootd2.fnal.gov:1094 locate -d /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root
>>
>> Fri Sep 15 21:27:13 CEST 2023
>>
>> [ERROR] Server responded with an error: [3011] No valid location found
>>
>>
>> --------------------
>>
>> On the server side, the first one showed:
>>
>> 230915 14:22:53 18227 acc_Audit: yujun.12664:
[log in to unmask] grant host *@lxplus734.cern.ch stat /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root
>>
>> 230915 14:22:53 17913 cms_Receive: cmsxrootd2 125 bytes on 727534553
>>
>> 230915 14:22:53 17913 cms_Decode: cmsxrootd2 sent yujun.12664:
[log in to unmask] 'Mwcmsio2.rc.ufl.edu:1094 Mrcmsxrootd-site1.fnal.gov:1094 Mrcmsxrootd-site3.fnal.gov:1094 Mrcmsxrootd-site2.fnal.gov:1094' /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root
>>
>> 230915 14:22:53 17913 cms_Receive: cmsxrootd2 32 bytes on 727535577
>>
>>
>>
>> While the 2nd one without "-m" option showed:
>>
>>
>> 230915 14:27:13 27864 acc_Audit: yujun.9149:
[log in to unmask] grant host *@lxplus734.cern.ch stat /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root
>>
>> 230915 14:27:13 17913 cms_Receive: cmsxrootd2 135 bytes on 743171027
>>
>> 230915 14:27:13 17913 cms_Decode: cmsxrootd2 sent yujun.9149:
[log in to unmask] 'Mw[2620:104:1f:1000:faf2:1eff:fed9:dd80]:1094 Mr[2620:6a:0:8420::f8]:1094 Mr[2620:6a:0:8421::243]:1094 Mr[2620:6a:0:8420::f9]:1094' /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root
>>
>> 230915 14:27:13 17913 cms_Receive: cmsxrootd2 4 bytes on 743172051
>>
>>
>> ===================
>>
>>
>> As you can see, the successful one sent multiple hostnames while the failed one sent multiple IP addresses:
>>
>> 230915 14:27:13 27864 acc_Audit: yujun.9149:
[log in to unmask] grant host *@lxplus734.cern.ch stat /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root
>>
>> 230915 14:27:13 17913 cms_Receive: cmsxrootd2 135 bytes on 743171027
>>
>> 230915 14:27:13 17913 cms_Decode: cmsxrootd2 sent yujun.9149:
[log in to unmask] 'Mw[2620:104:1f:1000:faf2:1eff:fed9:dd80]:1094 Mr[2620:6a:0:8420::f8]:1094 Mr[2620:6a:0:8421::243]:1094 Mr[2620:6a:0:8420::f9]:1094' /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root
>>
>> 230915 14:27:13 17913 cms_Receive: cmsxrootd2 4 bytes on 743172051
>>
>>
>> ------------------------------------
>>
>>
>> In contrast, another USCMS regional redirector xrootd.unl.edu could return results with and without "-m" option:
>>
>>
>> bash-4.2$ date; xrdfs xrootd.unl.edu:1094 locate -d /store/mc/RunIIFall17FSPrePremix/Neutrino_E-10_gun/GEN-SIM-DIGI-RAW/PUFor2016_94X_mcRun2_asymptotic_v3-v1/2810000/7AB76D28-6C1D-EE11-B331-FA163EEB94EB.root
>>
>> Fri Sep 15 21:38:17 CEST 2023
>>
>> cmsxrootd-site3.fnal.gov:1093 Server Read
>>
>> cmsio5.rc.ufl.edu:1094 Server ReadWrite
>>
>> cmsio7.rc.ufl.edu:1094 Server ReadWrite
>>
>> cmsio9.rc.ufl.edu:1094 Server ReadWrite
>>
>> cmsio4.rc.ufl.edu:1094 Server ReadWrite
>>
>> cmsio3.rc.ufl.edu:1094 Server ReadWrite
>>
>> cmsio8.rc.ufl.edu:1094 Server ReadWrite
>>
>> cmsio6.rc.ufl.edu:1094 Server ReadWrite
>>
>> cmsxrootd-site1.fnal.gov:1093 Server Read
>>
>> cmsxrootd-site2.fnal.gov:1093 Server Read
>>
>>
>> I was told their log file showed hostnames were sent even without "-m" option.
>>
>>
>>
>> For cmsxrootd2.fnal.gov XRootD redirector, I added
>>
>>
>> xrd.network dnr
>>
>>
>> option in our server side xrootd.cfg but it seems IP addresses still were sent. Could you please tell how I could force hostnames instead of IP addresses to be sent
>>
>> when running "xrdfs cmsxrootd2.fnal.gov:1094 locate -d" command?
>>
>>
>> Thanks in advance for any advice on this.
>>
>>
>>
>> Regards,
>>
>> Yujun
>>
>>
>>
>>
>>
>>
>> ########################################################################
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the XROOTD-L list, click the following link:
>>
https://urldefense.proofpoint.com/v2/url?u=https-3A__listserv.slac.stanford.edu_cgi-2Dbin_wa-3FSUBED1-3DXROOTD-2DL-26A-3D1&d=DwIBAg&c=gRgGjJ3BkIsb5y6s49QqsA&r=bFk3iAgykb1xfdm5362Q9w&m=u4ZeuNN02gPONHd_hstIHONjzndQrVW3s6axSEhW_-qx4IqYlkKlkgW5N3y0N0HY&s=HTkD3kSVa8SHsVmhFbg5SevVFe2CM4HmftLiX3yPCbg&e=
>>
>