 |
 |
Hi, I'm seeing lots of errors crop up from our Xrootd redirector. See an example below from a cmsd. Lots of servers have the file, yet the prep message indicates the file failed. In some cases, if the client tries to open the file immediately after prep'ing it, it gets an error message. Is there a knob available to continue to delay the client if there's not yet a result available? If you see the timestamps below, the client would have gotten a redirect if it had waited another 30s or so. Brian [bbockelm@xrootd-itb ~]$ grep 12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root /var/log/xrootd/cmsd.log 110303 12:17:28 15775 cms125.17871:[log in to unmask] do_PrepAdd: parms: b1ef5d81:0446.4d6fd626:903 * 0 rq /store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root 110303 12:17:28 15775 Select seeking /store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root 110303 12:17:28 15775 cms125.17871:[log in to unmask] do_Select: /store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root 110303 12:17:28 15775 WT4File rc=0 path=/store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root 110303 12:17:33 15775 SelPrep: /store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root failed; No servers are available to read the file. 110303 12:17:34 15775 cms125.17871:[log in to unmask] do_Select: /store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root 110303 12:17:34 15775 cms125.17871:[log in to unmask] do_Select: failed; No servers are available to read the file. /store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root 110303 12:17:46 15775 server.24015:23@red-gridftp4:1094 do_Have: /store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root 110303 12:17:50 15775 server.12835:21@red-gridftp11:1094 do_Have: /store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root 110303 12:17:51 15775 server.13920:22@red-gridftp9:1094 do_Have: /store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root 110303 12:17:52 15775 server.2282:26@red-gridftp10:1094 do_Have: /store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root 110303 12:17:52 15775 server.12574:25@red-gridftp1:1094 do_Have: /store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root 110303 12:17:52 15775 server.1981:27@red-gridftp2:1094 do_Have: /store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root 110303 12:17:52 15775 server.28934:16@red-gridftp5:1094 do_Have: /store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root 110303 12:17:52 15775 server.3999:20@red-gridftp12:1094 do_Have: /store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root 110303 12:17:52 15775 server.31009:19@red-gridftp6:1094 do_Have: /store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root 110303 12:17:52 15775 server.16746:18@red-gridftp7:1094 do_Have: /store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root 110303 12:17:52 15775 server.12739:17@red-gridftp8:1094 do_Have: /store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root 110303 12:17:52 15775 server.21911:24@red-gridftp3:1094 do_Have: /store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root [bbockelm@xrootd-itb ~]$ grep 12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root /var/log/xrootd/xrootd.log 110303 12:17:28 15632 acc_Audit: cms125.17871:[log in to unmask] grant host *@wngw.ifca.es read /store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root 110303 12:17:28 15632 Prepare Finder: Sending xrootd-itb.unl.edu b1ef5d81:0446.4d6fd626:903 /store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root 110303 12:17:28 15632 cms125.17871:[log in to unmask] ofs_open: 0-644 fn=/store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root 110303 12:17:29 15632 Decode xrootd-itb delays cms125.17871:[log in to unmask] 5 /store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root 110303 12:17:34 15632 cms125.17871:[log in to unmask] ofs_open: 0-644 fn=/store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root 110303 12:17:34 15632 Decode xrootd-itb gave cms125.17871:[log in to unmask] err -2 'No servers are available to read the file.' /store/test/xrootd/T2_US_Nebraska/store/data/Run2010B/Mu/AOD/Nov4ReReco_v1/0001/12A4A7E2-02EA-DF11-8A5B-E0CB4E29C4D0.root