 |
 |
Hello! Failures in file transfer cause job abortions. To understand what is happening I conducted an experiment were I tested the command xrdcp -d 1 -f <file> /dev/null 101 times for 2 files, in an attempt to identify servers guilty of transfer failures. The results are as follows: File 1. Transfer failed in 4 cases, always associated with host se01.indiacms.res.in File 2. Transfer failed in 9 cases, always associated with host pool0x.ifca.es The full results are shown in the attachments. I suppose the problem identified is related to our previous discussion, where we identified incompatible xrootd versions as being the cause. Viesturs A 2015-08-23 01:33, Andrew Hanushevsky escreveu: > The only practical solution is to contact the administrator. They are > ultimately reponsible for what happens at the site and we really can't > do > much from the outside. > > Andy > > On Thu, 20 Aug 2015, Viesturs Veckalns wrote: > >> What could be a practical solution to this problem? >> Viesturs >> >> A 2015-08-20 16:21, xrootd-dev escreveu: >>> Dear Viesturs, >>> >>> I am referring to the end servers, for example maite.iihe.ac.be . >>> I guess the only way is to contact the administrator of such a > server. >>> >>> The version installed on your internal machines look fine. >>> >>> Gerardo GANIS >>> CERN, PH Dept, SFT group >>> [log in to unmask] >>> tel/cell: (+41) 22 767 6439 / 75 411 1128 >>> >>> On 20 Aug 2015, at 17:09, Viesturs Veckalns > <[log in to unmask]> >>> wrote: >>> >>>> Dear Gerardo, >>>> The packages installed on our internal xroot servers are the >>> following: >>>> [root@xroot ~]# rpm -qa|egrep '(xroot|openssl)'|sort >>>> cms-xrootd-1.2-9.osg32.el6.noarch >>>> globus-gsi-openssl-error-3.5-1.osg32.el6.x86_64 >>>> globus-openssl-module-4.6-1.osg32.el6.x86_64 >>>> openssl098e-0.9.8e-18.el6_5.2.x86_64 >>>> openssl-1.0.1e-30.el6.11.x86_64 >>>> xrootd-4.2.1-1.osg32.el6.x86_64 >>>> xrootd-client-4.2.1-1.osg32.el6.x86_64 >>>> xrootd-client-devel-4.2.1-1.osg32.el6.x86_64 >>>> xrootd-client-libs-4.2.1-1.osg32.el6.x86_64 >>>> xrootd-cmstfc-1.5.1-10.osg32.el6.x86_64 >>>> xrootd-compat-3.3.6-1.el6.x86_64 >>>> xrootd-compat-client-libs-3.3.6-1.el6.x86_64 >>>> xrootd-compat-libs-3.3.6-1.el6.x86_64 >>>> xrootd-compat-server-libs-3.3.6-1.el6.x86_64 >>>> xrootd-devel-4.2.1-1.osg32.el6.x86_64 >>>> xrootd-lcmaps-0.0.7-11.osg32.el6.x86_64 >>>> xrootd-libs-4.2.1-1.osg32.el6.x86_64 >>>> xrootd-selinux-4.2.1-1.osg32.el6.noarch >>>> xrootd-server-4.2.1-1.osg32.el6.x86_64 >>>> xrootd-server-libs-4.2.1-1.osg32.el6.x86_64 >>>> >>>> and on the user interface: >>>> [root@ui6-cms01 ~]# rpm -qa|egrep '(xroot|openssl)'|sort >>>> globus-gsi-openssl-error-2.1-10.el6.i686 >>>> globus-gsi-openssl-error-2.1-10.el6.x86_64 >>>> globus-openssl-module-3.3-2.el6.i686 >>>> globus-openssl-module-3.3-2.el6.x86_64 >>>> nordugrid-arc-plugins-xrootd-4.2.0-1.el6.x86_64 >>>> openssl098e-0.9.8e-18.el6_5.2.i686 >>>> openssl098e-0.9.8e-18.el6_5.2.x86_64 >>>> openssl-1.0.1e-30.el6.11.i686 >>>> openssl-1.0.1e-30.el6.11.x86_64 >>>> xmlsec1-openssl-1.2.20-4.el6.x86_64 >>>> xrootd-client-libs-4.1.1-1.el6.x86_64 >>>> xrootd-compat-client-libs-3.3.6-1.el6.x86_64 >>>> xrootd-compat-libs-3.3.6-1.el6.x86_64 >>>> xrootd-libs-4.1.1-1.el6.x86_64 >>>> >>>> Viesturs >>>> >>>> A 2015-08-20 10:13, xrootd-dev escreveu: >>>>> Dear Viesturs, >>>>> >>>>> The second server (maite.iihe.ac.be) seems to be running a very >>> old >>>>> version of XRootD, older than 4 years. >>>>> While we try to be backward compatible, on such a long periods > and >>>>> version ranges is very difficult. In particular, >>>>> the version run is before the devs to openssl 1.x, which >>> introduced a >>>>> different hash technology. >>>>> >>>>> Could you find out which version is exactly run on the servers >>> that >>>>> give the problem? >>>>> >>>>> Gerardo GANIS >>>>> CERN, PH Dept, SFT group >>>>> [log in to unmask] >>>>> tel/cell: (+41) 22 767 6439 / 75 411 1128 >>>>> >>>>> On 19 Aug 2015, at 17:46, Viesturs Veckalns >>> <[log in to unmask]> >>>>> wrote: >>>>> >>>>>> Dear Gerardo, >>>>>> I ran export XrdSecDEBUG=2 >>>>>> >>>>>> and subsequently got 2 different results: >>>>>> 1. I don't know if the process ran well, but the file apparently >>> was >>>>> >>>>>> read in the end - see testfile.txt. >>>>>> 2. The same error as before - Authentication with gsi failed: >>> Your >>>>>> certificate's issuer is not trusted, that ended with Run: > [FATAL] >>>>>> Redirect limit has been reached - see testfile2.txt >>>>>> >>>>>> I think LIP CA certificate is in the directory >>>>>> /etc/grid-security/certificates because it contains a series of >>>>> files: >>>>>> >>>>>> LIPCA.crl_url >>>>>> LIPCA.info >>>>>> LIPCA.namespaces >>>>>> LIPCA.pem >>>>>> LIPCA.signing_policy >>>>>> >>>>>> Cheers, >>>>>> Viesturs >>>>>> >>>>>> >>>>>> A 2015-08-19 16:04, xrootd-dev escreveu: >>>>>>> Dear Viesturs, >>>>>>> >>>>>>> Can you set >>>>>>> >>>>>>> $ export XrdSecDEBUG=2 >>>>>>> >>>>>>> and run it again and post the result? >>>>>>> >>>>>>> Is the LIP CA certificate in the relevant CA directory, e.g. >>>>>>> /etc/grid-security/certificates ? >>>>>>> >>>>>>> Gerardo GANIS >>>>>>> CERN, PH Dept, SFT group >>>>>>> [log in to unmask] >>>>>>> tel/cell: (+41) 22 767 6439 / 75 411 1128 >>>>>>> >>>>>>> On 19 Aug 2015, at 15:24, Viesturs Veckalns >>>>> <[log in to unmask]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hello! >>>>>>>> I have a grid certificate issued by LIP CA, that is correctly >>>>> mapped >>>>>>> with CERN SiteDB. >>>>>>>> Unfortunately, it is impossible to receive files over grid >>>>>>> (certificate's issuer is not trusted being cited as a reason). >>> A >>>>>>> complete debug output is copied below. >>>>>>>> >>>>>>>> Our IT service thinks it is a problem with the xrootd service. >>>>>>>> Could you help? >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Viesturs >>>>>>>> >>>>>>>> xrdcp -d 1 -f >>>>>>> >>>>> >>> > root://cms-xrd-global.cern.ch//store/mc/RunIISpring15DR74/TT_TuneCUETP8M1_13TeV-powheg-pythia8/MINIAODSIM/Asympt50ns_MCRUN2_74_V9A-v4/10000/00D2A247-2910-E511-9F3D-0CC47A4DEDD2.root >>>>>>> /dev/null >>>>>>>> [2015-08-19 14:13:13.772928 +0100][Error ][XRootDTransport ] >>>>>>> [maite.iihe.ac.be:1094 #0.0] Authentication with gsi failed: >>> Your >>>>>>> certificate's issuer is not trusted. >>>>>>>> [2015-08-19 14:13:13.773148 +0100][Error ][XRootDTransport ] >>>>>>> [maite.iihe.ac.be:1094 #0.0] No protocols left to try >>>>>>>> [2015-08-19 14:13:13.773181 +0100][Error ][AsyncSock ] >>>>>>> [maite.iihe.ac.be:1094 #0.0] Socket error while handshaking: >>>>> [FATAL] >>>>>>> Auth failed >>>>>>>> [2015-08-19 14:13:13.773273 +0100][Error ][PostMaster ] >>>>>>> [maite.iihe.ac.be:1094 #0] Unable to recover: [FATAL] Auth >>> failed. >>>>>>>> [2015-08-19 14:13:13.773299 +0100][Error ][XRootD ] >>>>>>> [maite.iihe.ac.be:1094] Impossible to send message kXR_open >>> (file: >>>>>>> >>>>> >>> > /store/mc/RunIISpring15DR74/TT_TuneCUETP8M1_13TeV-powheg-pythia8/MINIAODSIM/Asympt50ns_MCRUN2_74_V9A-v4/10000/00D2A247-2910-E511-9F3D-0CC47A4DEDD2.root, >>>>>>> mode: 00, flags: kXR_open_read kXR_async kXR_retstat ). Trying >>> to >>>>>>> recover. >>>>>>>> [2015-08-19 14:13:13.773327 +0100][Error ][XRootD ] >>>>>>> [maite.iihe.ac.be:1094] Handling error while processing >>> kXR_open >>>>>>> (file: >>>>>>> >>>>> >>> > /store/mc/RunIISpring15DR74/TT_TuneCUETP8M1_13TeV-powheg-pythia8/MINIAODSIM/Asympt50ns_MCRUN2_74_V9A-v4/10000/00D2A247-2910-E511-9F3D-0CC47A4DEDD2.root, >>>>>>> mode: 00, flags: kXR_open_read kXR_async kXR_retstat ): [FATAL] >>>>> Auth >>>>>>> failed. >>>>>>>> [2015-08-19 14:13:14.111618 +0100][Error ][XRootD ] >>>>>>> [maite.iihe.ac.be:1094] Handling error while processing >>> kXR_open >>>>>>> (file: >>>>>>> >>>>> >>> > /store/mc/RunIISpring15DR74/TT_TuneCUETP8M1_13TeV-powheg-pythia8/MINIAODSIM/Asympt50ns_MCRUN2_74_V9A-v4/10000/00D2A247-2910-E511-9F3D-0CC47A4DEDD2.root?tried=maite.iihe.ac.be,maite.iihe.ac.be,xrootd-redic.pi.infn.it, >>>>>>> mode: 00, flags: kXR_open_read kXR_async kXR_retstat ): [FATAL] >>>>> Auth >>>>>>> failed. >>>>>>>> [2015-08-19 14:13:14.949391 +0100][Error ][XRootDTransport ] >>>>>>> [dcache-cms-xrootd.desy.de:1094 #0.0] Authentication with gsi >>>>> failed: >>>>>>> Your certificate's issuer is not trusted. >>>>>>>> [2015-08-19 14:13:14.949536 +0100][Error ][XRootDTransport ] >>>>>>> [dcache-cms-xrootd.desy.de:1094 #0.0] No protocols left to try >>>>>>>> [2015-08-19 14:13:14.949565 +0100][Error ][AsyncSock ] >>>>>>> [dcache-cms-xrootd.desy.de:1094 #0.0] Socket error while >>>>> handshaking: >>>>>>> [FATAL] Auth failed >>>>>>>> [2015-08-19 14:13:14.949644 +0100][Error ][PostMaster ] >>>>>>> [dcache-cms-xrootd.desy.de:1094 #0] Unable to recover: [FATAL] >>>>> Auth >>>>>>> failed. >>>>>>>> [2015-08-19 14:13:14.949662 +0100][Error ][XRootD ] >>>>>>> [dcache-cms-xrootd.desy.de:1094] Impossible to send message >>>>> kXR_open >>>>>>> (file: >>>>>>> >>>>> >>> > /store/mc/RunIISpring15DR74/TT_TuneCUETP8M1_13TeV-powheg-pythia8/MINIAODSIM/Asympt50ns_MCRUN2_74_V9A-v4/10000/00D2A247-2910-E511-9F3D-0CC47A4DEDD2.root?tried=maite.iihe.ac.be,maite.iihe.ac.be,xrootd-redic.pi.infn.it,maite.iihe.ac.be,maite.iihe.ac.be,xrootd.ba.infn.it, >>>>>>> mode: 00, flags: kXR_open_read kXR_async kXR_retstat ). Trying >>> to >>>>>>> recover. >>>>>>>> [2015-08-19 14:13:14.949688 +0100][Error ][XRootD ] >>>>>>> [dcache-cms-xrootd.desy.de:1094] Handling error while >>> processing >>>>>>> kXR_open (file: >>>>>>> >>>>> >>> > /store/mc/RunIISpring15DR74/TT_TuneCUETP8M1_13TeV-powheg-pythia8/MINIAODSIM/Asympt50ns_MCRUN2_74_V9A-v4/10000/00D2A247-2910-E511-9F3D-0CC47A4DEDD2.root?tried=maite.iihe.ac.be,maite.iihe.ac.be,xrootd-redic.pi.infn.it,maite.iihe.ac.be,maite.iihe.ac.be,xrootd.ba.infn.it, >>>>>>> mode: 00, flags: kXR_open_read kXR_async kXR_retstat ): [FATAL] >>>>> Auth >>>>>>> failed. >>>>>>>> [2015-08-19 14:13:15.181594 +0100][Error ][XRootD ] >>>>>>> [maite.iihe.ac.be:1094] Handling error while processing >>> kXR_open >>>>>>> (file: >>>>>>> >>>>> >>> > /store/mc/RunIISpring15DR74/TT_TuneCUETP8M1_13TeV-powheg-pythia8/MINIAODSIM/Asympt50ns_MCRUN2_74_V9A-v4/10000/00D2A247-2910-E511-9F3D-0CC47A4DEDD2.root?tried=+1213xrootd-redic.pi.infn.it, >>>>>>> mode: 00, flags: kXR_open_read kXR_async kXR_retstat ): [FATAL] >>>>> Auth >>>>>>> failed. >>>>>>>> >>>>>>> >>>>> >>> > [0B/0B][100%][==================================================][0B/s] >>>>>>> >>>>>>>> Run: [FATAL] Redirect limit has been reached >>>>>>>> >>>>>>>> ÿÿ >>>>>>>> Reply to this email directly or view it on GitHub. >>>>>>>> >>>>>>>> >>>>>>>> Use REPLY-ALL to reply to list >>>>>>>> >>>>>>>> To unsubscribe from the XROOTD-DEV list, click the following >>>>> link: >>>>>>>> >>>>> >>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Reply to this email directly or view it on GitHub [1]. >>>>>>> >>>>>>> * >>>>>>> >>>>>>> Links: >>>>>>> ------ >>>>>>> [1] >>>>> > https://github.com/xrootd/xrootd/issues/279#issuecomment-132631599 >>>>>> ÿÿ >>>>>> Reply to this email directly or view it on GitHub. >>>>>> >>>>>> >>>>>> Use REPLY-ALL to reply to list >>>>>> >>>>>> To unsubscribe from the XROOTD-DEV list, click the following >>> link: >>>>>> >>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>>>>> >>>>> >>>>> -- >>>>> Reply to this email directly or view it on GitHub [1]. >>>>> >>>>> * >>>>> >>>>> Links: >>>>> ------ >>>>> [1] >>> https://github.com/xrootd/xrootd/issues/279#issuecomment-132947363 >>>> ÿÿ >>>> Reply to this email directly or view it on GitHub. >>>> >>>> >>>> Use REPLY-ALL to reply to list >>>> >>>> To unsubscribe from the XROOTD-DEV list, click the following link: >>>> > https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>>> >>> >>> -- >>> Reply to this email directly or view it on GitHub [1]. >>> >>> * >>> >>> Links: >>> ------ >>> [1] > https://github.com/xrootd/xrootd/issues/279#issuecomment-133047733 >> >> >> --- >> Reply to this email directly or view it on GitHub: >> https://github.com/xrootd/xrootd/issues/279#issuecomment-133066775 > > -- > Reply to this email directly or view it on GitHub [1]. > > * > > Links: > ------ > [1] https://github.com/xrootd/xrootd/issues/279#issuecomment-133767693 Test command: xrdcp -d 1 -f root://cms-xrd-global.cern.ch//store/mc/Phys14DR/DYJetsToLL_M-50_13TeV-madgraph-pythia8/MINIAODSIM/PU20bx25_PHYS14_25_V1-v1/00000/A266FB5C-796C-E411-B6EE-0025901D493E.root /dev/null dataset: /DYJetsToLL_M-50_13TeV-madgraph-pythia8/Phys14DR-PU20bx25_PHYS14_25_V1-v1/MINIAODSIM" author: Viesturs Veckalns [log in to unmask] date: October 5, 2015 No. Server Transfer failed True/False ------------------------------------------------------------------------- 0 s17n02.hep.wisc.edu False 1 s17n02.hep.wisc.edu False 2 NONE SHOWN 3 nute.csc.fi False 4 hask.csc.fi False 5 s17n02.hep.wisc.edu False 6 s17n02.hep.wisc.edu False 7 xrootd.rcac.purdue.edu False 8 xrootd.rcac.purdue.edu False 9 s17n02.hep.wisc.edu False 10 xrootd.rcac.purdue.edu False 11 s17n02.hep.wisc.edu False 12 s17n02.hep.wisc.edu False 13 xrootd.rcac.purdue.edu False 14 s17n02.hep.wisc.edu False 15 xrootd.rcac.purdue.edu False 16 xrootd.rcac.purdue.edu False 17 xrootd.rcac.purdue.edu False 18 cmseos.fnal.gov False 19 s17n02.hep.wisc.edu False 20 s17n02.hep.wisc.edu False 21 cmseos.fnal.gov False 22 eoscms.cern.ch False 23 xrootd.rcac.purdue.edu False 24 s17n02.hep.wisc.edu False 25 s17n02.hep.wisc.edu False 26 nute.csc.fi False 27 hask.csc.fi False 28 nute.csc.fi False 29 xrootd.rcac.purdue.edu False 30 s17n02.hep.wisc.edu False 31 xrootd.rcac.purdue.edu False 32 xrootd.rcac.purdue.edu False 33 s17n02.hep.wisc.edu False 34 xrootd.rcac.purdue.edu False 35 se01.indiacms.res.in True 36 xrootd.rcac.purdue.edu False 37 xrootd.rcac.purdue.edu False 38 cmseos.fnal.gov False 39 s17n02.hep.wisc.edu False 40 se01.indiacms.res.in True 41 cmseos.fnal.gov False 42 xrootd.rcac.purdue.edu False 43 se01.indiacms.res.in True 44 xrootd.rcac.purdue.edu False 45 xrootd.rcac.purdue.edu False 46 s17n02.hep.wisc.edu False 47 xrootd.rcac.purdue.edu False 48 s17n02.hep.wisc.edu False 49 s17n02.hep.wisc.edu False 50 xrootd.rcac.purdue.edu False 51 xrootd.rcac.purdue.edu False 52 s17n02.hep.wisc.edu False 53 s17n02.hep.wisc.edu False 54 cmseos.fnal.gov False 55 NONE SHOWN 56 eoscms.cern.ch False 57 se01.indiacms.res.in True 58 NONE SHOWN 59 NONE SHOWN 60 cmseos.fnal.gov False 61 NONE SHOWN 62 NONE SHOWN 63 NONE SHOWN 64 NONE SHOWN 65 NONE SHOWN 66 cmseos.fnal.gov False 67 cmseos.fnal.gov False 68 NONE SHOWN 69 NONE SHOWN 70 s17n02.hep.wisc.edu False 71 s17n02.hep.wisc.edu False 72 s17n02.hep.wisc.edu False 73 s17n02.hep.wisc.edu False 74 cmseos.fnal.gov False 75 NONE SHOWN 76 s17n02.hep.wisc.edu False 77 s17n02.hep.wisc.edu False 78 cmseos.fnal.gov False 79 s17n02.hep.wisc.edu False 80 cmseos.fnal.gov False 81 s17n02.hep.wisc.edu False 82 s17n02.hep.wisc.edu False 83 s17n02.hep.wisc.edu False 84 s17n02.hep.wisc.edu False 85 s17n02.hep.wisc.edu False 86 s17n02.hep.wisc.edu False 87 s17n02.hep.wisc.edu False 88 s17n02.hep.wisc.edu False 89 s17n02.hep.wisc.edu False 90 s17n02.hep.wisc.edu False 91 s17n02.hep.wisc.edu False 92 s17n02.hep.wisc.edu False 93 s17n02.hep.wisc.edu False 94 s17n02.hep.wisc.edu False 95 s17n02.hep.wisc.edu False 96 cmseos.fnal.gov False 97 cmseos.fnal.gov False 98 s17n02.hep.wisc.edu False 99 s17n02.hep.wisc.edu False 100 s17n02.hep.wisc.edu False Test command xrdcp -d 1 -f root://cms-xrd-global.cern.ch//store/mc/RunIISpring15DR74/WW_TuneCUETP8M1_13TeV-pythia8/MINIAODSIM/Asympt50ns_MCRUN2_74_V9A-v1/10000/A89CCB89-4106-E511-BB7C-00259073E4A0.root /dev/null dataset: /WW_TuneCUETP8M1_13TeV-pythia8/RunIISpring15DR74-Asympt50ns_MCRUN2_74_V9A-v1/MINIAODSIM author: Viesturs Veckalns [log in to unmask] date: October 5, 2015 No. Server Transfer failed True/False ------------------------------------------------------------------------- 0 pool04.ifca.es False 1 NONE SHOWN 2 NONE SHOWN 3 NONE SHOWN 4 cabinet-7-7-0.t2.ucsd.edu False 5 se1.accre.vanderbilt.edu False 6 NONE SHOWN 7 NONE SHOWN 8 NONE SHOWN 9 NONE SHOWN 10 f01-065-135-e.gridka.de False 11 se1.accre.vanderbilt.edu False 12 se1.accre.vanderbilt.edu False 13 pool06.ifca.es True 14 se1.accre.vanderbilt.edu False 15 NONE SHOWN 16 se1.accre.vanderbilt.edu False 17 NONE SHOWN 18 se1.accre.vanderbilt.edu False 19 se1.accre.vanderbilt.edu False 20 pool03.ifca.es False 21 se1.accre.vanderbilt.edu False 22 pool05.ifca.es False 23 pool03.ifca.es True 24 pool03.ifca.es True 25 pool01.ifca.es True 26 pool01.ifca.es True 27 pool02.ifca.es False 28 se1.accre.vanderbilt.edu False 29 pool01.ifca.es True 30 se1.accre.vanderbilt.edu False 31 pool07.ifca.es True 32 se1.accre.vanderbilt.edu False 33 se1.accre.vanderbilt.edu False 34 pool08.ifca.es False 35 pool02.ifca.es True 36 NONE SHOWN 37 pool01.ifca.es True 38 pool06.ifca.es False 39 NONE SHOWN 40 NONE SHOWN 41 NONE SHOWN 42 NONE SHOWN 43 NONE SHOWN 44 NONE SHOWN 45 NONE SHOWN 46 NONE SHOWN 47 se1.accre.vanderbilt.edu False 48 NONE SHOWN 49 NONE SHOWN 50 se1.accre.vanderbilt.edu False 51 NONE SHOWN 52 NONE SHOWN 53 NONE SHOWN 54 NONE SHOWN 55 NONE SHOWN 56 NONE SHOWN 57 NONE SHOWN 58 NONE SHOWN 59 NONE SHOWN 60 NONE SHOWN 61 NONE SHOWN 62 NONE SHOWN 63 NONE SHOWN 64 NONE SHOWN 65 NONE SHOWN 66 NONE SHOWN 67 NONE SHOWN 68 NONE SHOWN 69 NONE SHOWN 70 NONE SHOWN 71 se1.accre.vanderbilt.edu False 72 NONE SHOWN 73 NONE SHOWN 74 NONE SHOWN 75 NONE SHOWN 76 NONE SHOWN 77 NONE SHOWN 78 NONE SHOWN 79 NONE SHOWN 80 NONE SHOWN 81 NONE SHOWN 82 NONE SHOWN 83 NONE SHOWN 84 NONE SHOWN 85 NONE SHOWN 86 NONE SHOWN 87 NONE SHOWN 88 NONE SHOWN 89 NONE SHOWN 90 NONE SHOWN 91 NONE SHOWN 92 NONE SHOWN 93 NONE SHOWN 94 NONE SHOWN 95 NONE SHOWN 96 NONE SHOWN 97 NONE SHOWN 98 NONE SHOWN 99 NONE SHOWN 100 NONE SHOWN --- Reply to this email directly or view it on GitHub: https://github.com/xrootd/xrootd/issues/279#issuecomment-145679639 ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1