 |
 |
Hi Patrick, Great! I do agree it's very odd behaviour. Something we obviously either never tested for or have forgotten there was some rare use case that required it :-) Andy On Wed, 16 Aug 2017, Patrick McGuigan wrote: > Hi Andy, > > Thanks. Dropping the /usr/bin/xrdadler32 from the redirector's chksum > configuration gets a version 4.6.1 client to do the checksum on the > dataserver and store the results in the extended attributes. > > Thanks, > > Patrick > > > On 08/16/2017 05:47 PM, Andrew Hanushevsky wrote: >> Hi Patrick, >> >> I assume your filesystems are using the XA format (where info gets recorded >> in the file's extended attributes). If so, I dodn't see why one needs to >> invoke xrdadler32 as it's builtin for some time. In any case, specifying >> that for the redirector makes little sense. So, yes, at least remove it >> there. >> >> Andy >> >> On Wed, 16 Aug 2017, Patrick McGuigan wrote: >> >>> Hi Andy, >>> >>> My version 3.3.6 redirector now uses: >>> xrootd.chksum adler32 /usr/bin/xrdadler32 >>> >>> but the version 4.6.1 xrdadler32 still has the problem. >>> >>> Should I remove the callout to /usr/bin/xrdadler32? >>> >>> Version 4.5.0 seems to be okay. I am happy with my solution since the >>> SRM's days are limited and we will be doing a software update on the >>> dataservers/redirector in the near future. >>> >>> >>> Regards, >>> >>> Patrick >>> >>> >>> On 08/16/2017 11:45 AM, Andrew Hanushevsky wrote: >>>> Yes, the issue we fixed (that people were complaining about) that issuing >>>> a checksum request to the redirector didn't redirect. So, we fixed that >>>> but that also meant the redirector needed to be told about checksums (I >>>> believe we did that for backward compatabilty). Clearly, we didn't >>>> succeede relative to your particular setup. So, while I don't (yet) >>>> understand why this behaviour occurs, I do thnk that adding that line >>>> will correct the issue. >>>> >>>> Andy >>>> >>>> On Wed, 16 Aug 2017, Yang, Wei wrote: >>>> >>>>> So Patrick, the problem you reported is that in 4.6.1 xrdadler32 doesnt >>>>> trigger a server side checksum calculation when xrdadler32 reached to >>>>> the redirector, and only trigger server side calculation when going >>>>> directly to the data server? >>>>> >>>>> I am running 4.4.0 for production and I dont see this issue. I recently >>>>> run into a similar issue on a test cluster with 4.6.1 and Andy told me >>>>> that adding xrootd.chksum xrdadler32 to the redirector should address >>>>> this issue. That seems to work but I will try it again. >>>>> >>>>> regards, >>>>> -- >>>>> Wei Yang | [log in to unmask] | 650-926-3338(O) >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On 8/16/17, 2:24 AM, "[log in to unmask] on behalf of Patrick >>>>> McGuigan" <[log in to unmask] on behalf of [log in to unmask]> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I ran into an issue with xrdadler32 that may or may not be a bug. >>>>>> >>>>>> >>>>>> My dataservers and redirector are running version xrootd 3.3.6 >>>>>> >>>>>> The dataservers are configured with the line: >>>>>> >>>>>> xrootd.chksum max 4 adler32 /usr/bin/xrdadler32 >>>>>> >>>>>> so that checksums are stored in the extended attributes of the native >>>>>> XFS file >>>>>> system. This works fine. >>>>>> >>>>>> >>>>>> Originally the redirector did not have a similar line but during >>>>>> debugging I added: >>>>>> xrootd.chksum adler32 /usr/bin/xrdadler32 >>>>>> >>>>>> >>>>>> I had to reinstall a client machine (an SRM host) based on the OSG >>>>>> software >>>>>> stack. The installation brought in version 4.6.1 for xrootd-client, >>>>>> xrootd-fuse, xrootd-libs, xrootd-client-libs. Specifically the >>>>>> packages are: >>>>>> <package>-4.6.1-1.osg33.el6.x86_64 >>>>>> >>>>>> >>>>>> After installation I ran into problems with code that uses xrdadler32 >>>>>> to >>>>>> retrieve checksums from files that are normally mounted via xrootdfs. >>>>>> The code >>>>>> that uses xrdadler32 exports an XROOTD_VMP so that calls are translated >>>>>> to >>>>>> native xroot URL's. The problem was that the code was not retrieving >>>>>> the >>>>>> checksum, but instead, calculating the checksum on the SRM host which >>>>>> required >>>>>> the SRM host to read the entire file locally. >>>>>> >>>>>> >>>>>> I tracked this back a little bit and discovered that if I used >>>>>> /usr/bin/xrdadler32 with an xroot URL (with the redirector as host), >>>>>> then the >>>>>> checksum would be performed without a checksum being stored on the >>>>>> dataservers >>>>>> native file system using extended attributes. If the xroot URL used >>>>>> the correct >>>>>> dataserver as the host, then the checksum was stored. >>>>>> >>>>>> >>>>>> I downgraded the xrootd installation on the SRM host to version >>>>>> xrootd-client-4.5.0-2.osg33.el6.x86_64 and everything seems to be >>>>>> working properly. >>>>>> >>>>>> >>>>>> I am not sure if this is a bug, given the age of the dataserver >>>>>> installation, >>>>>> but I was encouraged to report it. >>>>>> >>>>>> Regards, >>>>>> >>>>>> Patrick >>>>>> >>>>>> ######################################################################## >>>>>> Use REPLY-ALL to reply to list >>>>>> >>>>>> To unsubscribe from the XROOTD-L list, click the following link: >>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 >>>>> >>>>> ######################################################################## >>>>> Use REPLY-ALL to reply to list >>>>> >>>>> To unsubscribe from the XROOTD-L list, click the following link: >>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 >>>>> >>>> >>>> ######################################################################## >>>> Use REPLY-ALL to reply to list >>>> >>>> To unsubscribe from the XROOTD-L list, click the following link: >>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 >>> >> >> ######################################################################## >> Use REPLY-ALL to reply to list >> >> To unsubscribe from the XROOTD-L list, click the following link: >> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 > ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1