Hi Patrick,
Great! I do agree it's very odd behaviour. Something we obviously either
never tested for or have forgotten there was some rare use case that
required it :-)
Andy
On Wed, 16 Aug 2017, Patrick McGuigan wrote:
> Hi Andy,
>
> Thanks. Dropping the /usr/bin/xrdadler32 from the redirector's chksum
> configuration gets a version 4.6.1 client to do the checksum on the
> dataserver and store the results in the extended attributes.
>
> Thanks,
>
> Patrick
>
>
> On 08/16/2017 05:47 PM, Andrew Hanushevsky wrote:
>> Hi Patrick,
>>
>> I assume your filesystems are using the XA format (where info gets recorded
>> in the file's extended attributes). If so, I dodn't see why one needs to
>> invoke xrdadler32 as it's builtin for some time. In any case, specifying
>> that for the redirector makes little sense. So, yes, at least remove it
>> there.
>>
>> Andy
>>
>> On Wed, 16 Aug 2017, Patrick McGuigan wrote:
>>
>>> Hi Andy,
>>>
>>> My version 3.3.6 redirector now uses:
>>> xrootd.chksum adler32 /usr/bin/xrdadler32
>>>
>>> but the version 4.6.1 xrdadler32 still has the problem.
>>>
>>> Should I remove the callout to /usr/bin/xrdadler32?
>>>
>>> Version 4.5.0 seems to be okay. I am happy with my solution since the
>>> SRM's days are limited and we will be doing a software update on the
>>> dataservers/redirector in the near future.
>>>
>>>
>>> Regards,
>>>
>>> Patrick
>>>
>>>
>>> On 08/16/2017 11:45 AM, Andrew Hanushevsky wrote:
>>>> Yes, the issue we fixed (that people were complaining about) that issuing
>>>> a checksum request to the redirector didn't redirect. So, we fixed that
>>>> but that also meant the redirector needed to be told about checksums (I
>>>> believe we did that for backward compatabilty). Clearly, we didn't
>>>> succeede relative to your particular setup. So, while I don't (yet)
>>>> understand why this behaviour occurs, I do thnk that adding that line
>>>> will correct the issue.
>>>>
>>>> Andy
>>>>
>>>> On Wed, 16 Aug 2017, Yang, Wei wrote:
>>>>
>>>>> So Patrick, the problem you reported is that in 4.6.1 xrdadler32 doesnt
>>>>> trigger a server side checksum calculation when xrdadler32 reached to
>>>>> the redirector, and only trigger server side calculation when going
>>>>> directly to the data server?
>>>>>
>>>>> I am running 4.4.0 for production and I dont see this issue. I recently
>>>>> run into a similar issue on a test cluster with 4.6.1 and Andy told me
>>>>> that adding xrootd.chksum xrdadler32 to the redirector should address
>>>>> this issue. That seems to work but I will try it again.
>>>>>
>>>>> regards,
>>>>> --
>>>>> Wei Yang | [log in to unmask] | 650-926-3338(O)
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 8/16/17, 2:24 AM, "[log in to unmask] on behalf of Patrick
>>>>> McGuigan" <[log in to unmask] on behalf of [log in to unmask]>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I ran into an issue with xrdadler32 that may or may not be a bug.
>>>>>>
>>>>>>
>>>>>> My dataservers and redirector are running version xrootd 3.3.6
>>>>>>
>>>>>> The dataservers are configured with the line:
>>>>>>
>>>>>> xrootd.chksum max 4 adler32 /usr/bin/xrdadler32
>>>>>>
>>>>>> so that checksums are stored in the extended attributes of the native
>>>>>> XFS file
>>>>>> system. This works fine.
>>>>>>
>>>>>>
>>>>>> Originally the redirector did not have a similar line but during
>>>>>> debugging I added:
>>>>>> xrootd.chksum adler32 /usr/bin/xrdadler32
>>>>>>
>>>>>>
>>>>>> I had to reinstall a client machine (an SRM host) based on the OSG
>>>>>> software
>>>>>> stack. The installation brought in version 4.6.1 for xrootd-client,
>>>>>> xrootd-fuse, xrootd-libs, xrootd-client-libs. Specifically the
>>>>>> packages are:
>>>>>> <package>-4.6.1-1.osg33.el6.x86_64
>>>>>>
>>>>>>
>>>>>> After installation I ran into problems with code that uses xrdadler32
>>>>>> to
>>>>>> retrieve checksums from files that are normally mounted via xrootdfs.
>>>>>> The code
>>>>>> that uses xrdadler32 exports an XROOTD_VMP so that calls are translated
>>>>>> to
>>>>>> native xroot URL's. The problem was that the code was not retrieving
>>>>>> the
>>>>>> checksum, but instead, calculating the checksum on the SRM host which
>>>>>> required
>>>>>> the SRM host to read the entire file locally.
>>>>>>
>>>>>>
>>>>>> I tracked this back a little bit and discovered that if I used
>>>>>> /usr/bin/xrdadler32 with an xroot URL (with the redirector as host),
>>>>>> then the
>>>>>> checksum would be performed without a checksum being stored on the
>>>>>> dataservers
>>>>>> native file system using extended attributes. If the xroot URL used
>>>>>> the correct
>>>>>> dataserver as the host, then the checksum was stored.
>>>>>>
>>>>>>
>>>>>> I downgraded the xrootd installation on the SRM host to version
>>>>>> xrootd-client-4.5.0-2.osg33.el6.x86_64 and everything seems to be
>>>>>> working properly.
>>>>>>
>>>>>>
>>>>>> I am not sure if this is a bug, given the age of the dataserver
>>>>>> installation,
>>>>>> but I was encouraged to report it.
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Patrick
>>>>>>
>>>>>> ########################################################################
>>>>>> Use REPLY-ALL to reply to list
>>>>>>
>>>>>> To unsubscribe from the XROOTD-L list, click the following link:
>>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
>>>>>
>>>>> ########################################################################
>>>>> Use REPLY-ALL to reply to list
>>>>>
>>>>> To unsubscribe from the XROOTD-L list, click the following link:
>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
>>>>>
>>>>
>>>> ########################################################################
>>>> Use REPLY-ALL to reply to list
>>>>
>>>> To unsubscribe from the XROOTD-L list, click the following link:
>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
>>>
>>
>> ########################################################################
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the XROOTD-L list, click the following link:
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
|