Hi Andy,

Thanks for the quick reply.

When opening this issue, I perhaps didn't give sufficient information to fully understand what's happening, so let me provide more details.

First, the label sha is not ambiguous. It's defined in RFC 3230 as the SHA-1 algorithm from FIPS 180.

Second, unlike in the xroot protocol, HTTP 3230 allows the client to specify a list of algorithms, partially ordered by "priority". Here's a couple of examples:

• Want-Digest: adler32, md5
  Here, the client would like either an adler32 or MD5 checksum (or both), with no priority given to either algorithm.
• Want-Digest: md5,adler32;q=0.5
  In this example, the client would like either an adler32 or MD5 checksum, but this time, the MD5 checksum is preferred (the ";q=1" is implied).

I believe the Want-Digest request header is meant to implement a negotiation between the client and server. The main purpose is to avoid the server incurring overhead from generating checksum information that the client will ignore.

However, the RFC is annoyingly vague on what is correct behaviour if one (or more) of the requested algorithms isn't supported by the server. From reading the RFC, I believe the intention was that the server should ignore any algorithm it does not support (and not fail the request). RFC 3230 makes it clear that the server is free to ignore requests for any specific algorithm, which (I feel) lends weight to this interpretation. However, it's a deficiency in the RFC that this is not better described.

You mentioned that you didn't see an advantage of ignoring unknown algorithms.

For me, there is a clear advantage: it allows for smooth transitions as new algorithms are supported.

Apparently, dCache supports additional checksum algorithms than xrootd (SHA-512, SHA-256, SHA-1). An HTTP-TPC third-party transfer between two dCache instances may be protected using one of these new checksums.

If xrootd were simply to ignore Want-Digest checksum algorithms it doesn't support then this improved support would have no impact on HTTP-TPC transfers from an xrootd server to a dCache server. Such transfers would continue to be protected with ADLER32 or MD5.

The current behaviour of xrootd (failing the request) makes HTTP-TPC brittle and causes transfer failures.

Just to be explicit, this is the Want-Digest that the current master branch dCache sends:

Want-Digest: sha-512,sha-256;q=0.8,sha;q=0.6,md5;q=0.4,adler32;q=0.2

In terms of fixing this issue as quickly and with the least effort, I was wondering whether xrootd ever triggers checksum calculation if the client specifies a Want-Digest? If not then the server could simply ignore the Want-Digest header value and simply return all the known checksums for the file. That behaviour would be allowed by RFC 3230.

Cheers,
Paul.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.Message ID: <xrootd/xrootd/issues/1707/1125790804@github.com>

Use REPLY-ALL to reply to list

To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1