 |
 |
Following document doesn't speak about base64 encoding for crc32c https://www.iana.org/assignments/http-dig-alg/http-dig-alg.xml and Davix checksum extraction magic also don't really expect base64 encoded crc32c https://gitlab.cern.ch/dmc/davix/-/blob/devel/src/utils/checksum_extractor.cpp#L72-L111 btw: crc32c is not supported by all our storage implementation (e.g. dCache) https://github.com/dCache/dcache/blob/master/modules/dcache/src/main/java/org/dcache/util/Checksums.java#L50-L55 Petr On 3/8/23 16:00, Doidge, Matt wrote: > Hello all, > I've was tinkering after adding crc32c support to one of our xrootd servers and saw an odd behaviour with gfal-sum over https when trying to return a crc32c checksum - it's returning an encoded version of the checksum. gfal-sum over root works as expected. xrdcrc32c performed on the file locally returns the expected result. > > gfal-sum root://stor015.hec.lancs.ac.uk:1095/cephfs/grid/dteam/crc32chttpstest5.txt crc32c > root://stor015.hec.lancs.ac.uk:1095/cephfs/grid/dteam/crc32chttpstest5.txt d98fa965 > > $gfal-sum https://stor015.hec.lancs.ac.uk:1095/cephfs/grid/dteam/crc32chttpstest5.txt crc32c > https://stor015.hec.lancs.ac.uk:1095/cephfs/grid/dteam/crc32chttpstest5.txt 2Y+pZQ== > > $ echo "2Y+pZQ==" | base64 -d | xxd -ps > d98fa965 > > I tried to deduce if this is a problem within gfal tools, and looking into the verbose output (-vvv) I think the "2Y+pZQ==" is what is being returned by the server, from a snippet below: > > DEBUG Davix: Request sent; retry is 0. > INFO Davix: < HTTP/1.1 200 OK > INFO Davix: < Connection: Keep-Alive > INFO Davix: < Server: XrootD/v5.5.2 > INFO Davix: < Content-Length: 996 > INFO Davix: < Digest: crc32c=2Y+pZQ== > INFO Davix: < > DEBUG Davix: End of headers. > > The file exists on disk with the extended attribute below: > user.XrdCks.crc32c=0sY3JjMzJjAAAAAAAAAAAAAAAAAABkCJzmAAAAAAAAAATZj6llAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA > > (which again if you binary decode you can see the checksum in there). > > Note with adler32 we do not see anything out of the ordinary, checksums are returned as expected with https:// or root:// Our filesystem xroot is writing to is cephfs, the operating system is Rocky8, and xroot version 5.5.3. The line used to configure our checksums is just: > > xrootd.chksum max 32 adler32 crc32c > > I am wondering if this is an xroot bug (then I'll open an issue), a likely bug in our setup (then I'd appreciate a pointer to it!) or a problem with gfal (where I'll throw them a ticket). > > Thanks in advance all, > Matt > ######################################################################## > Use REPLY-ALL to reply to list > > To unsubscribe from the XROOTD-L list, click the following link: > https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1