 |
 |
Oups sorry! you're perfectly right David, thank you. env set to => XRDCLHTTP_AVOIDRANAGE=1 But the problem is still there. So now using curl, no problem. \o/ Using cvmfs, it seems that whatever the file, I got a file size of 25165824. Looking at xcache logs: At the beginning I have the first range (from 0 to 25165823): 210511 16:54:15 164541 sysXrdHttp: Parsing first line: GET //http://fiona.uvalight.net:8000/gwdata/O1/strain.16k/hdf.v1/H1/1136656384/H-H1_LOSC_16_V1-1137221632-4096.hdf5 HTTP/1.1 210511 16:54:15 164541 sysXrdHttp: rc:37 got hdr line: Host: our_xcache_server:1094 210511 16:54:15 164541 sysXrdHttp: rc:25 got hdr line: Range: bytes=0-25165823 210511 16:54:15 164541 sysXrdHttp: rc:13 got hdr line: Accept: */* 210511 16:54:15 164541 sysXrdHttp: rc:24 got hdr line: Connection: Keep-Alive 210511 16:54:15 164541 sysXrdHttp: rc:30 got hdr line: User-Agent: cvmfs Fuse 2.8.1 210511 16:54:15 164541 sysXrdHttp: rc:105 got hdr line: cvmfs-info: Part%20of%20/gwdata/O1/strain.16k/hdf.v1/H1/1136656384/H-H1_LOSC_16_V1-1137221632-4096.hdf5 Then download starts . . . Then I've got the next range (from 25165824-50331647) 210511 16:54:17 164540 sysXrdHttp: Parsing first line: GET //http://fiona.uvalight.net:8000/gwdata/O1/strain.16k/hdf.v1/H1/1136656384/H-H1_LOSC_16_V1-1137221632-4096.hdf5 HTTP/1.1 210511 16:54:17 164540 sysXrdHttp: rc:37 got hdr line: Host: our_xcache_server:1094 210511 16:54:17 164540 sysXrdHttp: rc:32 got hdr line: Range: bytes=25165824-50331647 210511 16:54:17 164540 sysXrdHttp: rc:13 got hdr line: Accept: */* 210511 16:54:17 164540 sysXrdHttp: rc:24 got hdr line: Connection: Keep-Alive 210511 16:54:17 164540 sysXrdHttp: rc:30 got hdr line: User-Agent: cvmfs Fuse 2.8.1 210511 16:54:17 164540 sysXrdHttp: rc:105 got hdr line: cvmfs-info: Part%20of%20/gwdata/O1/strain.16k/hdf.v1/H1/1136656384/H-H1_LOSC_16_V1-1137221632-4096.hdf5 Then download . . . But the next range is in fact the same than just before, so we got again (from 25165824-50331647): 210511 16:54:17 164541 sysXrdHttp: Parsing first line: GET //http://fiona.uvalight.net:8000/gwdata/O1/strain.16k/hdf.v1/H1/1136656384/H-H1_LOSC_16_V1-1137221632-4096.hdf5 HTTP/1.1 210511 16:54:17 164541 sysXrdHttp: rc:37 got hdr line: Host: our_xcache_server:1094 210511 16:54:17 164541 sysXrdHttp: rc:32 got hdr line: Range: bytes=25165824-50331647 210511 16:54:17 164541 sysXrdHttp: rc:13 got hdr line: Accept: */* 210511 16:54:17 164541 sysXrdHttp: rc:24 got hdr line: Connection: Keep-Alive 210511 16:54:17 164541 sysXrdHttp: rc:30 got hdr line: User-Agent: cvmfs Fuse 2.8.1 210511 16:54:17 164541 sysXrdHttp: rc:105 got hdr line: cvmfs-info: Part%20of%20/gwdata/O1/strain.16k/hdf.v1/H1/1136656384/H-H1_LOSC_16_V1-1137221632-4096.hdf5 210511 16:54:17 164541 sysXrdHttp: rc:18 got hdr line: Pragma: no-cache 210511 16:54:17 164541 sysXrdHttp: rc:25 got hdr line: Cache-Control: no-cache and it seems to loop like this 7 times in total! In the end there is no error but it seems to stop because of all these "retry"? Finally, in cvmfs repo, the catalog shows a file of size 517357705 after cvmfs copy, the file has a size of 25165824 (the first downloaded range I think) in xcache data, the file has a size of 83886080 (OMHO must be totally corrupted) regards, Bertrand Rigaud Centre de Calcul de l'IN2P3 - CNRS 21 avenue Pierre de Coubertin 69627 Villeurbanne CEDEX Tél : 04.78.93.08.80 ----- Mail original ----- De: "David Smith" <[log in to unmask]> À: "Bertrand Rigaud" <[log in to unmask]> Cc: "Wei Yang" <[log in to unmask]>, "Matevz Tadel" <[log in to unmask]>, "xrootd-l" <[log in to unmask]> Envoyé: Mardi 11 Mai 2021 15:36:49 Objet: Re: http ingest using xcache fails > On 11 May 2021, at 15:03, Bertrand RIGAUD <[log in to unmask]> wrote: > > Yes this one is even more important as it is parsed from url! > > I'm not sure if I understand well, when using curl + ?xrddclhttp_avoidrange everything works like a charm, whereas trying cp from cvmfs repo, the problem is still here. > I added the env var to my service: > > [Service] > Environment=HTTP_FILE_PLUG_IN_AVOIDRANGE_ENV=XRDCLHTTP_AVOIDRANGE Hi Bertrand, Perhaps you could try setting the environment in this way: Environment=XRDCLHTTP_AVOIDRANAGE=1 Yours, David -- ------------------------------------------------------------------------- David Smith e-mail: [log in to unmask] ------------------------------------------------------------------------- ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1