In some instances, Uproot may [produce HTTP requests](https://github.com/scikit-hep/uproot5/issues/881) with large Range requests: over 1024 items and length beyond 16k chars. XRootD unfortunately doesn't respond with standard HTTP error codes.
1. For Range requests beyond 1024 items, XRootD HTTP sends two responses: success, followed by failure. This is confusing for HTTP clients:
client request:
```
GET /store/user/AGC/nanoAOD/TT_TuneCUETP8M1_13TeV-powheg-pythia8/cmsopendata2015_ttbar_19980_PU25nsData2015v1_76X_mcRun2_asymptotic_v12_ext3-v1_00000_0000.root HTTP/1.1
Host: red-xfer13.unl.edu:1094
User-Agent: python-requests/2.28.1
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
Range: bytes=0-4, 10-14, ..., 10240-10244
```
server reply:
```
HTTP/1.1 206 Partial Content
Connection: Keep-Alive
Server: XrootD/v5.5.4
Content-Length: 105417
Content-Type: multipart/byteranges; boundary=123456
HTTP/1.1 500 Internal Server Error
Connection: Close
Server: XrootD/v5.5.4
Content-Length: 24
Read vector is too long
```
<details>
<summary>Backtrace of breakpoint on XrdHttpProtocol::StartSimpleResp for the two replies</summary>
```
#0 XrdHttpProtocol::StartSimpleResp (this=0x7f634400fee0, code=206, desc=0x0, header_to_add=0x7f6344021f10 "Content-Type: multipart/byteranges; boundary=123456", bodylen=105417, keepalive=true)
at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/XrdHttp/XrdHttpProtocol.cc:1517
#1 0x00007f635bacfc74 in XrdHttpProtocol::SendSimpleResp (this=0x7f634400fee0, code=<optimized out>, desc=<optimized out>, header_to_add=<optimized out>, body=0x0, bodylen=105417, keepalive=true)
at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/XrdHttp/XrdHttpProtocol.cc:1616
#2 0x00007f635badbc5c in XrdHttpReq::PostProcessHTTPReq (this=0x7f6344010040, final_=<optimized out>) at /usr/include/c++/8/bits/basic_string.h:2294
#3 0x00007f635badedf2 in XrdHttpReq::Data (this=0x7f6344010040, info=..., iovP_=0x7f634aa7b310, iovN_=2, iovL_=<optimized out>, final_=<optimized out>) at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/XrdHttp/XrdHttpReq.cc:511
#4 0x00007f636101e633 in XrdXrootdTransit::Send (this=0x7f63440204d0, rcode=rcode@entry=0, ioV=ioV@entry=0x7f634aa7b310, ioN=ioN@entry=2, ioL=ioL@entry=99) at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/XrdXrootd/XrdXrootdTransit.cc:675
#5 0x00007f636101aa56 in XrdXrootdResponse::Send (this=this@entry=0x7f63440208b8, IOResp=IOResp@entry=0x7f634aa7b300, iornum=iornum@entry=3, iolen=iolen@entry=99) at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/XrdXrootd/XrdXrootdResponse.cc:222
#6 0x00007f636102895d in XrdXrootdProtocol::do_Open (this=this@entry=0x7f63440204d8) at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/XrdXrootd/XrdXrootdXeq.cc:1627
#7 0x00007f63610192ec in XrdXrootdProtocol::Process2 (this=0x7f63440204d8) at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/XrdXrootd/XrdXrootdProtocol.cc:572
#8 0x00007f636101d495 in XrdXrootdTransit::Process (this=0x7f63440204d0, lp=0x7f6344008290) at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/XrdXrootd/XrdXrootdTransit.cc:397
#9 0x00007f6360d60400 in XrdLinkXeq::DoIt (this=<optimized out>) at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/Xrd/XrdLinkXeq.cc:320
#10 XrdLinkXeq::DoIt (this=0x7f6344008290) at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/Xrd/XrdLinkXeq.cc:308
#11 0x00007f6360d636a7 in XrdScheduler::Run (this=0x615740 <XrdGlobal::Sched>) at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/Xrd/XrdScheduler.cc:406
#12 0x00007f6360d637c9 in XrdStartWorking (carg=<optimized out>) at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/Xrd/XrdScheduler.cc:89
#13 0x00007f6360cf35e7 in XrdSysThread_Xeq (myargs=0x7f6354016370) at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/XrdSys/XrdSysPthread.cc:86
#14 0x00007f635fd4b1cf in start_thread () from /lib64/libpthread.so.0
#15 0x00007f635f9b6e73 in clone () from /lib64/libc.so.6
```
```
#0 XrdHttpProtocol::StartSimpleResp (this=0x7f634400fee0, code=500, desc=0x0, header_to_add=0x0, bodylen=24, keepalive=false) at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/XrdHttp/XrdHttpProtocol.cc:1517
#1 0x00007f635bacfc74 in XrdHttpProtocol::SendSimpleResp (this=0x7f634400fee0, code=<optimized out>, desc=<optimized out>, header_to_add=<optimized out>, body=0x7f6344029ff0 "Read vector is too long\n", bodylen=24, keepalive=false)
at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/XrdHttp/XrdHttpProtocol.cc:1616
#2 0x00007f635badc384 in XrdHttpReq::PostProcessHTTPReq (this=0x7f6344010040, final_=<optimized out>) at /usr/include/c++/8/bits/basic_string.h:2294
#3 0x00007f635badef6a in XrdHttpReq::Error (this=0x7f6344010040, info=..., ecode=3002, etext_=0x7f6361084892 "Read vector is too long") at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/XrdHttp/XrdHttpReq.cc:563
#4 0x00007f636101e520 in XrdXrootdTransit::Send (this=0x7f63440204d0, rcode=rcode@entry=4003, ioV=ioV@entry=0x7f63440208e0, ioN=ioN@entry=2, ioL=<optimized out>) at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/XrdXrootd/XrdXrootdTransit.cc:671
#5 0x00007f636101ac0e in XrdXrootdResponse::Send (this=this@entry=0x7f63440208b8, ecode=ecode@entry=kXR_ArgTooLong, msg=msg@entry=0x7f6361084892 "Read vector is too long") at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/XrdXrootd/XrdXrootdResponse.cc:252
#6 0x00007f6361024de0 in XrdXrootdProtocol::do_ReadV (this=this@entry=0x7f63440204d8) at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/XrdXrootd/XrdXrootdXeq.cc:2562
#7 0x00007f6361019190 in XrdXrootdProtocol::Process2 (this=0x7f63440204d8) at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/XrdXrootd/XrdXrootdProtocol.cc:484
#8 0x00007f636101d495 in XrdXrootdTransit::Process (this=0x7f63440204d0, lp=0x7f6344008290) at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/XrdXrootd/XrdXrootdTransit.cc:397
#9 0x00007f6360d60400 in XrdLinkXeq::DoIt (this=<optimized out>) at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/Xrd/XrdLinkXeq.cc:320
#10 XrdLinkXeq::DoIt (this=0x7f6344008290) at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/Xrd/XrdLinkXeq.cc:308
#11 0x00007f6360d636a7 in XrdScheduler::Run (this=0x615740 <XrdGlobal::Sched>) at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/Xrd/XrdScheduler.cc:406
#12 0x00007f6360d637c9 in XrdStartWorking (carg=<optimized out>) at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/Xrd/XrdScheduler.cc:89
#13 0x00007f6360cf35e7 in XrdSysThread_Xeq (myargs=0x7f6354016370) at /usr/src/debug/xrootd-5.5.4-1.1.osg36.el8.x86_64/xrootd/src/XrdSys/XrdSysPthread.cc:86
#14 0x00007f635fd4b1cf in start_thread () from /lib64/libpthread.so.0
#15 0x00007f635f9b6e73 in clone () from /lib64/libc.so.6
```
</details>
2. For exceptionally large Range requests (roughly beyond 16k chars), the TCP connection is closed with a RST, and no HTTP reply. There doesn't seem to be an exact length beyond which a TCP RST is always received. As the header length exceeds 16k, the chances increase.
In both cases, an HTTP `431 Request Header Fields Too Large` would probably be the most help for clients.
<details>
<summary>Sample python3 script</summary>
```python
#!/usr/bin/env python3
import logging
import http.client
http.client.HTTPConnection.debuglevel = 1
logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True
import requests
url = 'http://red-xfer13.unl.edu:1094/store/user/AGC/nanoAOD/TT_TuneCUETP8M1_13TeV-powheg-pythia8/cmsopendata2015_ttbar_19980_PU25nsData2015v1_76X_mcRun2_asymptotic_v12_ext3-v1_00000_0000.root'
####
ranges = [ f'{10*x}-{10*x+4}' for x in range(0,1025) ]
#ranges = [ f'{10*x}-{10*x+4}' for x in range(0,3000) ]
headers = { 'Range': 'bytes=' + ', '.join(ranges), }
print(f'Count of Range entries: {len(ranges)}')
print(f'Length of Range header: {len(headers["Range"])}')
requests.get(url, headers=headers)
```
</details>
--
Reply to this email directly or view it on GitHub:
https://github.com/xrootd/xrootd/issues/2003
You are receiving this because you are subscribed to this thread.
Message ID: <[log in to unmask]>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
