We're seeing climbing xrootd memory usage on a local redirector. From some initial debugging, it seems due to XrdHttpProtocol buffers. This patch frees the 1MB buffer when the XrdHttpProtocol objects are idle.
The HTTP protocol handler keeps a stack of idle XrdHttpProtocol objects in ProtStack. The stack depth is limited to ConnMax/3, and ConnMax is set to RLIMIT_NOFILE. Each idle handler holds a 1MB buffer. When LimitNOFILE is increased to 65k for better concurrency, it's possible to have ~21k XrdHttpProtocol objects in ProtStack for a total of ~21GB RAM allocated to buffers. This seems unexpectedly high, especially for redirectors.
Rather than adding another configuration knob to limit the memory usage, we could free the 1MB buffer when the object is idle. With XrdBuffManager already caching blocks of RAM, I'd hope there isn't too much overhead or fragmentation.
To show the high RAM usage, open a large number of concurrent HTTP connections:
ab -n 2000 -c 2000 https://xrootd.example.edu:1094/file
https://github.com/xrootd/xrootd/pull/939
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1