 |
 |
Hi, I just pushed a new branch to my github repo (https://github.com/bbockelm/xrootd/tree/throttles) which adds basic throttling mechanism to the xrootd server. I know it's not a particularly favorite topic around here, but it is one of the top requests of new sites as CMS requests them to provide access. The basic concept is to hook into the buffer manager and slow the distribution of buffers to connections. In addition, there's the plumbing to fairshare buffers evenly between users, meaning that one "bad apple" cannot starve everyone else. The algorithm implemented is to allocate each active user shares every N milliseconds. Unused shares during a round go into the "secondary" shares which can be stolen by other users on a first-come, first-served basis. This allows the server to be fully utilized up to the throttled level when there are active, but slow users. It should also help prevent starvation. All of this is at the cost of being able to violate the throttle by up to a factor of 2 for a single interval as the usage pattern shifts. Both bytes moved and IOPS are tracked. In the end, it's a fairly straightforward feature. What are the thoughts on getting it merged? There are no backward-compatibility ABI issues, so could go into the next new-feature release. The two items I have planned are: 1) Make this independent of the buffer manager. The buffering system is orthogonal enough to data rates and IOPS that it likely is simpler to disconnect the two (consider the fact we would prefer to have the buffers reused and in the case of sendfile or mmap'd IO, no buffers are really used). 2) Actually generate the throttle UID's referenced in the code to make the fairshare work. However, should be far enough along to start gathering opinions from others. Brian ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1