xrd.throttle data 20M IOPS 1000

(data and IOPS can be set separately or not at all). A ReadV with 500 chunks is counted as 500 IO operations.

In the end, I think sites would want to think quite carefully about deploying this: it's a per-server knob, not a site-wide knob. You'd really have to think about the appropriate values to use.

Brian

On Nov 11, 2012, at 9:46 AM, Doug Benjamin <[log in to unmask]> wrote:

Hi Brian,

What are the settings that one should use? Can the throttles be set in terms of Bytes/sec and/or IOPS?
I believe that ultimately even ATLAS will want throttling if we want wide scale acceptance of federated storage.

Cheers,

Doug

On 11/11/12 4:35 PM, Brian Bockelman wrote:

[log in to unmask]" type="cite"> Hi,

I just pushed a new branch to my github repo (https://github.com/bbockelm/xrootd/tree/throttles) which adds basic throttling mechanism to the xrootd server. I know it's not a particularly favorite topic around here, but it is one of the top requests of new sites as CMS requests them to provide access.

The basic concept is to hook into the buffer manager and slow the distribution of buffers to connections. In addition, there's the plumbing to fairshare buffers evenly between users, meaning that one "bad apple" cannot starve everyone else.

The algorithm implemented is to allocate each active user shares every N milliseconds. Unused shares during a round go into the "secondary" shares which can be stolen by other users on a first-come, first-served basis. This allows the server to be fully utilized up to the throttled level when there are active, but slow users. It should also help prevent starvation. All of this is at the cost of being able to violate the throttle by up to a factor of 2 for a single interval as the usage pattern shifts. Both bytes moved and IOPS are tracked.

In the end, it's a fairly straightforward feature. What are the thoughts on getting it merged? There are no backward-compatibility ABI issues, so could go into the next new-feature release.

The two items I have planned are:

1) Make this independent of the buffer manager. The buffering system is orthogonal enough to data rates and IOPS that it likely is simpler to disconnect the two (consider the fact we would prefer to have the buffers reused and in the case of sendfile or mmap'd IO, no buffers are really used).

2) Actually generate the throttle UID's referenced in the code to make the fairshare work.

However, should be far enough along to start gathering opinions from others.

Brian

Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1

Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1