 |
 |
Hi, Bill thanks a lot for the info. Amazing - the theoretical max value for the Gbit Ethernet line from the server seems to be reached (taking additional TCP/IP overhead bytes into account). But I was thinking of a kind of benchmark which can be used in an acceptance test for a procurement. Let's say you have 1000 CPU cores (=parallel jobs) and 100 file servers, and you want to make sure that all jobs can be supplied at a constant rate of 4 MB/s with data from the file servers, assuming that they read in a nicely distributed way (not all from the same server). For economic reasons one will not have a true fat tree setup for this kind of network but a kind of blocking configuration, so the switches will be oversubscribed, and one will want to make sure that a criterion like the above can be met. The test should be fairly easy to install and use for the bidders. Your plots indicate that xrootd itself with the correct (larger) block size and mmap files may be used for this, since it is able to fill the connection without undue overhead, i.e. in these cases it indeed probes the network capabilities. I was already toying with the idea of using it for such a test. The clients you used were probably C++ programs (ROOT)? I was at first thinking of limiting the rate at which each client reads and then slowly increase it and look at each server's throughput. But naturally I could also read with full load. I don't know too much about congestion effects at the switches. The test would reflect a realistic situation but it would be convoluted with all kinds of effects. Thanks, Derek On Wednesday 05 April 2006 21.43, Bill Weeks wrote: > Hi, > I have been doing a lot of benchmarking of xrootd using memory mapped > files, e.g. no disk I/O. Mostly I have focused on latency as opposed to > bandwidth, but I can show you a bandwidth result using multiple clients. > All the machines in this test are using GigE. The test uses a single > xrootd server and up to 40 clients. > > Basically, the bandwidth depends on the client blocksize with the > network being the bottleneck with as few as 5 clients at the larger > blocksizes. At the smaller blocksizes, the xrootd server bottlenecks > on the CPU. Since the test is not doing any disk I/O, it shows the > peak capability of the xrootd server. > > I've attached two plots, one showing the aggregate bandwidth, and the > other the average client bandwidth. > --Bill Weeks -- Dr. Derek Feichtinger Tel: +41 56 310 47 33 AIT Group email: [log in to unmask] PSI http://people.web.psi.ch/feichtinger CH-5232 Villigen PSI