Hi, Bill
thanks a lot for the info. Amazing - the theoretical max value for the Gbit
Ethernet line from the server seems to be reached (taking additional TCP/IP
overhead bytes into account).
But I was thinking of a kind of benchmark which can be used in an acceptance
test for a procurement. Let's say you have 1000 CPU cores (=parallel jobs)
and 100 file servers, and you want to make sure that all jobs can be supplied
at a constant rate of 4 MB/s with data from the file servers, assuming that
they read in a nicely distributed way (not all from the same server). For
economic reasons one will not have a true fat tree setup for this kind of
network but a kind of blocking configuration, so the switches will be
oversubscribed, and one will want to make sure that a criterion like the
above can be met.
The test should be fairly easy to install and use for the bidders.
Your plots indicate that xrootd itself with the correct (larger) block size
and mmap files may be used for this, since it is able to fill the connection
without undue overhead, i.e. in these cases it indeed probes the network
capabilities. I was already toying with the idea of using it for such a test.
The clients you used were probably C++ programs (ROOT)?
I was at first thinking of limiting the rate at which each client reads and
then slowly increase it and look at each server's throughput. But naturally I
could also read with full load. I don't know too much about congestion
effects at the switches. The test would reflect a realistic situation but it
would be convoluted with all kinds of effects.
Thanks,
Derek
On Wednesday 05 April 2006 21.43, Bill Weeks wrote:
> Hi,
> I have been doing a lot of benchmarking of xrootd using memory mapped
> files, e.g. no disk I/O. Mostly I have focused on latency as opposed to
> bandwidth, but I can show you a bandwidth result using multiple clients.
> All the machines in this test are using GigE. The test uses a single
> xrootd server and up to 40 clients.
>
> Basically, the bandwidth depends on the client blocksize with the
> network being the bottleneck with as few as 5 clients at the larger
> blocksizes. At the smaller blocksizes, the xrootd server bottlenecks
> on the CPU. Since the test is not doing any disk I/O, it shows the
> peak capability of the xrootd server.
>
> I've attached two plots, one showing the aggregate bandwidth, and the
> other the average client bandwidth.
> --Bill Weeks
--
Dr. Derek Feichtinger Tel: +41 56 310 47 33
AIT Group email: [log in to unmask]
PSI http://people.web.psi.ch/feichtinger
CH-5232 Villigen PSI
|