Hi Sergey,
Actually, youy shouldn't need to do anything, really; if you've set up a
default cluster, the redirector will automatically try to load balance
using a round-robbin algorithm, this presumes that each server has "enough"
space. If either assumption is not the case, then read on...
The particular configuration options you need to consider are:
olb.sched
olb.space
If you haven't specified sched then selection will be on a round-robbin
basis. Exactly what you want. So, check for that.
Make sure that space allows servers to be selected. All the redirector
wants to know is that a server has enough space (i.e., more than the disk
minimum that can be specified with the space directive). The default is
10GB.
When copying files, make sure to copy them in set order. That way, they
will be spread out across all your servers. Note, however, copying in new
files will not be instantaneous since the redirector tries to confirm that
no other copy exists. Therefore, copy them in parallel to amortize the wait
cost across the copies. I am in the process is simplifying this by
creating a commandline "prepare" program that will allow you to prime the
redirector for the incomming files and practically avoid the waits
altogether.
In any case, don't expect a perfect spread. Timing issues always creep in
and make the selection process not 100% deterministic. This usually annoys
perfectionsists but, in practice, works just as well.
Andy
On Fri, 22 Feb 2008, Sergey Panitkin wrote:
> Hi experts,
>
> I would like to copy a set of files to an xrootd farm, so that each farm node
> gets an equal number of files from that set, regardless of what the file
> sizes are and regardless of available/used disk space on each node.
>
> Is there an easy way to accomplish this using xrdcp ?
>
> Cheers,
> Sergey
>
|