Hi Sergey,
The options is "olb.sched" not "shed" (too save you some frustration). We
don't have a rebalancing tool as of yet. I understand the need with
respect to proof. It's on the list (we'll gladly take contributions :-)
As for new nodes, the 2nd generation olbd called cmsd (new protocol,
better performance, more consistency, etc) allows you to also select nodes
based on space utilization. That goes a long way to avoid packing brand
new nodes that just happen to be empty. It's in beta test as we speak with
a roll out expected in a month for Alice followed by US Atlas.
Andy
On Fri, 22 Feb 2008, Sergey Panitkin wrote:
> Andy,
>
> Thanks a lot for the advice!
> I'll try the "olb.shed" option.
> Perfect spread is not really required - approximate equipartition of datasets
> is good enough.
>
> Is it possible to add a "rebalance" tool to the xrootd set of utilities?
>
> When I extended existing farm with brand new nodes, my new datasets are all
> directed to new machines, since disks there are empty.
> As I understand "opt.sched" will solve the problem for new datasets.
> Great.
>
> But it would be nice to redistribute/rebalance the old datasets over the
> whole extended farm.
>
> I guess, for pure xrootd configuration it doesn't really matter that much,
> but I'm working with PROOF farm.
> And for PROOF local data placement is important.
>
> This would be a great addition to Xrootd and a big help for admins of PROOF
> clusters.
>
> Cheers,
> Sergey
>
>
> Andrew Hanushevsky wrote:
>> Hi Sergey,
>>
>> Actually, youy shouldn't need to do anything, really; if you've set up a
>> default cluster, the redirector will automatically try to load balance
>> using a round-robbin algorithm, this presumes that each server has "enough"
>> space. If either assumption is not the case, then read on...
>>
>> The particular configuration options you need to consider are:
>>
>> olb.sched
>> olb.space
>>
>> If you haven't specified sched then selection will be on a round-robbin
>> basis. Exactly what you want. So, check for that.
>>
>> Make sure that space allows servers to be selected. All the redirector
>> wants to know is that a server has enough space (i.e., more than the disk
>> minimum that can be specified with the space directive). The default is
>> 10GB.
>>
>> When copying files, make sure to copy them in set order. That way, they
>> will be spread out across all your servers. Note, however, copying in new
>> files will not be instantaneous since the redirector tries to confirm that
>> no other copy exists. Therefore, copy them in parallel to amortize the wait
>> cost across the copies. I am in the process is simplifying this by creating
>> a commandline "prepare" program that will allow you to prime the redirector
>> for the incomming files and practically avoid the waits altogether.
>>
>> In any case, don't expect a perfect spread. Timing issues always creep in
>> and make the selection process not 100% deterministic. This usually annoys
>> perfectionsists but, in practice, works just as well.
>>
>> Andy
>>
>> On Fri, 22 Feb 2008, Sergey Panitkin wrote:
>>
>>> Hi experts,
>>>
>>> I would like to copy a set of files to an xrootd farm, so that each farm
>>> node gets an equal number of files from that set, regardless of what the
>>> file sizes are and regardless of available/used disk space on each node.
>>>
>>> Is there an easy way to accomplish this using xrdcp ?
>>>
>>> Cheers,
>>> Sergey
>>>
>
|