Hi Matevz,
You are getting caught in the lookup" phase. Distributed lookup will always
scale better then central lookup, when a lookup *has* to be performed. The
redirect part is what to do when a lookup can be avoided because the
information is already cached. Immed is always the best option is you have a
true distributed file system underneath.
Anyway, I can't say that I have convinced people that distributed normally
has better scaling, and I have tried. Unfortunately, the majority still
seems to gravitate to centralized vertical design options because they are
more comforting.
Andy
-----Original Message-----
From: Matevz Tadel
Sent: Tuesday, February 04, 2014 2:46 PM
To: xrootd-dev
Subject: cms.dfs question
Hi,
We (AAA) are doing redirection rate scaling tests and noticed a large
difference
between *hadoop* sites based on how cms.dfs is setup.
This works great (scaling beyond 300Hz):
cms.dfs lookup distrib redirect immed
and this saturates at ~20Hz:
cms.dfs lookup central redirect immed
I'm puzzled, because I'd expect that "redirect immed" trumps whatever lookup
setting one might choose. We were lucky -- we had two sites that chose
different
values for lookup :)
Matevz
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
|