Hi Brian,

I took a very deliberate look at the proposal. Well, oddly, it looks like 
cmsd is about 95% already there and to get it to 100% shouldn't really 
require a protocol change. So, let's take this one point at a time:

1)      The cmsd is already capable of handling servers that front a common 
filesystem. That's what "role proxy" helps to do. Does it work? Well, 
almost. The non-query feature was never implemented because the way people 
used "proxy" didn't require it. So, with a bit of tweaking you will be able 
to front a shared filesystem with a proxy cluster and that cluster will 
never ask whether you have a file or not. It will simply assume you do and 
if you don't it won't matter because nobody else does.



2)      I appreciate the idea of improving the name caching but the 
particular set of algorithms used to cache entries in the current scheme 
each run in constant time (that includes cache management which is actually 
a much bigger overhead item). I am not at all sure that creating an inode 
structure will provide any significant improvement and, in fact, because the 
data structure is more complicated, will likely make it less efficient. Let's 
say I'm highly skeptical.



Now, if point (2) is actually driven by point (1), which is your desire to 
minimize the number of queries to a shared filesystem, then there are 
actually better ways of doing this in the redirector itself. For instance, 
when dealing with shared filesystem there is no reason why the redirector 
can bypass querying servers and should be able to do the query itself. In 
fact, we will make this more convenient (it's already possible) largely to 
improve the efficiency of firewalled servers relative to meta-managers.



If the above is not the case, then I'm somewhat at a loss about what problem 
is being solved here. Surely, the hit rate is much higher than the miss rate 
in any properly functioning analysis scenario (in fact, the expected hit 
rate should be 100%). Under that assumption, trying to optimize negative 
responses is not a particularly effective strategy. On the other hand, the 
following sentence in your mail file is incomplete:



"Let's say a client is processing directory /a/{1,2,3,4}.  The first call 
(prepare or open) will be to /a/1.  If a site doesn't have anything in 
directory /a/, or if /a/ doesn't exist, the dataserver will respond 
with -EEXIST,/a.  Then, when the client requests /a/2, any namespaces 
without the /a directory will"...



So, perhaps I'm missing something because I don't know what the "will" will 
be :-)



Andy



----- Original Message ----- 
From: "Brian Bockelman" <[log in to unmask]>
To: "xrootd-dev" <[log in to unmask]>
Sent: Wednesday, January 05, 2011 8:31 AM
Subject: Proposed improvements for cmsd protocol


Hi folks,

I have two proposed improvements for the cmsd protocol.  Both are designed 
to help the scalability of the global namespace project:

1) Namespace IDs: Used by data servers which share a complete namespace 
(i.e., serve to load-balance access for an external filesystem).  In such a 
case, one only needs to query a single server for file information as all 
will have the same response.  Currently, cmsd will send a query to all 
servers.  For this case, I propose the data servers will all share a 
namespace ID, a unique string identifier given at configuration time.  They 
can inform their manager/supervisor of this ID, and the manager/supervisor 
will use this to minimize repeated queries.  Obviously, this is just an 
optimization - if two data servers are assigned to different supervisors, it 
will not apply.
  - Implementation: Add a char* member to CmsLoginData to allow the 
namespace to be specified, and pass this information to the Link.  This 
char* will be treated as the namespace key.  Then, keep a separate bitmask 
in XrdCmsCluster for namespaces IDs.  Upon login, the XrdCmsNode will have a 
namespace ID assigned to it (based on the namespace key).  For any 
broadcast, keep track of the namespace IDs used when looping through 
possible nodes, and skip nodes if their namespace ID has already been used.
  - The XrdCmsCache object would then operate on namespaces, not nodes.
2) Hierarchical caching:  Right now, the cache in a manager/supervisor is a 
hash table based keyed by the LFN.  Since access patterns (especially in 
CMS) tend to correlate with the filesystem hierarchy, I propose to convert 
the cache to an inode-like structure.  If a disk server does not have a 
file, it will still reply with -EEXIST, but also return the most specific 
missing directory.  The negative response is given to the cache and used to 
better filter out namespaces to query.
  - Let's say a client is processing directory /a/{1,2,3,4}.  The first call 
(prepare or open) will be to /a/1.  If a site doesn't have anything in 
directory /a/, or if /a/ doesn't exist, the dataserver will respond 
with -EEXIST,/a.  Then, when the client requests /a/2, any namespaces 
without the /a directory will
  - Implementation: I *think* this can be done mostly by keeping the 
negative directory answers in the XrdCmsPList and having Find filter out 
namespaces appropriately.  XrdCmsPList would be mostly rewritten, but it's 
only going to cache responses and never trigger any queries.  Hence, it 
should be simpler than XrdCmsCache, although it will need a new timer for 
expiring entries.

Thoughts?  It's not trivial work, but I think the approach is correct.

Brian