Hi folks,
I have two proposed improvements for the cmsd protocol. Both are designed to help the scalability of the global namespace project:
1) Namespace IDs: Used by data servers which share a complete namespace (i.e., serve to load-balance access for an external filesystem). In such a case, one only needs to query a single server for file information as all will have the same response. Currently, cmsd will send a query to all servers. For this case, I propose the data servers will all share a namespace ID, a unique string identifier given at configuration time. They can inform their manager/supervisor of this ID, and the manager/supervisor will use this to minimize repeated queries. Obviously, this is just an optimization - if two data servers are assigned to different supervisors, it will not apply.
- Implementation: Add a char* member to CmsLoginData to allow the namespace to be specified, and pass this information to the Link. This char* will be treated as the namespace key. Then, keep a separate bitmask in XrdCmsCluster for namespaces IDs. Upon login, the XrdCmsNode will have a namespace ID assigned to it (based on the namespace key). For any broadcast, keep track of the namespace IDs used when looping through possible nodes, and skip nodes if their namespace ID has already been used.
- The XrdCmsCache object would then operate on namespaces, not nodes.
2) Hierarchical caching: Right now, the cache in a manager/supervisor is a hash table based keyed by the LFN. Since access patterns (especially in CMS) tend to correlate with the filesystem hierarchy, I propose to convert the cache to an inode-like structure. If a disk server does not have a file, it will still reply with -EEXIST, but also return the most specific missing directory. The negative response is given to the cache and used to better filter out namespaces to query.
- Let's say a client is processing directory /a/{1,2,3,4}. The first call (prepare or open) will be to /a/1. If a site doesn't have anything in directory /a/, or if /a/ doesn't exist, the dataserver will respond with -EEXIST,/a. Then, when the client requests /a/2, any namespaces without the /a directory will
- Implementation: I *think* this can be done mostly by keeping the negative directory answers in the XrdCmsPList and having Find filter out namespaces appropriately. XrdCmsPList would be mostly rewritten, but it's only going to cache responses and never trigger any queries. Hence, it should be simpler than XrdCmsCache, although it will need a new timer for expiring entries.
Thoughts? It's not trivial work, but I think the approach is correct.
Brian
|