 |
 |
On Jan 5, 2011, at 10:31 AM, Brian Bockelman wrote: > Hi folks, > > I have two proposed improvements for the cmsd protocol. Both are designed to help the scalability of the global namespace project: > > 1) Namespace IDs: Used by data servers which share a complete namespace (i.e., serve to load-balance access for an external filesystem). In such a case, one only needs to query a single server for file information as all will have the same response. Currently, cmsd will send a query to all servers. For this case, I propose the data servers will all share a namespace ID, a unique string identifier given at configuration time. They can inform their manager/supervisor of this ID, and the manager/supervisor will use this to minimize repeated queries. Obviously, this is just an optimization - if two data servers are assigned to different supervisors, it will not apply. > - Implementation: Add a char* member to CmsLoginData to allow the namespace to be specified, and pass this information to the Link. This char* will be treated as the namespace key. Then, keep a separate bitmask in XrdCmsCluster for namespaces IDs. Upon login, the XrdCmsNode will have a namespace ID assigned to it (based on the namespace key). For any broadcast, keep track of the namespace IDs used when looping through possible nodes, and skip nodes if their namespace ID has already been used. > - The XrdCmsCache object would then operate on namespaces, not nodes. Actually, I take this part back. XrdCmsCache will still want to work on nodes, but the response from one node will suffice for the rest. Broadcast will only perform one query per namespace and mark the rest unqueried (meaning that if the client comes back again and the node queried from the namespace hasn't responded, we can query one of the other nodes). It will iterate through the nodetable starting at a changing offset so the same node in the namespace isn't used each time. As I said, not trivial. But does seem to be possible. Brian