Hi Brian,

I think what you really want, on the first order, is time-zone selection. 
That is on our short-term plan. All servers currently report time-zone the 
old client does not but the new client will (we obviously can back-port to 
the old client). While this doesn't solve Nort/South issues it gets close 
enough for now. Our longer term plan is to have the client rebroker the 
selection based on actual server performance; which as you have noted is 
more important than the place your getting data from.

Andy

On Fri, 6 Jul 2012, Brian Bockelman wrote:

> Hi,
>
> Yesterday, two different people bugged me about smarter redirection algorithms, so I figured I should start thinking about them again.  I have come up with a slight modification of the existing redirection algorithm.  This modification, explained in detail below, reuses most of the existing, is effectively a no-op in the default case, and is hopefully a very small modification.
>
> Basically, it seems we need to be able to do three things:
>
> (0) Pass client information to the cmsd.  [Pretty easy - this can be added to the opaque information by the cmsd client.]
> (1) Make a decision about which nodes to query on behalf of a client.  I.e., there may be some nodes that are too costly for a particular client, and should only be queried as a last-resort.  The obvious example is not querying sites across the Atlantic if there are sites on this side not yet queried.  Reducing the number of sites queried reduces the load on the entire system.
> (2) Of the nodes with the file, the cmsd should be able to prefer one set over another according to which ones are least costly.  If there are nodes on both sides of the Atlantic available to serve the file, the cmsd should select the local one.
>
> Right now, there is an XMI interface that allows us to bypass the entire cmsd stack and implement our own.  I'd *really* like to avoid that, because the first thing I would need to do is re-implement most of the existing cmsd stack.
>
> Instead, I propose a new XMI callout to provide a "preference mask".  The preference mask is a fixed-size array of Smask_t.  A node's preference is the highest position in the array where it is selected.  So, if bit 3 is in mask[N], but not mask[N+1], its preference is N (highest preference "wins").
>
> The default preference mask is, when no callout is provided, [0xFFFFFFFF, 0x0].  The preference mask is determined based on the path and opaque information (which will be extended to, at least, contain the client hostname), and passed to the XrdCmsCluster.Select method.
>
> XrdCmsSelect can be extended to keep track of all nodes which have been queried.  For a new query, only nodes with the highest preference are queried.  Each time the client returns back to XrdCmsCluster.Select, it iterates backwards through the preference masks until it finds one that does not have nodes queried.
>
> For example - suppose a client is determined to be on the US West coast.  The preference mask would be (1) all known nodes, (2) all US nodes, (3) all California nodes.  The first time it asks for a file, the server will broadcast the request to (3).  It will either (a) find the file and go to SelNode, (b) not get back responses within the fast-lookup time and delay the client or (c) determine all nodes of preference 3 do not have the file.  In the case of (c), it will immediately broadcast a query to nodes of preference (2) and delay the client.
>
> The next time the client returns, the preference mask is calculated from the opaque info.  Assuming case (b) above for illustration, if the file location is still not known, XrdCmsCluster will see that all the nodes in preference (3) are already queried and broadcast a query to nodes of preference (2).  Note that we do not wait until the query has timed-out for nodes of preference (3) before querying (2); we just wait until the next client re-connect.
>
> To make sure you understand how it works, figure out what happens if a client from the west coast and a client from Germany (assume their preferences for 2 and 3 are non-overlapping) both query within a second of each other.
>
> Now, when one or more nodes are found, we also reuse the preference mask in SelNode to determine which one is returned to the client.  Iterate through the preference masks - most specific to least specific - to find the first case where the intersection of the mask and the available nodes is nonzero.  Then, apply the "normal" weighted selection criteria to this set.
>
> For example - suppose we've determined a file is in Lincoln, Chicago and London; the client is again on the US West coast.  The intersection of preference (3) nodes and the file locations is null.  Then, we consider the intersection of (3) and the locations; this returns only Lincoln and Chicago.  These two sites are used for the remainder of the logic in SelNode.
>
> While I don't think our region-based system is going away anytime soon, I think this will help with clients starting at the global redirector.  This should also help us grow as we wouldn't need to maintain many layers of regions.  We could have a three layer scheme (Global, Continental, Local) with sufficiently smart redirectors so we don't need to add another layer of the hierarchy between Continental and Local.
>
> Thoughts?  I think this is simple enough I can implement it within the existing code.
>
> Brian
>
> ########################################################################
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the XROOTD-DEV list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>

########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1