 |
 |
Hi Afrian, Answers withon the email... On Fri, 14 Jun 2019, Adrian Sevcenco wrote: > Hi! What would be the recommended settings for an xrootd installation > that use a distributed file system like gluster? I can't recommend specific settings but can indicate required directives to allow the whole thing to correctly function. You need to specify the cms.dfs directive. http://xrootd.org/doc/dev410/cms_config.htm#_Toc8247261 This tells the cmsd that you are using a distributed file system. It needs to know that to change file location to be consistent with DFS semantics. > beside the dfs what other tweaks will make sure that the load is equally > distributed among the file servers? You will need to enable load scheduling to get the best effect here. The default is round-robbin which works in many cases but is far from ideal when you want to avoid hot spots. Consult the cms.sched and cms.perf directives: http://xrootd.org/doc/dev410/cms_config.htm#_Toc8247267 http://xrootd.org/doc/dev410/cms_config.htm#_Toc8247264 > it would seem that the redirector remembers the last data server that > answered a file query so the same server is used again ...how can this be > disabled? No need to disable anything here if you use the cms.dfs directive. > also, given the uniformity of the storage space, is there a need for a > redirector?(or cmsd)? could a simple xrootd (and with dns aliasing) > distribute the load equally? Using DNS will not distribute load equally and will cause severe latency issues should one or more of the servers become unavailable. While it may appear to work you will soon find that it works very poorly. > also, for really large installations (and not only) i seen that there is a > mechanism for keeping a metadata cache of all namespace(s) - XrdCnsd Yes, we have tried to deprecate that but people come up wanting it from time to time; so, it still has a life of its own. > can this be used to bypass the metadata query of files and reduce the load of > the actual block device? No, the XrdCnsd is meant to keep an inventory so that should you loose a disk array you will know what you lost. It is not a sufficiently real-time inventory suitable for file lookups. > also, i seen the in the documentation the XrdCnsd is started by xrootd, > which i do not think is a safe model (service started by a service) > is there a way for ofs.notify to send the information to an already started > service? Strange, without an xrootd running there is no reason to have a cnsd running so it makes sense to have the cnsd treated as a helper service to make sure you do not loose any events; which may happen if they are run as independent services. But to answer your question, yes, there is. Route the messages to a pipe and tell the cnsd to get the messages from the pipe. In order for this to be as rugged as you apparently want you would actually route the mesages to something like activeMQ which would in turn feed the cnsd, though that becomes relatively complicated and I would not recommend that. Also, the pipe solution comes with it's own set of issues as you can imagine. > Also, this inventory is a collection of what? in the data servers the > namespace is a collection of symlinks The inventory is a collection of files that exist in the xrootd exported namespace. It is nothing more than that. People wanted to track file creations and removals to have an inventory of what is on the server. It was specifically created for nothing-shared clusters. Nobody has used it in a DFS cluster; so we have no experience using it with a DFS. Andy ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1