Hi Matevz,
Below is the sample config fle that will appear in he manual. Once we
resolve the "statlib" issue, I will add that as well.
Andy
# Tell everyone who the manager is
#
all.manager redirector:1213
# Everyone will export /data red-only with the stage option.
# The stage option requests that if the file isn't found in
# the cluster the redirector should send the client to a PFC
# server with enough space to cache the file.
#
all.export /data stage r/o
# First configure the redirector using if-else-fi
#
if redirector
all.role manager
else
# All the PFC's act as virtual data servers
#
all.role server
# However, load the proxy plugin and the disk caching plugin.
# In reality it will function as a disk caching proxy server.
#
ofs.osslib libXrdPss.so
pss.cachelib libFileCache.so
# Tell the proxy where the data is coming from (arbitrary).
#
pss.origin someserver.domain.org
# Tell the PFC's where the disk cache resides (arbitrary).
#
pfc.cachedir /pfc-cache
On Fri, 5 Dec 2014, Matevz Tadel wrote:
> Hi,
>
> Alja and I have been testing a caching-proxy cluster, the idea is that there
> are several independent caching-proxies all reporting to a common redirector
> so that one gets more disk space, better performance and redundancy. Here are
> the scripts we've been using:
>
> http://uaf-2.t2.ucsd.edu/~matevz/xrd/proxy-cluster/
>
> pooxy-klus.cfg is the config and start-pooxy.sh the startup script.
> cabinet-10-10-10 is the redirector.
>
> All servers export /store with the stage option and there is a trivial
> oss.stagecmd /opt/stage-fake.pl that basically just touches the file,
> assuming that proxy will then pull in the parts of the file that are actually
> needed.
>
> On proxy servers, we configure xrootd to use
> ofs.osslib /opt/xrootd/lib64/libXrdPss-4.so
> pss.origin xrootd.t2.ucsd.edu:1094
> pss.cachelib libXrdFileCache-4.so
> while cmsd just gets
> xrootd.fslib /opt/xrootd/lib64/libXrdOfs-4.so
> so that it is able to look what files already exist on the disk.
>
> The whole thing works ... but apparently stage option is too strong. When
> opening a file that exists on proxy machine A it also happens that the client
> gets redirected to machine X that does not have it -- so it has to be pulled
> in from remote location one more time.
>
> In particular, this happens after the cluster restart. xrdcp is used to
> access the files. xrdcp does stat before the open. both are redirected
> randomly the first time after restart ... after that the server that was
> selected for open keeps getting selected consistently.
>
> Any ideas what we could do? Or should we be looking for a completely
> different solution?
>
> Could it be proxy cmsds are not seeing the files right? I tried bumping trace
> to debug but there is no info there about found/not found cmsd queries.
>
> Cheers,
> Matevz
>
> ########################################################################
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the XROOTD-DEV list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
|