Hi Andy,
I copied log files to:
http://uaf-2.t2.ucsd.edu/~alja/xrd/
Where redirector is cabinet-10-10-10 and the proxy servers are
cabinet-10-10-11, cabinet-10-10-9, and cabinet-10-10-6.
Before running the redirector test '/store/user/matevz/A_FILE' existent
only on cabinet-10-10-6. From the redirector log you can see stat was
made on correct node (cabinet-10-10-6), but later it went to
cabinet-10-10-9 to read it.
Alja
On 12/05/14 14:27, Andrew Hanushevsky wrote:
> Hi Matevz,
>
> That doesn't sound rightand it should not work that way so something
> else is going on. Could you rerun that with "-d" command line option to
> capture the inconsistency and send me the log.
>
> Andy
>
> On Fri, 5 Dec 2014, Matevz Tadel wrote:
>
>> Hi,
>>
>> Alja and I have been testing a caching-proxy cluster, the idea is that
>> there are several independent caching-proxies all reporting to a
>> common redirector so that one gets more disk space, better performance
>> and redundancy. Here are the scripts we've been using:
>>
>> http://uaf-2.t2.ucsd.edu/~matevz/xrd/proxy-cluster/
>>
>> pooxy-klus.cfg is the config and start-pooxy.sh the startup script.
>> cabinet-10-10-10 is the redirector.
>>
>> All servers export /store with the stage option and there is a trivial
>> oss.stagecmd /opt/stage-fake.pl that basically just touches the file,
>> assuming that proxy will then pull in the parts of the file that are
>> actually needed.
>>
>> On proxy servers, we configure xrootd to use
>> ofs.osslib /opt/xrootd/lib64/libXrdPss-4.so
>> pss.origin xrootd.t2.ucsd.edu:1094
>> pss.cachelib libXrdFileCache-4.so
>> while cmsd just gets
>> xrootd.fslib /opt/xrootd/lib64/libXrdOfs-4.so
>> so that it is able to look what files already exist on the disk.
>>
>> The whole thing works ... but apparently stage option is too strong.
>> When opening a file that exists on proxy machine A it also happens
>> that the client gets redirected to machine X that does not have it --
>> so it has to be pulled in from remote location one more time.
>>
>> In particular, this happens after the cluster restart. xrdcp is used
>> to access the files. xrdcp does stat before the open. both are
>> redirected randomly the first time after restart ... after that the
>> server that was selected for open keeps getting selected consistently.
>>
>> Any ideas what we could do? Or should we be looking for a completely
>> different solution?
>>
>> Could it be proxy cmsds are not seeing the files right? I tried
>> bumping trace to debug but there is no info there about found/not
>> found cmsd queries.
>>
>> Cheers,
>> Matevz
>>
>> ########################################################################
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the XROOTD-DEV list, click the following link:
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>>
>
> ########################################################################
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the XROOTD-DEV list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
|