Hi Alja,
Yes, please see the email I just sent out to explain why it works the way
it does and what to do to make it work the way you want.
Andy
On Fri, 5 Dec 2014, Alja Mrak-Tadel wrote:
> Hi Andy,
>
> I copied log files to:
> http://uaf-2.t2.ucsd.edu/~alja/xrd/
>
> Where redirector is cabinet-10-10-10 and the proxy servers are
> cabinet-10-10-11, cabinet-10-10-9, and cabinet-10-10-6.
>
>
> Before running the redirector test '/store/user/matevz/A_FILE' existent only
> on cabinet-10-10-6. From the redirector log you can see stat was made on
> correct node (cabinet-10-10-6), but later it went to cabinet-10-10-9 to read
> it.
>
> Alja
>
>
>
> On 12/05/14 14:27, Andrew Hanushevsky wrote:
>> Hi Matevz,
>>
>> That doesn't sound rightand it should not work that way so something
>> else is going on. Could you rerun that with "-d" command line option to
>> capture the inconsistency and send me the log.
>>
>> Andy
>>
>> On Fri, 5 Dec 2014, Matevz Tadel wrote:
>>
>>> Hi,
>>>
>>> Alja and I have been testing a caching-proxy cluster, the idea is that
>>> there are several independent caching-proxies all reporting to a
>>> common redirector so that one gets more disk space, better performance
>>> and redundancy. Here are the scripts we've been using:
>>>
>>> http://uaf-2.t2.ucsd.edu/~matevz/xrd/proxy-cluster/
>>>
>>> pooxy-klus.cfg is the config and start-pooxy.sh the startup script.
>>> cabinet-10-10-10 is the redirector.
>>>
>>> All servers export /store with the stage option and there is a trivial
>>> oss.stagecmd /opt/stage-fake.pl that basically just touches the file,
>>> assuming that proxy will then pull in the parts of the file that are
>>> actually needed.
>>>
>>> On proxy servers, we configure xrootd to use
>>> ofs.osslib /opt/xrootd/lib64/libXrdPss-4.so
>>> pss.origin xrootd.t2.ucsd.edu:1094
>>> pss.cachelib libXrdFileCache-4.so
>>> while cmsd just gets
>>> xrootd.fslib /opt/xrootd/lib64/libXrdOfs-4.so
>>> so that it is able to look what files already exist on the disk.
>>>
>>> The whole thing works ... but apparently stage option is too strong.
>>> When opening a file that exists on proxy machine A it also happens
>>> that the client gets redirected to machine X that does not have it --
>>> so it has to be pulled in from remote location one more time.
>>>
>>> In particular, this happens after the cluster restart. xrdcp is used
>>> to access the files. xrdcp does stat before the open. both are
>>> redirected randomly the first time after restart ... after that the
>>> server that was selected for open keeps getting selected consistently.
>>>
>>> Any ideas what we could do? Or should we be looking for a completely
>>> different solution?
>>>
>>> Could it be proxy cmsds are not seeing the files right? I tried
>>> bumping trace to debug but there is no info there about found/not
>>> found cmsd queries.
>>>
>>> Cheers,
>>> Matevz
>>>
>>> ########################################################################
>>> Use REPLY-ALL to reply to list
>>>
>>> To unsubscribe from the XROOTD-DEV list, click the following link:
>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>>>
>>
>> ########################################################################
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the XROOTD-DEV list, click the following link:
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
|