 |
 |
Hi everyone, Thanks for clarifying things! I agree we should make it work for your use-cases asap and then see how the full feature set should look. I'll also talk to CMS and OSG/StashCache folks to get their input. Cheers, Matevz On 3/5/18 12:57 AM, Andrew Hanushevsky wrote: > Hi Max, > > I ditto as well. Security issues aside, I think we can provide a passthrough > cache with little difficulty. > > Andy > > On Mon, 5 Mar 2018, Fischer, Max (SCC) wrote: > >> Hi all, >> >> for completeness, I have to Ditto Wei Yang's Ditto. >> >> We are trying to push for caching for user analyses in National Analysis >> Facility style - just the bare batch system, as little collaboration >> frameworks as possible. Pass-through caching works wonders in these cases. >> It is true that analysis frameworks can handle the read/write switch, but >> ultimately this is always a workaround. If we rely on frameworks, they will be >> outdated. If we rely on users to manage it themselves, well... >> >> Having pass-through caching is critical to use XRootD in this use-case. >> While I hate to defer security considerations, delegating resource usage (in >> this case writing) to services is widely used in our community anyway. >> >> Cheers, >> Max >> >>> Am 04.03.2018 um 15:24 schrieb Yang, Wei <[log in to unmask]>: >>> >>> Hi Matevz, >>> >>> I think one use case is to have Xcache completely replacing the storage at a >>> site. I see the pain of using two paths, one for read (via xcache) and one >>> for write (bypassing xcache). We can sort this out in application level (such >>> as having atlas pilot to prefix xcache url for read but not for write) but it >>> would be nice to introduce this capability to the xcache so that applications >>> donıt have to think of it. >>> >>> You raised the concern about authorization for writing. Until we sort out >>> proxy delegation, we donıt have a perfect solution. But maybe we can just >>> assume for now the x509 proxy used by xcache will be used, and sort this out >>> later? Andy mentioned to me that pass through is possible. >>> >>> Tengıs approach is interesting but wonıt solve the immediate problem - even >>> if we check in his code now, it takes forever for ATLAS to start using that >>> release of xrootd. >>> >>> regards, >>> -- >>> Wei Yang | [log in to unmask] | 650-926-3338(O) >>> >>> >>> >>> >>> >>> >>> >>> >>> -----Original Message----- >>> From: Matevz Tadel <[log in to unmask]> >>> Date: Friday, March 2, 2018 at 6:33 PM >>> To: Wei Yang <[log in to unmask]>, Fabrizio Furano <[log in to unmask]>, >>> Andrew Hanushevsky <[log in to unmask]> >>> Cc: xrootd-l <[log in to unmask]>, Mihai Patrascoiu >>> <[log in to unmask]>, Matevz Tadel <[log in to unmask]> >>> Subject: Re: Testing the XrdPss with cachelib - Issues with writing >>> >>>> Hi Wei, >>>> >>>> On 3/2/18 1:33 AM, Yang, Wei wrote: >>>>> Dittos >>>> >>>> Can you please elaborate ... in what way would you use this in ATLAS? >>>> >>>> Cheers, >>>> Matevz >>>> >>>>> -- >>>>> Wei Yang | [log in to unmask] | 650-926-3338 >>>>> >>>>> ________________________________________ >>>>> From: [log in to unmask] <[log in to unmask]> on behalf of >>>>> Fabrizio Furano <[log in to unmask]> >>>>> Sent: Friday, March 2, 2018 1:29:59 AM >>>>> To: Hanushevsky, Andrew B. >>>>> Cc: xrootd-l; Mihai Patrascoiu; Matevz Tadel >>>>> Subject: Re: Testing the XrdPss with cachelib - Issues with writing >>>>> >>>>> Hi Andy, >>>>> >>>>> thanks for the hints. My opinion is that a passthrough option would be >>>>> a very big plus for the caching proxy, and I was sincerely surprised >>>>> by this limitation. >>>>> >>>>> For some of our use cases I believe that we will start using it without >>>>> caching, as I can't imagine how to give a transparent service by having >>>>> to force clients to use different paths for reading and writing. >>>>> >>>>> What do you think ? >>>>> >>>>> Thank you >>>>> Fabrizio >>>>> >>>>> On 03/01/2018 06:09 PM, Andrew Hanushevsky wrote: >>>>>> Correct, we don't support a writable cache because that's not what a cache >>>>>> is all about. Does the origin support writes? If so, >>>>>> we could potentially provide a cache passthrough option. Otherwise, if the >>>>>> readable and writable path differ, he could have run >>>>>> two proxies -- one caching for read access and a passthru proxy for >>>>>> writing then setup a redirect directive in the primary proxy >>>>>> (i.e. the one that will be initially used) to direct writes to the >>>>>> passthru proxy based on path. >>>>>> >>>>>> Andy >>>>>> >>>>>> On Thu, 1 Mar 2018, Fabrizio Furano wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Mihai is trying to setup a small Pss machine that also works as a cache. >>>>>>> >>>>>>> For reading it seems to work fine, yet it stubbornly refuses to write >>>>>>> files with >>>>>>> this error: >>>>>>> >>>>>>>> 180301 17:27:38 13788 XrdFileCache_Manager: debug Cache::Attach() >>>>>>>> root:[log in to unmask]:1094//eos/xdc/testing/hello.txt?&oss.asize=14&oss.lcl=1 >>>>>>>> location: <deferred open> >>>>>>>> 180301 17:27:38 13788 root.3008:20@xdc-test-fst1 ofs_fstat: >>>>>>>> fn=/eos/xdc/testing/hello.txt >>>>>>>> 180301 17:27:38 13788 root.3008:20@xdc-test-fst1 XrootdResponse: 0100 >>>>>>>> sending 46 data bytes; status=0 >>>>>>>> 180301 17:27:38 13788 root.3008:20@xdc-test-fst1 XrootdProtocol: 0100 >>>>>>>> req=write dlen=14 >>>>>>>> 180301 17:27:38 13788 root.3008:20@xdc-test-fst1 XrootdProtocol: 0100 >>>>>>>> fh=0 write 14@0 >>>>>>>> 180301 17:27:38 13788 root.3008:20@xdc-test-fst1 ofs_write: 14@0 >>>>>>>> fn=/eos/xdc/testing/hello.txt >>>>>>>> 180301 17:27:38 13788 ofs_write: root.3008:20@xdc-test-fst1 Unable to >>>>>>>> write /eos/xdc/testing/hello.txt; operation not supported >>>>>>>> 180301 17:27:38 13788 root.3008:20@xdc-test-fst1 XrootdProtocol: 0100 >>>>>>>> discarding 0 bytes >>>>>>>> 180301 17:27:38 13788 root.3008:20@xdc-test-fst1 XrootdResponse: 0100 >>>>>>>> sending err 3005: Unable to write >>>>>>>> /eos/xdc/testing/hello.txt; operation not supported >>>>>>>> 180301 17:27:38 13788 root.3008:20@xdc-test-fst1 XrootdProtocol: 0100 >>>>>>>> req=close dlen=0 >>>>>>>> 180301 17:27:38 13788 root.3008:20@xdc-test-fst1 ofs_close: use=1 >>>>>>>> fn=/eos/xdc/testing/hello.txt >>>>>>> >>>>>>> If we remove the directive pss.cachelib then writes work instead. >>>>>>> >>>>>>> Can anyone give us a clue please ? >>>>>>> >>>>>>> Thank you >>>>>>> Fabrizio and Mihai >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> ######################################################################## >>>>> Use REPLY-ALL to reply to list >>>>> >>>>> To unsubscribe from the XROOTD-L list, click the following link: >>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 >>>>> >>>>> ######################################################################## >>>>> Use REPLY-ALL to reply to list >>>>> >>>>> To unsubscribe from the XROOTD-L list, click the following link: >>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 >>>>> >>>> >>> >>> ######################################################################## >>> Use REPLY-ALL to reply to list >>> >>> To unsubscribe from the XROOTD-L list, click the following link: >>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 >> >> > > ######################################################################## > Use REPLY-ALL to reply to list > > To unsubscribe from the XROOTD-L list, click the following link: > https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1