 |
 |
Hi Andy, yes, I have spent the last few years on various caches and I see the pain. For the use cases that I know about the restriction that you mention would be more than acceptable, and the "total" solution would probably have limited usefulness (and a high development cost). Also not caching (hence just tunnelling) a file that is opened for write would be fine. At the end most of the heavy use cases are just WORM, and whoever enables this is supposed to be aware of the limitations... hopefully. Also limiting the prohibition to opens for append/mpodify would be fine I guess. Fabrizio On 03/02/2018 10:46 AM, Andrew Hanushevsky wrote: > Hi Fabrizio, > > The reason we didn't do it for a disk caching proxy is because cache synchrnization is extremely difficult as you may be reading > as well as writing the file. The memory caching proxy does support reading and writing as synchnizing a memory cache is trivial. > So, even if we did a passthrough option it would be very restrictive (i.e. you can either read the data or write the data but > not both unless the cache is disabled and opening a file for writing would necessarily have to flush the cache for that file). > That we could also do. > > Probably the fastest solution here (at the moment) is to write an ofs plugin wrapper for the caching proxy that checks for write > opens and does the appropriate redirection. We can do that also by extending the static redirect directive to differentiate > between read paths and write paths; thuogh that opens the proxy to cache synchronization issues unless files are not bth read > and written at the same time. > > Andy > > On Fri, 2 Mar 2018, Fabrizio Furano wrote: > >> Hi Andy, >> >> thanks for the hints. My opinion is that a passthrough option would be >> a very big plus for the caching proxy, and I was sincerely surprised >> by this limitation. >> >> For some of our use cases I believe that we will start using it without >> caching, as I can't imagine how to give a transparent service by having >> to force clients to use different paths for reading and writing. >> >> What do you think ? >> >> Thank you >> Fabrizio >> >> On 03/01/2018 06:09 PM, Andrew Hanushevsky wrote: >>> Correct, we don't support a writable cache because that's not what a cache is all about. Does the origin support writes? If so, >>> we could potentially provide a cache passthrough option. Otherwise, if the readable and writable path differ, he could have run >>> two proxies -- one caching for read access and a passthru proxy for writing then setup a redirect directive in the primary proxy >>> (i.e. the one that will be initially used) to direct writes to the passthru proxy based on path. >>> >>> Andy >>> >>> On Thu, 1 Mar 2018, Fabrizio Furano wrote: >>> >>>> Hi, >>>> >>>> Mihai is trying to setup a small Pss machine that also works as a cache. >>>> >>>> For reading it seems to work fine, yet it stubbornly refuses to write files with >>>> this error: >>>> >>>>> 180301 17:27:38 13788 XrdFileCache_Manager: debug Cache::Attach() >>>>> root:[log in to unmask]:1094//eos/xdc/testing/hello.txt?&oss.asize=14&oss.lcl=1 location: <deferred open> >>>>> 180301 17:27:38 13788 root.3008:20@xdc-test-fst1 ofs_fstat: fn=/eos/xdc/testing/hello.txt >>>>> 180301 17:27:38 13788 root.3008:20@xdc-test-fst1 XrootdResponse: 0100 sending 46 data bytes; status=0 >>>>> 180301 17:27:38 13788 root.3008:20@xdc-test-fst1 XrootdProtocol: 0100 req=write dlen=14 >>>>> 180301 17:27:38 13788 root.3008:20@xdc-test-fst1 XrootdProtocol: 0100 fh=0 write 14@0 >>>>> 180301 17:27:38 13788 root.3008:20@xdc-test-fst1 ofs_write: 14@0 fn=/eos/xdc/testing/hello.txt >>>>> 180301 17:27:38 13788 ofs_write: root.3008:20@xdc-test-fst1 Unable to write /eos/xdc/testing/hello.txt; operation not >>>>> supported >>>>> 180301 17:27:38 13788 root.3008:20@xdc-test-fst1 XrootdProtocol: 0100 discarding 0 bytes >>>>> 180301 17:27:38 13788 root.3008:20@xdc-test-fst1 XrootdResponse: 0100 sending err 3005: Unable to write >>>>> /eos/xdc/testing/hello.txt; operation not supported >>>>> 180301 17:27:38 13788 root.3008:20@xdc-test-fst1 XrootdProtocol: 0100 req=close dlen=0 >>>>> 180301 17:27:38 13788 root.3008:20@xdc-test-fst1 ofs_close: use=1 fn=/eos/xdc/testing/hello.txt >>>> >>>> If we remove the directive pss.cachelib then writes work instead. >>>> >>>> Can anyone give us a clue please ? >>>> >>>> Thank you >>>> Fabrizio and Mihai >>>> >>>> >>> >> >> ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1