Hi Artem,
On Wed, Mar 02, 2005 at 01:08:46PM -0800, Artem Trunov wrote:
> > The real problem here is that the person is using a backdoor to modify
> > the file. Had the user modified the file on the disk cache and let it
> > migrate back down, there would not be a problem. Using backdoors opens
> > a whole range of problems, only one of which the user experienced.
>
> It should not be called a back door, it's a perfectly valid situation of
> separation production from analysis. We do this all the time at SLAC. The
> only difference is that we have read-only files, and this simplifies our
> life a lot. Whether D0 user can do it our way really depends on how they
> do their production. But I think that such function Jean-Yves described
> should exists and be configurable for those who find it necessary.
Yes, it is clear that there is a fundamental difference between "add new
file" and "update file" operations. There is no problem with "add new file"
directly into HPSS. For "update file" to happen via some interface other
than xrootd itself you would presumably need a true cache manager on the
backend (as opposed to the combination of the mps scripts and HPSS). The
question is then one of making sure that the advantages of xrootd/olbd
are not lost in the process and Andy has made a proposal as to how SRM
could be used on the backend to do that. I think trying to retrofit this
functionality onto the HPSS+mps+xrootd/olbd combination would be somewhat
messy.
Pete
> > I'm not opposed to putting in an option to do the check but I think
> > serious consideration needs to be given whether this is *really* the
> > mode of operation you want or will be happy with.
> >
> > Andy
> >
> > On Wed, 2 Mar 2005, Jean-Yves Nief wrote:
> >
> > > hello all,
> > >
> > > one of the D0 user accessing files via xrootd encountered the
> > > following issue: after having accessed a file via xrootd (so after being
> > > staged from the "master" copy stored in HPSS), he modified the master
> > > copy in HPSS and wanted to access the modified file via xrootd: but as
> > > the old version of the file was already on the disk cache, no staging
> > > occured of course (but that is the expected behavior obviously) and he
> > > grabbed the old one version, which is not what he wanted. Well as an
> > > emergency solution and as it was the first time it happened, I've
> > > deleted the old version on the disk cache so he could proceed.
> > > However, I think it would be nice to have some control on the validity
> > > of the cache: one solution would be to add the following test: in case
> > > the file is already in the cache, compare the creation time on the cache
> > > disk (t1) with the last modified time of the file stored in HPSS (t2):
> > > if t1<t2 then restage the file.
> > > it will be a very small overhead to the mechanism, each time a file is
> > > accessed: it have just to issue a "statx" request to the MSS.
> > > or maybe there is a more simple solution.
> > > cheers,
> > > JY
> > >
> > >
> >
-------------------------------------------------------------------------
Peter Elmer E-mail: [log in to unmask] Phone: +41 (22) 767-4644
Address: CERN Division PPE, Bat. 32 2C-14, CH-1211 Geneva 23, Switzerland
-------------------------------------------------------------------------
