Hi Brian,
OK, you got you wish along with a way to bypass it (the default being "keep
it local"). Now, are you going to be at the XRootD Workshop? If not, can
someone be there to give a presentation on the latest developments on
SciTokens et al?
Andy
-----Original Message-----
From: Bockelman, Brian
Sent: Monday, May 06, 2019 1:39 AM
To: Andrew Hanushevsky
Cc: Matevz Tadel ; Michal Kamil Simon ; xrootd-dev
Subject: Re: Proposal for new opaque URL parameter using= complementing
tried=
Hi Andy,
In this case - there's good reason to not send clients offsite (even if the
offsite server is providing better performance, the WAN costs more...) when
there's a perfectly good copy onsite. We can be sure to drop the "resel"
from the "triedrc" when the job is looking for an additional source because
of an error instead of wanting faster sources.
I think it would be useful to keep the "file not found" code to only trigger
when the file is actually not found.
Brian
> On May 6, 2019, at 7:35 AM, Andrew Hanushevsky <[log in to unmask]>
> wrote:
>
> Well, isn't the point of reselection is to find the best possible site
> which could be offsite? We could give you an option to keep it local but
> you would need to add that to the conig file.
>
> On Sun, 5 May 2019, Bockelman, Brian wrote:
>
>> Hi Matevz,
>>
>> The other thing that should go into the cmsd is to avoid doing a
>> ?redirect on file not found? for reselection.
>>
>> This would help immensely in cases like FNAL which uses this for all
>> jobs, causing the multi source CMSSW to pull data that is onsite, from
>> offsite, due to reselection.
>>
>> (After telling them for 5 years to change, I guess we can tweak the
>> software ;) )
>>
>> Brian
>>
>> Sent from my iPhone
>>
>>> On May 3, 2019, at 6:57 PM, Matevz Tadel <[log in to unmask]> wrote:
>>>
>>> Hi,
>>>
>>> Andy realized that an option for this already exists -- triedrc=resel
>>>
>>> Andy impleented a change in cmsd that allows disabling opening of a new
>>> file on reselection, goes under cmsd.sched nomultisrc.
>>>
>>> Brian, now we have to propagate this into XrdAdapter.
>>>
>>> Matevz
>>>
>>>> On 4/18/19 10:45 AM, Andrew Hanushevsky wrote:
>>>> After some discusion with Matevz, we decided to simplify this, So, it
>>>> won't be exactly what was outlined but will be functionally the same.
>>>> This requires soem development in the cmsd. That said, an issue should
>>>> be cut.
>>>> Andy
>>>>> On Thu, 18 Apr 2019, Michal Kamil Simon wrote:
>>>>> Hi,
>>>>>
>>>>> It sounds reasonable to me :-)
>>>>>
>>>>> Matevz: could you create an issue in github so we don't loose
>>>>> track of this topic? ;-)
>>>>>
>>>>> Cheers,
>>>>> Michal
>>>>> ________________________________________
>>>>> From: [log in to unmask] [[log in to unmask]] on
>>>>> behalf of Bockelman, Brian [[log in to unmask]]
>>>>> Sent: 17 April 2019 03:59
>>>>> To: [log in to unmask]
>>>>> Cc: xrootd-dev
>>>>> Subject: Re: Proposal for new opaque URL parameter using=
>>>>> complementing tried=
>>>>>
>>>>> Yes! We definitely could benefit from this on the CMS side!
>>>>>
>>>>> Sent from my iPhone
>>>>>
>>>>>> On Apr 15, 2019, at 5:21 PM, Matevz Tadel <[log in to unmask]> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> [This is mostly for Andy, Brian, and Michal.]
>>>>>>
>>>>>> In the context of XCache cluster used by CMSSW multi-source jobs
>>>>>> there is an issue with cmssw jobs requesting opening of a second
>>>>>> source on the cache cluster using the tried= opaque parameter to
>>>>>> point to cache server already in use. This leads to creation of
>>>>>> another replica of the same file in the cache cluster.
>>>>>>
>>>>>> The cache still needs to honor tried= in case there is a problem with
>>>>>> the existing server. However, asking for a new "extra" server in the
>>>>>> context of cache does not make much sense.
>>>>>>
>>>>>> To distinguish these two conditions I propose to introduce a new
>>>>>> opaque directive, "using=", used to signal to the redirector that the
>>>>>> client is already using the listed servers.
>>>>>>
>>>>>> On cmsd side this would be accompanied with a cms.dfs multisource
>>>>>> count ("sister" option to cms.dfs retries). These two would then
>>>>>> control how many errors and parallel accesses are allowed for a
>>>>>> client session.
>>>>>>
>>>>>> Does this make sense?
>>>>>>
>>>>>> Matevz
>>>>>>
>>>>>> ########################################################################
>>>>>> Use REPLY-ALL to reply to list
>>>>>>
>>>>>> To unsubscribe from the XROOTD-DEV list, click the following link:
>>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>>>>>
>>>>> ########################################################################
>>>>> Use REPLY-ALL to reply to list
>>>>>
>>>>> To unsubscribe from the XROOTD-DEV list, click the following link:
>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>>>>>
>>>>> ########################################################################
>>>>> Use REPLY-ALL to reply to list
>>>>>
>>>>> To unsubscribe from the XROOTD-DEV list, click the following link:
>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>>>>>
>>>
>>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
