Hello everyone,
from the logs you have supplied it's clear that XrdCl sees URLs
starting with host locators containing user names, ie:
root:[log in to unmask]:1094//store/mc/Summer12/SMS-T2tt_mStop-150to350_mLSP-0to250_8TeV-Py
XrdCl (and XrdClient as well) would treat the entire
"[log in to unmask]:1094" string as a unique host identifier wile
establishing connections. Getting later:
root:[log in to unmask]:1094//store/mc/SAM/GenericTTbar/GEN-SI
would open another physical connection to "[log in to unmask]:1094"
and so on.
Now, looking at XrdPosix, I cannot find anything that would do such
a thing. There are however some possibilities of getting this from the
outside:
1) via XROOTD_VMP envvar:
https://github.com/xrootd/xrootd/blob/xrdposixcl/src/XrdPosix/XrdPosixXrootdPath.cc#L58
2) by feeding the open call with a valid xrootd url:
https://github.com/xrootd/xrootd/blob/xrdposixcl/src/XrdPosix/XrdPosixXrootdPath.cc#L122
Alja, are you sure that it's not something that you are doing yourself?
Cheers,
Lukasz
On 27.08.2013 07:24, Alja Mrak-Tadel wrote:
> If it is still relevant ... Here is a link to xrootd log with XrdCl in
> debug log level:
> https://amraktad.web.cern.ch/amraktad/mail/xrdcl.log
>
> Alja
>
>
> On 8/26/13 7:37 PM, Matevz Tadel wrote:
>> Great, thanks Lukasz!
>>
>> Matevz
>>
>> On 8/26/13 2:55 PM, Lukasz Janyst wrote:
>>> I am actually also away. It looks like something in XrdPosix is
>>> forcing new
>>> connections. I will have a closer look once I am back on Wednesday.
>>>
>>> Lukasz
>>>
>>> On 8/26/13 11:46 PM, Lukasz Janyst wrote:
>>>> I call it a trick because the only context I have seen it used in is
>>>> forcing
>>>> new physical connections instead of reusing existing ones.
>>>>
>>>> Lukasz
>>>>
>>>> On 8/26/13 11:45 PM, Lukasz Janyst wrote:
>>>>> Hi Matevz,
>>>>>
>>>>> Andy's on vacation until 2nd Sep.
>>>>>
>>>>> Lukasz
>>>>>
>>>>> On 8/26/13 11:28 PM, Matevz Tadel wrote:
>>>>>> Hi Lukasz, Everybody,
>>>>>>
>>>>>> On 8/24/13 5:55 AM, Lukasz Janyst wrote:
>>>>>>> Yes, in fact the username "trick" to trigger the new connections
>>>>>>> would still
>>>>>>> work with the new client and that's likely what is happening here.
>>>>>>
>>>>>> Hmmh ... why do you call it a trick? And where are the usernames
>>>>>> coming
>>>>>> from? This is all running within a single xrootd-proxy process ...
>>>>>> so it is
>>>>>> always the same user, I don't see how this could change in any
>>>>>> way. (On
>>>>>> server side and in monitoring, this is obviously a different user,
>>>>>> as port
>>>>>> number is also combined into it.)
>>>>>>
>>>>>> Anyway ... this seems a really bad idea for the proxy :)
>>>>>>
>>>>>> Here is a dump of server/monitoring-side usernames for a bunch of
>>>>>> consecutive file-close events. You can see some of them get reused
>>>>>> ... but
>>>>>> 105 of such connections stayed for 3 days+ (after reading ~300
>>>>>> files with
>>>>>> 300 XrdCls in a single process).
>>>>>>
>>>>>> * 413974 * alja.2636:[log in to unmask] *
>>>>>> * 415017 * alja.2636:[log in to unmask] *
>>>>>> * 416011 * alja.2636:[log in to unmask] *
>>>>>> * 416476 * alja.2636:[log in to unmask] *
>>>>>> * 417680 * alja.2636:[log in to unmask] *
>>>>>> * 419305 * alja.2636:[log in to unmask] *
>>>>>> * 419376 * alja.2636:[log in to unmask] *
>>>>>> * 419520 * alja.2636:[log in to unmask] *
>>>>>> * 419599 * alja.2636:[log in to unmask] *
>>>>>> * 419942 * alja.2636:[log in to unmask] *
>>>>>> * 420628 * alja.2636:[log in to unmask] *
>>>>>> * 421065 * alja.2636:[log in to unmask] *
>>>>>> * 422202 * alja.2636:[log in to unmask] *
>>>>>> * 422688 * alja.2636:[log in to unmask] *
>>>>>> * 423059 * alja.2636:[log in to unmask] *
>>>>>> * 423080 * alja.2636:[log in to unmask] *
>>>>>> * 424121 * alja.2636:[log in to unmask] *
>>>>>> * 424264 * alja.2636:[log in to unmask] *
>>>>>> * 431576 * alja.2636:[log in to unmask] *
>>>>>> * 488603 * alja.2636:[log in to unmask] *
>>>>>> * 489027 * alja.2636:[log in to unmask] *
>>>>>> * 489340 * alja.2636:[log in to unmask] *
>>>>>> * 494316 * alja.2636:[log in to unmask] *
>>>>>> * 494699 * alja.2636:[log in to unmask] *
>>>>>> * 496100 * alja.2636:[log in to unmask] *
>>>>>>
>>>>>>> In fact you can see what is exactly happening if you set the
>>>>>>> XRD_LOGLEVEL and
>>>>>>> XRD_LOGFILE envvars as described in the xrdcopy man page.
>>>>>>
>>>>>> Alja will run our test with this on ... but whatever the outcome,
>>>>>> these
>>>>>> connections should still be closed at some point.
>>>>>>
>>>>>> Andy, can you please comment on this? [Does anybody know, is Andy
>>>>>> away ...
>>>>>> or I should kick his chair in a separate email?]
>>>>>>
>>>>>> Cheers,
>>>>>> Matevz
>>>>>>
>>>>>>> Cheers,
>>>>>>> Lukasz
>>>>>>>
>>>>>>> On 23.08.2013 21:58, Fabrizio Furano wrote:
>>>>>>>> Hi guys,
>>>>>>>>
>>>>>>>> you are reminding to me a historical thing with this thread.
>>>>>>>> You may
>>>>>>>> want to doublecheck these two points, and see if the defaults or
>>>>>>>> the
>>>>>>>> behavior is appropriate to the behavior of your proxy:
>>>>>>>>
>>>>>>>> - by choice the connections to the redirectors had a very long
>>>>>>>> TTL,
>>>>>>>> one day if I remember correctly
>>>>>>>> - at some point, due to some interaction with the sec (don't
>>>>>>>> remember
>>>>>>>> exactly what by now) the behavior was changed to having one
>>>>>>>> physical
>>>>>>>> connection per process per user. Maybe Gerri remembers the
>>>>>>>> rationale of
>>>>>>>> this better than me. My point is to raise that if one sees many
>>>>>>>> phyconns
>>>>>>>> from the same process, they could be linked to different user
>>>>>>>> ids. Worth
>>>>>>>> checking IMHO.
>>>>>>>>
>>>>>>>> Hope that helps.
>>>>>>>>
>>>>>>>> Cheers
>>>>>>>> Fabrizio
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 8/23/13 9:42 PM, Brian Bockelman wrote:
>>>>>>>>> On Aug 23, 2013, at 2:05 PM, Alja Mrak Tadel
>>>>>>>>> <[log in to unmask]> wrote:
>>>>>>>>>
>>>>>>>>>>>> normally XrdCl would open one connection per server it
>>>>>>>>>>>> contacts.
>>>>>>>>>>>> Not per file.
>>>>>>>>>>>
>>>>>>>>>>> OK, how do we get 105 connections from a single process to the
>>>>>>>>>>> meta-manager then? :)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> FYI:: The connections listed with netstat are to the origin
>>>>>>>>>> xrootd.unl.edu.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> So - we see duplicate connections to the redirector, but no
>>>>>>>>> duplicate
>>>>>>>>> connections to the data server?
>>>>>>>>>
>>>>>>>>> Could be a client bug, of course!
>>>>>>>>>
>>>>>>>>> Brian
>>>>>>>>> ########################################################################
>>>>>>>>>
>>>>>>>>> Use REPLY-ALL to reply to list
>>>>>>>>>
>>>>>>>>> To unsubscribe from the XROOTD-DEV list, click the following link:
>>>>>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
