Hi Andy,
could you please elaborate on the serialization issue? We have
virtual streams and the new client fully supports them not only for data
requests but for _all_ kinds of requests. So, I don't really see this
serialization you're talking about. In fact, I would expect multiple
streams to have worse performance due to constant re-authentication.
Cheers,
Lukasz
On 01.09.2013 05:28, Andrew Hanushevsky wrote:
> OK, this is what started the whole problem and a dive into the wrong
> direction of serializing everything in the proxy server. Indeed, a long
> time ago Lukasz and I discussed whether the client should close idle
> connections and I said why bother, it would simplify the code by just
> allowing the server to do it. While that was the right response for
> standard client jobs it was wrong for long-lived servers that act like
> clients (i.e. proxy server). First, the server-side timeout is set off
> by default. You need to enable it and few people do as it rarely causes
> a problem (sites with badly behaving VM hypervisors normally turn the
> timeout on). Second, I didn't consider the side-effect on the proxy
> server. So, what to do? I suspect the only real solution to this issue
> is to implement an idle timeout in the new client. It's likely that we
> would need two timeouts (just like in the old client) -- one for
> redirectors (a longish timeout) and one for servers (a shortisj
> timeout). The actual values would be controlled by some "envar".
>
> Lukasz, what's the possibility of adding this?
>
> Andy
>
> On Thu, 22 Aug 2013, Matevz Tadel wrote:
>
>> Hi Lukasz, Andy,
>>
>> On 8/22/13 7:52 AM, Lukasz Janyst wrote:
>>> Hi Alja,
>>>
>>> that's a feature. The connection manager used to have idle time
>>> after which
>>> the connections were closed but Andy's preference was to manage
>>> connection life
>>> from the server-side.
>>
>> Hmmh ... then it seems the servers are not doing a too good job :( If
>> you look at the netstat dump, there are 105 connections to CMS
>> meta-manager at xrootd.unl.edu alone and 150 connections to various
>> data servers ... and this is after replaying a one-day-worth of access
>> from UCSD (320 files) and waiting for /three days/. So, the servers
>> don't close all of them.
>>
>> The problem here is that proxy process stays up for a long time and it
>> instantiates a new XrdCl for every file it is downloading.
>>
>> Andy, another problem here is with detailed monitoring ... I never get
>> the last message of the 't' stream, the one containing the close so
>> monitor thinks the files are still kept opened. As there is no
>> disconnect and there are is no new activity on the channel, the
>> buffers keeps sitting on the server. If this stays like this ... can I
>> ask for 'close' event to cause flushing of the 't' stream when io[v]
>> is in effect?
>>
>> Cheers,
>> Matevz
>>
>>
>>> Cheers,
>>> Lukasz
>>>
>>> On 22.08.2013 16:15, Alja Mrak-Tadel wrote:
>>>> Hello,
>>>>
>>>> I'm testing a proxy server with new client (branch xrdposixcl) and I
>>>> noticed that socket connections stay open for days. Here is an example
>>>> of netstat command on the machine I run xrootd proxy:
>>>> http://uaf-2.t2.ucsd.edu/~alja/netstat.txt
>>>> Notice also there are several connections to the meta manager,
>>>> xrootd.unl.edu.
>>>>
>>>> From xrootd log http://uaf-2.t2.ucsd.edu/~alja/proxy.log I can see
>>>> XrdCl::File::Close() is called but it is difficult to trace why sockets
>>>> are left open.
>>>>
>>>> I'm working on a caching proxy server, where it is important that
>>>> connections get closed reasonably soon ... the proxy is potentially
>>>> accessing all Xrootd servers in a federation.
>>>>
>>>> Thanks,
>>>> Alja
>>>>
>>>> ########################################################################
>>>>
>>>> Use REPLY-ALL to reply to list
>>>>
>>>> To unsubscribe from the XROOTD-DEV list, click the following link:
>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>>>
>>> ########################################################################
>>> Use REPLY-ALL to reply to list
>>>
>>> To unsubscribe from the XROOTD-DEV list, click the following link:
>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>>
>> ########################################################################
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the XROOTD-DEV list, click the following link:
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
|