On 03.09.2013 19:33, Brian Bockelman wrote:
> On Sep 3, 2013, at 2:40 AM, Lukasz Janyst <[log in to unmask]> wrote:
>> On 03.09.2013 01:06, Andrew Hanushevsky wrote:
>>> 1) All bytes are transferred through a single pipe. That means if I
>>> have two clients and one is reading 1K blocks and the other is reading
>>> 1M blocks the 1K block client suffers the same latency as if it had been
>>> reading 1M blocks. Same is true for writing. One xrdcp can destroy
>>> performance for everybody.
>>
>> You have kXR_oksofar responses that would split large reads into small chunks making it possible to interleave other responses. You use this quite extensively.
>
> This still suffers from head-of-line blocking inherent in a TCP connection. For example, when you call sendfile() on a 1KB read which hits a slow disk, you may be blocking the 1KB read from the page cache from the next client.
It's inherent to slow/busy disks and sendfile interface then, not
TCP connection. On top, it only happens when you use sendfile instead of
caching blocks in memory, which really depends on the use-case. But OK,
at least it's a convincing argument.
>>> 2) Should a TCP packet get dropped and we go into packet recovery
>>> then all clients see the increased latency. Since we have everything
>>> going through a single pipe the odds of that happening increase.
>>
>> True, but single packet loses rarely cause non-negligible performance issues. Multiple packet loses would affect all connections equally.
>
> Yes and no. TCP is remarkably sensitive to packet loss + latency. Multiple TCP streams are likely going to get you an order-magnitude more performance in this case.
>
> As a practical matter, on a WAN (even on R&D networks), your best bet is to have multiple TCP streams. It irritates me to no end - you'd think there would be a better transport protocol after 40 years of internet - but it's a fact of life.
>
> I think it would be a quite interesting *experiment* to compare these things, however!
Yeah, it's latency vs. max window size and TCP using packet loses to
probe bandwidth. But still, provided that you have sufficient window
size to compensate for latency, multiple may be streams faster, but I
don't believe they will be order of magnitude faster.
>> So does a NIC and Ethernet.
>
> However, NIC / ethernet do not suffer from head-of-line blocking; if the HTTP connection to Google stalls, your SSH session to lxplus is still usable.
You would get head-of-line blocking only while using sendfile or if
xrootd server queues IO requests to disks from one channel in a single
thread.
Cheers,
Lukasz
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
|