LISTSERV 16.5 - XROOTD-DEV Archives

Thanks for clarifying.

Al

________________________________________________
Albert L. Rossi
Senior Software Developer
Scientific Computing Division, Scientific Data Services, Distributed Data Development
FCC 229A
Mail Station 369 (FCC 2W)
Fermi National Accelerator Laboratory
Batavia, IL 60510
(630) 840-3023

________________________________
From: xrootd-dev ***@***.***>
Sent: Tuesday, August 9, 2022 1:41 PM
To: xrootd/xrootd ***@***.***>
Cc: Albert Rossi ***@***.***>; Author ***@***.***>
Subject: Re: [xrootd/xrootd] Default Chunk Size (Issue #1756)

Correct, clearly the documentation is incorrect. That will be corrected.
As for why 8 Mib? Well, there is no particular reason other than making it
big enough but not too big to retransmit too much data when you have to
recover a write or read. The original client used 4 Mib but that seemed
too small so it was doubled as a stating point. Mind you that the server
is free to manage the read or write with whatever buffer size it wants to
use so just because the chunk size is 8 Mib doesn't mean the server needs
to use a matching buffer. Indeed, native xrootd uses a 2 Mib buffer.

Comparing other transfer agents we can find that they arguably usee an
outlandishly large transfer chunk size (i.e. 32K Mib or even 64K Mib) --
just look at Amazon or Google.

Andy

On Tue, 9 Aug 2022, Albert Rossi wrote:

> I imagine you are aware that the documentation reports
>
> ```
> 4.2.6 XRD CPCHUNKSIZE
> Size of a single data chunk handled by xrdcp / XrdCl::CopyProcess.
> Default: 16KiB
> ```
>
> but the code (XrdClConstants.hh, line ), has
>
> ```
> const int DefaultCPChunkSize = 8388608;
> ```
>
> and this is what I see being used when I run xrdcp with `d -3`.
>
> I do have a question, though. Why 8 MiB? Can you provide some clarification for this choice?
>
> Thanks, Al
>
> --
> Reply to this email directly or view it on GitHub:
> https://github.com/xrootd/xrootd/issues/1756<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_xrootd_xrootd_issues_1756&d=DwQCaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=60rQ0HHqHmEY1P6VSdyuTQ&m=NgQGoNJHyJOrjMTLEP6CNuviNO3tFvHFeSWVniDi8RW9KP0P_YGuv_vUY3gNuVar&s=X_NHeO-CVdiTt-pl0zOIXLDumBqe_g6l0qiVV76kxWY&e=>
> You are receiving this because you are subscribed to this thread.
>
> Message ID: ***@***.***>
>
> ########################################################################
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the XROOTD-DEV list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1<https://urldefense.proofpoint.com/v2/url?u=https-3A__listserv.slac.stanford.edu_cgi-2Dbin_wa-3FSUBED1-3DXROOTD-2DDEV-26A-3D1&d=DwQCaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=60rQ0HHqHmEY1P6VSdyuTQ&m=NgQGoNJHyJOrjMTLEP6CNuviNO3tFvHFeSWVniDi8RW9KP0P_YGuv_vUY3gNuVar&s=9j2_QyhWTg1QChxLw6M7BVYdIQdFnMENv2h8TJBPolQ&e=>

—
Reply to this email directly, view it on GitHub<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_xrootd_xrootd_issues_1756-23issuecomment-2D1209747013&d=DwMCaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=60rQ0HHqHmEY1P6VSdyuTQ&m=NgQGoNJHyJOrjMTLEP6CNuviNO3tFvHFeSWVniDi8RW9KP0P_YGuv_vUY3gNuVar&s=P2VQkXne4kwtrokTIVgtJJZhYB21fsM841FWrmjg95w&e=>, or unsubscribe<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AA6NBHHXZISVUXXCNL65FBDVYKQ67ANCNFSM56BREGNA&d=DwMCaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=60rQ0HHqHmEY1P6VSdyuTQ&m=NgQGoNJHyJOrjMTLEP6CNuviNO3tFvHFeSWVniDi8RW9KP0P_YGuv_vUY3gNuVar&s=Q7J0IVUEDuLjsY361svNODMMTKRR7fRqjUs7JkIYOG4&e=>.
You are receiving this because you authored the thread.Message ID: ***@***.***>

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.

[ { "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": "https://github.com/xrootd/xrootd/issues/1756#issuecomment-1274217735", "url": "https://github.com/xrootd/xrootd/issues/1756#issuecomment-1274217735", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

Use REPLY-ALL to reply to list

To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1