Thanks for clarifying. Al ________________________________________________ Albert L. Rossi Senior Software Developer Scientific Computing Division, Scientific Data Services, Distributed Data Development FCC 229A Mail Station 369 (FCC 2W) Fermi National Accelerator Laboratory Batavia, IL 60510 (630) 840-3023 ________________________________ From: xrootd-dev ***@***.***> Sent: Tuesday, August 9, 2022 1:41 PM To: xrootd/xrootd ***@***.***> Cc: Albert Rossi ***@***.***>; Author ***@***.***> Subject: Re: [xrootd/xrootd] Default Chunk Size (Issue #1756) Correct, clearly the documentation is incorrect. That will be corrected. As for why 8 Mib? Well, there is no particular reason other than making it big enough but not too big to retransmit too much data when you have to recover a write or read. The original client used 4 Mib but that seemed too small so it was doubled as a stating point. Mind you that the server is free to manage the read or write with whatever buffer size it wants to use so just because the chunk size is 8 Mib doesn't mean the server needs to use a matching buffer. Indeed, native xrootd uses a 2 Mib buffer. Comparing other transfer agents we can find that they arguably usee an outlandishly large transfer chunk size (i.e. 32K Mib or even 64K Mib) -- just look at Amazon or Google. Andy On Tue, 9 Aug 2022, Albert Rossi wrote: > I imagine you are aware that the documentation reports > > ``` > 4.2.6 XRD CPCHUNKSIZE > Size of a single data chunk handled by xrdcp / XrdCl::CopyProcess. > Default: 16KiB > ``` > > but the code (XrdClConstants.hh, line ), has > > ``` > const int DefaultCPChunkSize = 8388608; > ``` > > and this is what I see being used when I run xrdcp with `d -3`. > > I do have a question, though. Why 8 MiB? Can you provide some clarification for this choice? > > Thanks, Al > > -- > Reply to this email directly or view it on GitHub: > https://github.com/xrootd/xrootd/issues/1756<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_xrootd_xrootd_issues_1756&d=DwQCaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=60rQ0HHqHmEY1P6VSdyuTQ&m=NgQGoNJHyJOrjMTLEP6CNuviNO3tFvHFeSWVniDi8RW9KP0P_YGuv_vUY3gNuVar&s=X_NHeO-CVdiTt-pl0zOIXLDumBqe_g6l0qiVV76kxWY&e=> > You are receiving this because you are subscribed to this thread. > > Message ID: ***@***.***> > > ######################################################################## > Use REPLY-ALL to reply to list > > To unsubscribe from the XROOTD-DEV list, click the following link: > https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1<https://urldefense.proofpoint.com/v2/url?u=https-3A__listserv.slac.stanford.edu_cgi-2Dbin_wa-3FSUBED1-3DXROOTD-2DDEV-26A-3D1&d=DwQCaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=60rQ0HHqHmEY1P6VSdyuTQ&m=NgQGoNJHyJOrjMTLEP6CNuviNO3tFvHFeSWVniDi8RW9KP0P_YGuv_vUY3gNuVar&s=9j2_QyhWTg1QChxLw6M7BVYdIQdFnMENv2h8TJBPolQ&e=> — Reply to this email directly, view it on GitHub<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_xrootd_xrootd_issues_1756-23issuecomment-2D1209747013&d=DwMCaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=60rQ0HHqHmEY1P6VSdyuTQ&m=NgQGoNJHyJOrjMTLEP6CNuviNO3tFvHFeSWVniDi8RW9KP0P_YGuv_vUY3gNuVar&s=P2VQkXne4kwtrokTIVgtJJZhYB21fsM841FWrmjg95w&e=>, or unsubscribe<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AA6NBHHXZISVUXXCNL65FBDVYKQ67ANCNFSM56BREGNA&d=DwMCaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=60rQ0HHqHmEY1P6VSdyuTQ&m=NgQGoNJHyJOrjMTLEP6CNuviNO3tFvHFeSWVniDi8RW9KP0P_YGuv_vUY3gNuVar&s=Q7J0IVUEDuLjsY361svNODMMTKRR7fRqjUs7JkIYOG4&e=>. You are receiving this because you authored the thread.Message ID: ***@***.***> -- Reply to this email directly or view it on GitHub: https://github.com/xrootd/xrootd/issues/1756#issuecomment-1274217735 You are receiving this because you commented. Message ID: <[log in to unmask]> ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1