If you plan to do that you really need to copordinate with RAL as they are
the current maintainers of that plugin. I don't think it would be wise to
change that without going through it with them as they have been making
active changes.
On Wed, 13 Oct 2021, Fabrizio Furano wrote:
> Hi,
>
> AFAIU (Brian) you are proposing a buffering layer that
> accumulates sequential writes before flushing to the fs layer.
>
> I think that the behaviour of such a buffer would be
> dependent on the needs of the underlying fs implementation, and
> in this case (correct me please if I am wrong) the underlying implementation
> (and the only one that would need it) is XrdCeph. So I would
> implement it in XrdCeph as a private write buffer...
>
>
>
> f
>
>
>
>
> On 10/12/21 11:09 PM, xrootd-dev wrote:
>> It's not clear what is meant by "filesystem layer". I do not like mucking
>> around a generic interface to simply satisfy one protocol that may not
>> staisfy another. So, I'd like to keep the complexity level to a minimum.
>> We have a hard enough time with http as it is.
>>
>> On Tue, 12 Oct 2021, Brian P Bockelman wrote:
>>
>>> Hi James -- I think this sounds like a great idea.
>>>
>>> For TPC, you found the correct location for increasing the buffer size. Note `TPCHandler::m_block_size` is the size of the
>> HTTP request generated when HTTP-TPC is doing multistreams and must be a multiple of the block size (otherwise you may hit
>> deadlocks). My recommendation is to make it no smaller than 16MB.
>>>
>>> I never did much exploration into optimal block sizes; without it, `libcurl` would generate extremely small writes (16KB or
>> smaller, depending on how often you poll the TCP socket) and the Ceph RADOS integration would outright fail with unaligned
>> writes. So, I'd welcome your work!
>>>
>>> The hard decision point is the buffering in XrdHttp. Right now, the size of the buffer for PUT appears to simply be whatever
>> comes off the socket. You _may_ put the buffer in the XrdHttp layer ... or it might make more sense at the filesystem layer. We
>> already "paid the price" in XrdTpc in terms of code complexity but it's not immediately clear the same decision makes sense for
>> XrdHttp.
>>>
>>> --
>>> You are receiving this because you are subscribed to this thread.
>>> Reply to this email directly or view it on GitHub:
>>> https://github.com/xrootd/xrootd/issues/1532#issuecomment-941288163
>>>
>>> ########################################################################
>>> Use REPLY-ALL to reply to list
>>>
>>> To unsubscribe from the XROOTD-DEV list, click the following link:
>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>>
>> ?
>> You are receiving this because you were mentioned.
>> Reply to this email directly, view it on GitHub <https://github.com/xrootd/xrootd/issues/1532#issuecomment-941560859>, or
>> unsubscribe <https://github.com/notifications/unsubscribe-auth/ABJBUTYPWHRLQUOEGXDQ7X3UGSPYBANCNFSM5F2J2QJA>.
>> Triage notifications on the go with GitHub Mobile for iOS
>> <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android
>> <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
>>
>>
>
>
>
> --
> You are receiving this because you were mentioned.
> Reply to this email directly or view it on GitHub:
> https://github.com/xrootd/xrootd/issues/1532#issuecomment-941997652
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
[
{
"@context": "http://schema.org",
"@type": "EmailMessage",
"potentialAction": {
"@type": "ViewAction",
"target": "https://github.com/xrootd/xrootd/issues/1532#issuecomment-942001717",
"url": "https://github.com/xrootd/xrootd/issues/1532#issuecomment-942001717",
"name": "View Issue"
},
"description": "View this Issue on GitHub",
"publisher": {
"@type": "Organization",
"name": "GitHub",
"url": "https://github.com"
}
}
]
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
