Presumably, the log snippet Brian pointer to is the cache client issuing
the read to Ceph at xrootd.echo.stfc.ac.uk:1094, right? So, it would
appear the client is requesting 627 reads each averaging about 14K and
each at a different offset. If xrootd.echo.stfc.ac.uk is not a caching
server then that would represent a request for roughly 39GB (yes
gigabytes) of data of which only 9MB would be extracted and sent back.
Frankly, I don't see how Ceph could possibly complete the request in 300
seconds unless there was almost no load on it. So, is
xrootd.echo.stfc.ac.uk a caching gateway?
Note that Ceph does not support readv() so it has to unroll the vector
read and issue each read individually. I suppose it could have been
more clever and coalesced the reads that happen to be in the same 64MB
page but that was not how it was written.
Andy
On Fri, 7 Aug 2020, Matev? Tadel wrote:
> Yes, that would be my guess, that proxy doesn't get the response from the remote / ceph.
>
> If cache has the memory, it will unravel the vread request and request all the cache blocks it needs and issue each read request independently. Once it's out of memory, it will simply forward the vread as it got it.
>
> I could send multiple block-reads as vread ... but Andy told me that from some block size (O(16 kB)?)
> onwards that doesn't matter much.
>
> I am, however, a little mortified over the 64 MB block size people were recommending here. If vector reads are used I usually tell people to rather go down to 32 kB :)
>
> During the last Lyon workshop Andreas Peters said that the big problem with the ceph ofs interface is that it internally serializes the vread requests and handles them sequentially. I don't know how this applies here if cache already unravels the vreads.
>
> Is lhcb usage similar to what we do in cmssw, i.e., vread baskets for all activated branches?
>
> --
> You are receiving this because you were mentioned.
> Reply to this email directly or view it on GitHub:
> https://github.com/xrootd/xrootd/issues/1259#issuecomment-670672447
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
[
{
"@context": "http://schema.org",
"@type": "EmailMessage",
"potentialAction": {
"@type": "ViewAction",
"target": "https://github.com/xrootd/xrootd/issues/1259#issuecomment-670686784",
"url": "https://github.com/xrootd/xrootd/issues/1259#issuecomment-670686784",
"name": "View Issue"
},
"description": "View this Issue on GitHub",
"publisher": {
"@type": "Organization",
"name": "GitHub",
"url": "https://github.com"
}
}
]
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1