 |
 |
Oh, I thought you said these jobs do full file scan and not actual physics analysis (which could access data in a different manner than "get full event data for every event" kind of test). I'm not saying the actual physics analysis does not do the same thing ... in some cases it does, that's why I've been asking about the data-tier etc :)
I will re-run the monitoring data analysis on Monday, selecting only access that read all the file to see if things change drastically. I don't think it really will as vector reads follow the same pattern for both successful and failed jobs ... they just end sooner for failed jobs.
While fixing the xroot-ceph to be smarter about vector reads will help a lot (and needs to be done anyway because as things are now it just calls for trouble whenever vector reads hit ceph) as it will allow it to read a block once instead of 500-times for each vector read request, it will still have to read the same block about 64M/200k ~ 300 times to satisfy successive vector read requests that hit the same block.
You still need to figure out why xcache is failing so badly on some machines and figure out what you are prepared to do to alleviate the situation ... it could be:
I don't know enough about ceph to understand if there is a chance of buffering / caching in the ceph access points ... but this would likely saturate at some point as well and having outside / xcache-like buffers is the only scalable way.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1