 |
 |
FYI. Lukasz ________________________________________ From: Sebastien Ponce Sent: 14 March 2015 22:40 To: [log in to unmask] Cc: Tom Byrne; Shaun de Witt; [log in to unmask]; Lukasz Janyst; castor-dev (Discuss any development issue about the CASTOR software); castor-operations (CastorHSM operations mailing list) Subject: XrootD plugin : some news As I've done some more progress last week, I wanted to give you some news on the status of my testing of the xrootd plugin. First I've solved another couple of bugs (in particular the problem of reading back 2GB files) and everything seems now functional, at least when used for CASTOR pools. Then I've spent quite some time in performance testing (you reported problems around that). And the good news is that I've solved the performance issues too. My setup is that I have 4 clients writing data to CASTOR, which has a pool of 5 "diskservers" (with no disks) writing to a pool of 8 nodes in ceph. All nodes have 10Gb network. Results are the following : - I can write a single file from a single client to CASTOR/Ceph at 500MB/s (yes, megabytes) - I can saturate the ceph cluster at the same level as ceph itself does (via there bench test), that is 800MB/s input or 500MB/s input + 500MB/s output concurrently. - this last state (500MB/s both direction) could be sustained for >12h Now that was the good news.... the bad one is that I had to change some parameters of the system to achieve that, including some that cannot be changed (I recompiled). This has to do with buffer sizes : - I used -DICPParallelChunks 10 -DICPChunkSize 33554432 in xrdcp - I changed the xrd.cf file on the xrootd server with this extra line : "xrootd.async segsize 33554432" (take care, it's segsize and not segsz as the doc states. I've reported that issue) - and I had to recompile xroot to change the maximum authorized limit for that segsize parameter (was 4M) - I've used the hammer prerelease of ceph (0.93) On the recompilation issue, I'm in link with Andy Hanushevsky to get an extra config entry in the next release. Concerning the hammer release of ceph, I had to recompile the libradosstriper by hand to fix a compilation issue that has been introduced when LTTng was introduced in librados. So the default release won't work... and the giant one is broken for files > 2GB... and on top there won't be any giant release anymore in principle. Anyway, this will all settle down and there remain only 2 points on my list for this week : - listing of files - EC pools For EC pools, Dan is providing one to me, so I'll be able to test. Cheers, Sebastien ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1