 |
 |
Hi Rob, OK, here is a cut and paste from a 2009 presentation... The xrootd already supports parallel TCP paths . Significant improvement in WAN transfer rate . Specified as xrdcp -S num New Xtreme copy mode option . Uses multiple data sources torrent-style . Specified as xrdcp -x . Transfers to CERN; examples: . 1 source (.de): 12MB/sec ( 1 stream) . 1 source (.us): 19MB/sec ( 15 streams) . 4 sources (3 x .de + .ru): 27MB/sec ( 1 stream each) . 4 sources + || streams: 42MB/Sec (15 streams each) . 5 sources (3 x .de + .it + .ro): 54MB/Sec (15 streams each) Now, originally you used -x but I see there is a bug in handling this legacy option and it doesn't turn on multi-source copying. So, use the new flag (we made old xrdcp understand the new options) which is --sources <n> which specified the maximum number of additional sources you want (limit is 32). Consider using more than one stream with -S) and the data source is typically a redirector. You can always get interactive help by xrdcp-old --help Andy On Fri, 21 Nov 2014, Rob Gardner wrote: > Andy, > > Yes, this is something I think OSG will be interested in for distributed, public caches for small (mainly non-HEP) users. What I worry about with AAA and FAX is that we start to use the network more heavily at our larger sites, and create choke points for the odd user. Having a second (and third..) source will help not only the overall speed, it will ensure a threshold of adequate speed. > > Sounds like we could get started using the xrdcp-old. Do we have any documentation on how to use it (google only provides me with error reports from 2011). > > Rob > > >> On Nov 21, 2014, at 4:44 PM, Andrew Hanushevsky <[log in to unmask]> wrote: >> >> Hi Rob, >> >> It does but now with the new client. You need to use the old client to get that feature. The previous copy command is now called xrdcp-old. As for performance, based on old tests you can see about a 5x increase in transfer speed if you have 4 decent sources. Is this something we should move up in the priority scale to get implemented in the new client? >> >> Andy >> >> On Fri, 21 Nov 2014, Rob Gardner wrote: >> >>> Hello >>> >>> Apologies if this has already been discussed, but does the extreme copy mode (torrent-like, multiple source sites) work in the current XrootD release? If so, any user documentation and/or test results can anyone share? >>> >>> Best, >>> >>> Rob >>> >>> ######################################################################## >>> Use REPLY-ALL to reply to list >>> >>> To unsubscribe from the XROOTD-L list, click the following link: >>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 >>> >> >> ######################################################################## >> Use REPLY-ALL to reply to list >> >> To unsubscribe from the XROOTD-L list, click the following link: >> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 > > ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1