 |
 |
Hi there, Thanks for responding! Unfortunately, the FileSystem.copy() already runs in its own thread, but locks the process because the Python 2.x interpreter won't interrupt it :-( For now, I'm OK with shelling out to xrdcp for the parallel copy. I hope in the future to support direct streaming of data to and from Castor because we're seeing high contention on the RAID array where the data lands from Castor ( 800 MB/s inbound tends to kill outbound performance!), and this will involve moving to File.write() ops which should work better because the write loops can be interleaved. Cheers, Roger Downing STFC Daresbury Laboratory, Keckwick Lane, Warrington WA4 4AD UK tel: +44 1925 603937 ________________________________ From: Justin Lewis Salmon [[log in to unmask]] Sent: 03 February 2014 22:07 To: xrootd/xrootd-python Cc: Downing, Roger (STFC,DL,SC) Subject: Re: [xrootd-python] Avoiding the GIL (#8) Hi Roger, I don't believe that the new XRootD client (upon which pyxrootd is based) currently supports asynchronous copy jobs (@ljanyst<https://github.com/ljanyst> please correct me if I'm wrong?) hence why pyxrootd doesn't support it either and will block with both FileSystem.copy() and CopyProcess. I know threading in Python is a bit of a nightmare, but (tentative suggestion) you could try FileSystem.copy() in a separate "thread"? Cheers, Justin — Reply to this email directly or view it on GitHub<https://github.com/xrootd/xrootd-python/issues/8#issuecomment-34006176>. -- Scanned by iCritical. --- Reply to this email directly or view it on GitHub: https://github.com/xrootd/xrootd-python/issues/8#issuecomment-34008825 ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1