 |
 |
Hi, I (finally) found a bit of time to work on a port of Xrootd to (native) InfiniBand, and a proof-of-concept version is now sort of working, meaning that I can now transfer files with xrdcp. If possible, I would like to show first results with this version at the ACAT05 conference in Berlin in about 3 weeks from now. I have a few questions to you: 1/ what would be meaningful benchmark for xrootd ? For the moment, I'm only using point-to-point transfers (remote read to /dev/null, mainly) which is a bit boring ... 2/ Is there a ready-to-use test suite that can/should be used ? 3/ I still have one problem with xrdcp. For the client I have added a few lines of code in the XrdClientPhyConnection.cc which initializes my InfiniBand connection, and bypasses the recv/write calls. Everything works fine until the end. The Server log looks like this: 050501 21:36:41 5282 schwicke.17140:12@iwrcgop027 XrootdProtocol: 0000 req=3003 dlen=0 050501 21:36:41 5282 schwicke.17140:12@iwrcgop027 XrootdFile: closing r /tmp/testfile.dat 050501 21:36:41 5282 schwicke.17140:12@iwrcgop027 XrootdProtocol: 0000 close fh=0 050501 21:36:41 5282 schwicke.17140:12@iwrcgop027 XrootdResponse: 0000 sending OK 050501 21:36:44 5282 XrootdXeq: schwicke.17140:12@iwrcgop027 disc 0:04:24 (link read error) 050501 21:36:44 5282 schwicke.17140:12@iwrcgop027 XrdPoll: sending poller 0 detach for link 12 050501 21:36:44 5282 XrdPoll: Poller 0 detached fd 12 entry 1 now at 1 050501 21:36:44 5282 schwicke.17140:12@iwrcgop027 XrdPoll: FD 12 detached from poller 0; num=0 For TCP/IP I see that the last recv request succeeds in the poll command but ends with zero bytes of data, resulting in a ENOMSG return code. For InfiniBand, this call simply times out, giving the above link read error. A workaround would be to send a zero size message over the InfiniBand link just before the connection is closed for good. I tried to do that inside the XrdClientPhyConnection destructor, but that one is never called in xrdcp. A side effect of this is that the resources used by the InfiniBand connection need to be cleaned up and freed by the driver resource tracking mechanisms. Where should this be done instead? Thank's a lot in advance, Ulrich -- __________________________________________ Dr. Ulrich Schwickerath Forschungszentrum Karlsruhe GRID-Computing and e-Science Institut for Scientific Computing (IWR) P.O. Box 36 40 76021 Karlsruhe, Germany Tel: +49(7247)82-8607 Fax: +49(7247)82-4972 e-mail: [log in to unmask] PGP DH/DSS Key: ID 0xCEB9826F Fingerprint: 5537 8473 CD26 507E 8EE2 BAAF 98E2 FD16 CEB9 826F __________________________________________