 |
 |
Hi Ulirich, The xrdcp test is fine simnce it will show the raw performance and scaling potential and relevant to infiniband performance. However, it would be interesting to see a random read test using small blocks to see how Infiniband latency compares to other transports. As far as the end of session handling and zero-length messages; I would need to look at the Inifinband path to see how one can avoid that without changing the client in this case. Of course, if you' already changing the client, then you could potentially send a zero-length message at the very end (just for Infiniband). Any chance of sending a patch to me? Andy ----- Original Message ----- From: "Ulrich Schwickerath" <[log in to unmask]> To: <[log in to unmask]> Sent: Monday, May 02, 2005 7:23 AM Subject: xrootd on (native) InfiniBand prototype > Hi, > > I (finally) found a bit of time to work on a > port of Xrootd to (native) InfiniBand, and a > proof-of-concept version is now sort of working, meaning > that I can now transfer files with xrdcp. If possible, I would > like to show first results with this version at the ACAT05 > conference in Berlin in about 3 weeks from now. > I have a few questions to you: > > 1/ what would be meaningful benchmark for xrootd ? > For the moment, I'm only using point-to-point transfers > (remote read to /dev/null, mainly) which is a bit boring ... > > 2/ Is there a ready-to-use test suite that can/should be used ? > > 3/ I still have one problem with xrdcp. For the client I have added > a few lines of code in the XrdClientPhyConnection.cc which > initializes my InfiniBand connection, and bypasses the recv/write > calls. Everything works fine until the end. The Server log looks > like this: > 050501 21:36:41 5282 schwicke.17140:12@iwrcgop027 XrootdProtocol: 0000 > req=3003 dlen=0 > 050501 21:36:41 5282 schwicke.17140:12@iwrcgop027 XrootdFile: closing > r /tmp/testfile.dat > 050501 21:36:41 5282 schwicke.17140:12@iwrcgop027 XrootdProtocol: 0000 > close > fh=0 > 050501 21:36:41 5282 schwicke.17140:12@iwrcgop027 XrootdResponse: 0000 > sending > OK > 050501 21:36:44 5282 XrootdXeq: schwicke.17140:12@iwrcgop027 disc 0:04:24 > (link read error) > 050501 21:36:44 5282 schwicke.17140:12@iwrcgop027 XrdPoll: sending poller > 0 > detach for link 12 > 050501 21:36:44 5282 XrdPoll: Poller 0 detached fd 12 entry 1 now at 1 > 050501 21:36:44 5282 schwicke.17140:12@iwrcgop027 XrdPoll: FD 12 detached > from > poller 0; num=0 > For TCP/IP I see that the last recv request succeeds in the poll > command but ends with zero bytes of data, resulting in a ENOMSG return > code. For InfiniBand, this call simply times out, giving the above > link read error. A workaround would be to send a zero size message over > the > InfiniBand link just before the connection is closed for good. I tried > to do that inside the XrdClientPhyConnection destructor, but that one > is never called in xrdcp. A side effect of this is that the resources > used by the InfiniBand connection need to be cleaned up and freed by the > driver resource tracking mechanisms. Where should this be done instead? > > > Thank's a lot in advance, > Ulrich > > -- > __________________________________________ > Dr. Ulrich Schwickerath > Forschungszentrum Karlsruhe > GRID-Computing and e-Science > Institut for Scientific Computing (IWR) > P.O. Box 36 40 > 76021 Karlsruhe, Germany > > Tel: +49(7247)82-8607 > Fax: +49(7247)82-4972 > > e-mail: [log in to unmask] > PGP DH/DSS Key: ID 0xCEB9826F > Fingerprint: 5537 8473 CD26 507E 8EE2 BAAF 98E2 FD16 CEB9 826F > __________________________________________ >