 |
 |
Hi, Andy, I need to clean up the code a bit before I can release it, but then I do not see a problem to give you access to it if you like. Keep in mind that it is just a proof-of-concept solution, tough, pre-alpha (I started two or three weeks ago ...) :-) I'll let you know where you can find it off-list, or send you a patch. Whatever you prefer. Ulrich > The xrdcp test is fine simnce it will show the raw performance and scaling > potential and relevant to infiniband performance. However, it would be > interesting to see a random read test using small blocks to see how > Infiniband latency compares to other transports. > As far as the end of session handling and zero-length messages; I would > need to look at the Inifinband path to see how one can avoid that without > changing the client in this case. Of course, if you' already changing the > client, then you could potentially send a zero-length message at the very > end (just for Infiniband). Any chance of sending a patch to me? > > Andy > > > ----- Original Message ----- > From: "Ulrich Schwickerath" <[log in to unmask]> > To: <[log in to unmask]> > Sent: Monday, May 02, 2005 7:23 AM > Subject: xrootd on (native) InfiniBand prototype > > > Hi, > > > > I (finally) found a bit of time to work on a > > port of Xrootd to (native) InfiniBand, and a > > proof-of-concept version is now sort of working, meaning > > that I can now transfer files with xrdcp. If possible, I would > > like to show first results with this version at the ACAT05 > > conference in Berlin in about 3 weeks from now. > > I have a few questions to you: > > > > 1/ what would be meaningful benchmark for xrootd ? > > For the moment, I'm only using point-to-point transfers > > (remote read to /dev/null, mainly) which is a bit boring ... > > > > 2/ Is there a ready-to-use test suite that can/should be used ? > > > > 3/ I still have one problem with xrdcp. For the client I have added > > a few lines of code in the XrdClientPhyConnection.cc which > > initializes my InfiniBand connection, and bypasses the recv/write > > calls. Everything works fine until the end. The Server log looks > > like this: > > 050501 21:36:41 5282 schwicke.17140:12@iwrcgop027 XrootdProtocol: 0000 > > req=3003 dlen=0 > > 050501 21:36:41 5282 schwicke.17140:12@iwrcgop027 XrootdFile: closing > > r /tmp/testfile.dat > > 050501 21:36:41 5282 schwicke.17140:12@iwrcgop027 XrootdProtocol: 0000 > > close > > fh=0 > > 050501 21:36:41 5282 schwicke.17140:12@iwrcgop027 XrootdResponse: 0000 > > sending > > OK > > 050501 21:36:44 5282 XrootdXeq: schwicke.17140:12@iwrcgop027 disc 0:04:24 > > (link read error) > > 050501 21:36:44 5282 schwicke.17140:12@iwrcgop027 XrdPoll: sending poller > > 0 > > detach for link 12 > > 050501 21:36:44 5282 XrdPoll: Poller 0 detached fd 12 entry 1 now at 1 > > 050501 21:36:44 5282 schwicke.17140:12@iwrcgop027 XrdPoll: FD 12 detached > > from > > poller 0; num=0 > > For TCP/IP I see that the last recv request succeeds in the poll > > command but ends with zero bytes of data, resulting in a ENOMSG return > > code. For InfiniBand, this call simply times out, giving the above > > link read error. A workaround would be to send a zero size message over > > the > > InfiniBand link just before the connection is closed for good. I tried > > to do that inside the XrdClientPhyConnection destructor, but that one > > is never called in xrdcp. A side effect of this is that the resources > > used by the InfiniBand connection need to be cleaned up and freed by the > > driver resource tracking mechanisms. Where should this be done instead? > > > > > > Thank's a lot in advance, > > Ulrich > > > > -- > > __________________________________________ > > Dr. Ulrich Schwickerath > > Forschungszentrum Karlsruhe > > GRID-Computing and e-Science > > Institut for Scientific Computing (IWR) > > P.O. Box 36 40 > > 76021 Karlsruhe, Germany > > > > Tel: +49(7247)82-8607 > > Fax: +49(7247)82-4972 > > > > e-mail: [log in to unmask] > > PGP DH/DSS Key: ID 0xCEB9826F > > Fingerprint: 5537 8473 CD26 507E 8EE2 BAAF 98E2 FD16 CEB9 826F > > __________________________________________ -- __________________________________________ Dr. Ulrich Schwickerath Forschungszentrum Karlsruhe GRID-Computing and e-Science Institut for Scientific Computing (IWR) P.O. Box 36 40 76021 Karlsruhe, Germany Tel: +49(7247)82-8607 Fax: +49(7247)82-4972 e-mail: [log in to unmask] PGP DH/DSS Key: ID 0xCEB9826F Fingerprint: 5537 8473 CD26 507E 8EE2 BAAF 98E2 FD16 CEB9 826F __________________________________________