 |
 |
Hi, Fabrizio, > Is this your problem, i.e. that you have no points where to put your > "fast disconnection request" at the client side when the app simply exits ? yes, this is exactly the problem. I want to touch as little code as possible, for obvious reasons :-) Cheers, Ulrich > > > > Fabrizio > > Ulrich Schwickerath wrote: > > Hi, Fabrizio, > > > > thank's a lot for the fast answer! > > > >> Re-looking at your log I don't understand the problem. xrdcp is > >>supposed to exit after having closed the file, so the connection is > >>closed and the server notices this because there are no more bytes to > >>read from the connection. Do I miss something? > > > > Well, I'm using native InfiniBand. Basically, what you have is a pair of > > connected send and receive queue on both sides. If you want to send > > something, you post a send request on the sender side. The receiver, on > > the other hand, posts a receive request (containing the adress of the > > buffer where the data should go), and then polls the receive queue. If > > some data has arrived, it will see a completion event. At that point the > > data has already arrived. The problem in this case is that if the sender > > does not send anything, the receiver will poll the receive queue until > > the time out is reached, and that is exactly what happens. My idea was to > > post a send request with no data attached, so that the receiver will not > > time out but see no data which would emulate the behavior that is > > expected. > > > >> Anyway, the newer xrdcp, using the latest client, should avoid > >>requesting data over the eof, that was one of the latest commits, but it > >>implements a different schema for data transfers, which now are done > >>both synchronously and asynchronously (i.e. in parallel, while the > >>application "thinks"). The problem is that this stuff is in the head. > >>Here it works, so if you are adventurous you can test it and report here > >>if you have troubles. > > > > I started with 20050413-0433, so I'll update to the new version that just > > came in :-) > > > > Thank's a lot! > > Ulrich > > > >>Ulrich Schwickerath wrote: > >>>Hi, > >>> > >>>I (finally) found a bit of time to work on a > >>>port of Xrootd to (native) InfiniBand, and a > >>>proof-of-concept version is now sort of working, meaning > >>>that I can now transfer files with xrdcp. If possible, I would > >>>like to show first results with this version at the ACAT05 > >>>conference in Berlin in about 3 weeks from now. > >>>I have a few questions to you: > >>> > >>>1/ what would be meaningful benchmark for xrootd ? > >>> For the moment, I'm only using point-to-point transfers > >>> (remote read to /dev/null, mainly) which is a bit boring ... > >>> > >>>2/ Is there a ready-to-use test suite that can/should be used ? > >>> > >>>3/ I still have one problem with xrdcp. For the client I have added > >>>a few lines of code in the XrdClientPhyConnection.cc which > >>>initializes my InfiniBand connection, and bypasses the recv/write > >>>calls. Everything works fine until the end. The Server log looks > >>>like this: > >>>050501 21:36:41 5282 schwicke.17140:12@iwrcgop027 XrootdProtocol: 0000 > >>>req=3003 dlen=0 > >>>050501 21:36:41 5282 schwicke.17140:12@iwrcgop027 XrootdFile: closing > >>>r /tmp/testfile.dat > >>>050501 21:36:41 5282 schwicke.17140:12@iwrcgop027 XrootdProtocol: 0000 > >>>close fh=0 > >>>050501 21:36:41 5282 schwicke.17140:12@iwrcgop027 XrootdResponse: 0000 > >>>sending OK > >>>050501 21:36:44 5282 XrootdXeq: schwicke.17140:12@iwrcgop027 disc > >>> 0:04:24 (link read error) > >>>050501 21:36:44 5282 schwicke.17140:12@iwrcgop027 XrdPoll: sending > >>> poller 0 detach for link 12 > >>>050501 21:36:44 5282 XrdPoll: Poller 0 detached fd 12 entry 1 now at 1 > >>>050501 21:36:44 5282 schwicke.17140:12@iwrcgop027 XrdPoll: FD 12 > >>> detached from poller 0; num=0 > >>>For TCP/IP I see that the last recv request succeeds in the poll > >>>command but ends with zero bytes of data, resulting in a ENOMSG return > >>>code. For InfiniBand, this call simply times out, giving the above > >>>link read error. A workaround would be to send a zero size message over > >>>the InfiniBand link just before the connection is closed for good. I > >>>tried to do that inside the XrdClientPhyConnection destructor, but that > >>>one is never called in xrdcp. A side effect of this is that the > >>> resources used by the InfiniBand connection need to be cleaned up and > >>> freed by the driver resource tracking mechanisms. Where should this be > >>> done instead? > >>> > >>> > >>>Thank's a lot in advance, > >>>Ulrich -- __________________________________________ Dr. Ulrich Schwickerath Forschungszentrum Karlsruhe GRID-Computing and e-Science Institut for Scientific Computing (IWR) P.O. Box 36 40 76021 Karlsruhe, Germany Tel: +49(7247)82-8607 Fax: +49(7247)82-4972 e-mail: [log in to unmask] PGP DH/DSS Key: ID 0xCEB9826F Fingerprint: 5537 8473 CD26 507E 8EE2 BAAF 98E2 FD16 CEB9 826F __________________________________________