 |
 |
Yay, I've got something from valgrind ... it's not in MessageHandler but has to do with Message. Let me know if this makes sense, I'll keep pounding at it for a while :) [tcpkill was not good enough, finding socket fd with netstat and then calling close(fd) in gdb did the trick).] ``` [2019-03-25 16:35:00.782184 -0700][Error ][XRootD ] [[log in to unmask]:1094] MsgHandler created: 53fab20 (message: kXR_read (handle: 0x00000000, offset: 36175872, size: 524288) ). [2019-03-25 16:35:00.783144 -0700][Error ][XRootD ] [[log in to unmask]:1094] Moving MsgHandler: 53fab20 (message: kXR_read (handle: 0x00000000, offset: 36175872, size: 524288) ) from out-queu to in-queue. [2019-03-25 16:35:00.811473 -0700][Error ][XRootD ] [[log in to unmask]:1094] Calling MsgHandler: 53fab20 (message: kXR_read (handle: 0x00000000, offset: 36175872, size: 524288) ) with status: [SUCCESS] . [2019-03-25 16:35:00.812114 -0700][Error ][XRootD ] [[log in to unmask]:1094] Destroying MsgHandler: 53fab20. [2019-03-25 16:35:16.526807 -0700][Error ][XRootD ] [[log in to unmask]:1094] MsgHandler created: 5454260 (message: kXR_read (handle: 0x00000000, offset: 36700160, size: 524288) ). [2019-03-25 16:35:16.537508 -0700][Error ][Poller ] <[::ffff:169.228.130.108]:38338><--><[::ffff:169.228.130.61]:1094> Unable to enable write notifications: modifying poll events [2019-03-25 16:35:16.538760 -0700][Error ][XRootD ] [[log in to unmask]:1094] Unable to send the message kXR_read (handle: 0x00000000, offset: 36700160, size: 524288): [FATAL] Poller error ==119570== Thread 87: ==119570== Invalid read of size 2 ==119570== at 0x6547A4E: XrdCl::SIDManager::ReleaseSID(unsigned char*) (XrdClSIDManager.cc:61) ==119570== by 0x655F97A: XrdCl::MessageUtils::SendMessage(XrdCl::URL const&, XrdCl::Message*, XrdCl::ResponseHandler*, XrdCl::MessageSendParams const&, XrdCl::LocalFileHandler*) (XrdClMessageUtils.cc:122) ==119570== by 0x6566096: XrdCl::FileStateHandler::IssueRequest(XrdCl::URL const&, XrdCl::Message*, XrdCl::ResponseHandler*, XrdCl::MessageSendParams&) (XrdClFileStateHandler.cc:1956) ==119570== by 0x656BC9C: XrdCl::FileStateHandler::SendOrQueue(XrdCl::URL const&, XrdCl::Message*, XrdCl::ResponseHandler*, XrdCl::MessageSendParams&) (XrdClFileStateHandler.cc:1588) ==119570== by 0x656F1F3: XrdCl::FileStateHandler::Read(unsigned long, unsigned int, void*, XrdCl::ResponseHandler*, unsigned short) (XrdClFileStateHandler.cc:692) ==119570== by 0x6563129: XrdCl::File::Read(unsigned long, unsigned int, void*, XrdCl::ResponseHandler*, unsigned short) (XrdClFile.cc:194) ==119570== by 0x62D8088: XrdPosixFile::Read(XrdOucCacheIOCB&, char*, long long, int) (XrdPosixFile.cc:409) ==119570== by 0x6B0EB2F: XrdFileCache::File::ProcessBlockRequests(std::list<XrdFileCache::Block*, std::allocator<XrdFileCache::Block*> >&, bool) (XrdFileCacheFile.cc:466) ==119570== by 0x6B128A1: XrdFileCache::File::Read(XrdFileCache::IO*, char*, long long, int) (XrdFileCacheFile.cc:627) ==119570== by 0x6B1AC23: XrdFileCache::IOEntireFile::Read(char*, long long, int) (XrdFileCacheIOEntireFile.cc:174) ==119570== by 0x62DBF41: XrdPosixXrootd::Pread(int, void*, unsigned long, long) (XrdPosixXrootd.cc:725) ==119570== by 0x60BC33F: XrdPssFile::Read(void*, long, unsigned long) (XrdPss.cc:811) ==119570== Address 0x54b1ce0 is 0 bytes inside a block of size 24 free'd ==119570== at 0x4A06C20: realloc (vg_replace_malloc.c:662) ==119570== by 0x653F0D9: XrdCl::XRootDTransport::MultiplexSubStream(XrdCl::Message*, unsigned short, XrdCl::AnyObject&, XrdCl::PathID*) (XrdClBuffer.hh:90) ==119570== by 0x6539E37: XrdCl::Stream::Send(XrdCl::Message*, XrdCl::OutgoingMsgHandler*, bool, long) (XrdClStream.cc:300) ==119570== by 0x653777D: XrdCl::Channel::Send(XrdCl::Message*, XrdCl::OutgoingMsgHandler*, bool, long) (XrdClChannel.cc:312) ==119570== by 0x6536867: XrdCl::PostMaster::Send(XrdCl::URL const&, XrdCl::Message*, XrdCl::OutgoingMsgHandler*, bool, long) (XrdClPostMaster.cc:191) ==119570== by 0x655F8E3: XrdCl::MessageUtils::SendMessage(XrdCl::URL const&, XrdCl::Message*, XrdCl::ResponseHandler*, XrdCl::MessageSendParams const&, XrdCl::LocalFileHandler*) (XrdClMessageUtils.cc:113) ==119570== by 0x6566096: XrdCl::FileStateHandler::IssueRequest(XrdCl::URL const&, XrdCl::Message*, XrdCl::ResponseHandler*, XrdCl::MessageSendParams&) (XrdClFileStateHandler.cc:1956) ==119570== by 0x656BC9C: XrdCl::FileStateHandler::SendOrQueue(XrdCl::URL const&, XrdCl::Message*, XrdCl::ResponseHandler*, XrdCl::MessageSendParams&) (XrdClFileStateHandler.cc:1588) ==119570== by 0x656F1F3: XrdCl::FileStateHandler::Read(unsigned long, unsigned int, void*, XrdCl::ResponseHandler*, unsigned short) (XrdClFileStateHandler.cc:692) ==119570== by 0x6563129: XrdCl::File::Read(unsigned long, unsigned int, void*, XrdCl::ResponseHandler*, unsigned short) (XrdClFile.cc:194) ==119570== by 0x62D8088: XrdPosixFile::Read(XrdOucCacheIOCB&, char*, long long, int) (XrdPosixFile.cc:409) ==119570== by 0x6B0EB2F: XrdFileCache::File::ProcessBlockRequests(std::list<XrdFileCache::Block*, std::allocator<XrdFileCache::Block*> >&, bool) (XrdFileCacheFile.cc:466) ==119570== ==119570== (action on error) vgdb me ... ==119570== Continuing ... [2019-03-25 16:39:31.869607 -0700][Error ][XRootD ] [[log in to unmask]:1094] Destroying MsgHandler: 5454260. [2019-03-25 16:39:33.684680 -0700][Error ][XRootD ] [[log in to unmask]:1094] MsgHandler created: 30e1e600 (message: kXR_read (handle: 0x00000000, offset: 37224448, size: 524288) ). [2019-03-25 16:39:46.888592 -0700][Error ][Poller ] <[::ffff:169.228.130.108]:38338><--><[::ffff:169.228.130.61]:1094> Unable to disable write notifications: modifying poll events [2019-03-25 16:39:46.892323 -0700][Error ][XRootD ] [[log in to unmask]:1094] Impossible to send message kXR_read (handle: 0x00000000, offset: 37224448, size: 524288). Trying to recover. [2019-03-25 16:39:46.893646 -0700][Error ][XRootD ] [[log in to unmask]:1094] Unable to get the response to request kXR_read (handle: 0x00000000, offset: 37224448, size: 524288) [2019-03-25 16:39:46.894128 -0700][Error ][XRootD ] [[log in to unmask]:1094] Passing to the thread-pool MsgHandler: 30e1e600 (message: kXR_read (handle: 0x00000000, offset: 37224448, size: 524288) ). ``` -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/xrootd/xrootd/issues/943#issuecomment-476418844 ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1