It looks like the built-in poller is deadlocking in:

XrdSys::IOEvents::Channel::Enable vs XrdSys::IOEvents::Poller::CbkTMO. In threads 26 and 24.

Thread 26 (Thread 0x40304940 (LWP 17344)):
#0  0x0000003da6a0d524 in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x0000003da6a08e35 in _L_lock_1127 () from /lib64/libpthread.so.0
#2  0x0000003da6a08d33 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x000000326f222637 in Lock (this=0x2aaaae41d180, cP=0x2aaaacf4f200, events=<value optimized out>, eNum=0, eTxt=0x0) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysPthread.hh:149
#4  Lock (this=0x2aaaae41d180, cP=0x2aaaacf4f200, events=<value optimized out>, eNum=0, eTxt=0x0) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysPthread.hh:197
#5  XrdSys::IOEvents::Poller::CbkXeq (this=0x2aaaae41d180, cP=0x2aaaacf4f200, events=<value optimized out>, eNum=0, eTxt=0x0) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysIOEvents.cc:626
#6  0x000000326f22286c in XrdSys::IOEvents::Poller::CbkTMO (this=0x2aaaae41d180) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysIOEvents.cc:548
#7  0x000000326f2229bb in XrdSys::IOEvents::PollE::Begin (this=0x2aaaae41d180, syncsem=<value optimized out>, retcode=<value optimized out>, eTxt=<value optimized out>)
    at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysIOEventsPollE.icc:202
#8  0x000000326f221aa4 in XrdSys::IOEvents::BootStrap::Start (parg=0x40e802b0) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysIOEvents.cc:110
#9  0x000000326f21fc2f in XrdSysThread_Xeq (myargs=<value optimized out>) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysPthread.cc:86
#10 0x0000003da6a0673d in start_thread () from /lib64/libpthread.so.0
#11 0x0000003da62d44bd in clone () from /lib64/libc.so.6

Thread 25 (Thread 0x42738940 (LWP 17345)):
#0  0x0000003da6a0d524 in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x0000003da6a08e35 in _L_lock_1127 () from /lib64/libpthread.so.0
#2  0x0000003da6a08d33 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x00002aaaad668394 in Lock (this=0x2aaaacf94140, now=128) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysPthread.hh:149
#4  XrdCl::Stream::Tick (this=0x2aaaacf94140, now=128) at /usr/src/debug/xrootd/xrootd/src/XrdCl/XrdClStream.cc:310
#5  0x00002aaaad665f2f in XrdCl::Channel::Tick (this=<value optimized out>, now=1365673017) at /usr/src/debug/xrootd/xrootd/src/XrdCl/XrdClChannel.cc:303
#6  0x00002aaaad666acf in (anonymous namespace)::TickGeneratorTask::Run (this=<value optimized out>, now=128) at /usr/src/debug/xrootd/xrootd/src/XrdCl/XrdClChannel.cc:171
#7  0x00002aaaad671d50 in XrdCl::TaskManager::RunTasks (this=0x2aaaae41d0c0) at /usr/src/debug/xrootd/xrootd/src/XrdCl/XrdClTaskManager.cc:216
#8  0x00002aaaad671e59 in RunRunnerThread (arg=0x2aaaacf94170) at /usr/src/debug/xrootd/xrootd/src/XrdCl/XrdClTaskManager.cc:36
---Type <return> to continue, or q <return> to quit---
#9  0x0000003da6a0673d in start_thread () from /lib64/libpthread.so.0
#10 0x0000003da62d44bd in clone () from /lib64/libc.so.6

Thread 24 (Thread 0x41b3d940 (LWP 17368)):
#0  0x0000003da6a0d524 in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x0000003da6a08e35 in _L_lock_1127 () from /lib64/libpthread.so.0
#2  0x0000003da6a08d33 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x000000326f22211c in Lock (this=0x2aaaae41d180, cP=0x80) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysPthread.hh:149
#4  XrdSysMutexHelper (this=0x2aaaae41d180, cP=0x80) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysPthread.hh:208
#5  XrdSys::IOEvents::Poller::TmoAdd (this=0x2aaaae41d180, cP=0x80) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysIOEvents.cc:965
#6  0x000000326f22300a in XrdSys::IOEvents::Channel::Enable (this=0x2aaaacf4f200, events=<value optimized out>, timeout=1, eText=0x41b3c768)
    at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysIOEvents.cc:373
#7  0x00002aaaad663293 in XrdCl::PollerBuiltIn::EnableWriteNotification (this=<value optimized out>, socket=0x2aaaacf99190, notify=<value optimized out>, timeout=1)
    at /usr/src/debug/xrootd/xrootd/src/XrdCl/XrdClPollerBuiltIn.cc:404
#8  0x00002aaaad669423 in EnableUplink (this=0x2aaaacf94140, path=...) at /usr/src/debug/xrootd/xrootd/src/XrdCl/XrdClAsyncSocketHandler.hh:96
#9  XrdCl::Stream::EnableLink (this=0x2aaaacf94140, path=...) at /usr/src/debug/xrootd/xrootd/src/XrdCl/XrdClStream.cc:186
#10 0x00002aaaad669729 in XrdCl::Stream::Send (this=0x2aaaacf94140, msg=0x2aaaaf3dee20, handler=0x2aaaaf13ff08, stateful=true, expires=1365673317)
    at /usr/src/debug/xrootd/xrootd/src/XrdCl/XrdClStream.cc:273
#11 0x00002aaaad665ede in XrdCl::Channel::Send (this=0x2aaaac1edec0, msg=0x2aaaaf3dee20, handler=0x2aaaaf13ff08, stateful=true, expires=1365673317)
    at /usr/src/debug/xrootd/xrootd/src/XrdCl/XrdClChannel.cc:266
#12 0x00002aaaad66539e in XrdCl::PostMaster::Send (this=<value optimized out>, url=<value optimized out>, msg=0x2aaaaf3dee20, handler=0x2aaaaf13ff08, stateful=true, expires=1365673317)
    at /usr/src/debug/xrootd/xrootd/src/XrdCl/XrdClPostMaster.cc:169
#13 0x00002aaaad684f10 in XrdCl::MessageUtils::SendMessage (url=..., msg=0x2aaaaf3dee20, handler=<value optimized out>, sendParams=...) at /usr/src/debug/xrootd/xrootd/src/XrdCl/XrdClMessageUtils.cc:109
#14 0x00002aaaad68e836 in XrdCl::FileStateHandler::SendOrQueue (this=0x2aaaac1ed600, url=<value optimized out>, msg=0x2aaaaf3dee20, handler=0x2aaaaf02d9e0, sendParams=...)
    at /usr/src/debug/xrootd/xrootd/src/XrdCl/XrdClFileStateHandler.cc:1129
#15 0x00002aaaad690716 in XrdCl::FileStateHandler::Stat (this=0x2aaaac1ed600, force=<value optimized out>, handler=0x41b3cf00, timeout=0)
    at /usr/src/debug/xrootd/xrootd/src/XrdCl/XrdClFileStateHandler.cc:496
#16 0x00002aaaad687904 in XrdCl::File::Stat (this=<value optimized out>, force=false, handler=0xffffffffffffffff, timeout=53752) at /usr/src/debug/xrootd/xrootd/src/XrdCl/XrdClFile.cc:103
#17 0x00002aaaad68858e in XrdCl::File::Stat (this=0x2aaaac388810, force=true, response=@0x41b3d020, timeout=0) at /usr/src/debug/xrootd/xrootd/src/XrdCl/XrdClFile.cc:114
#18 0x00002aaaad2db2e9 in XrdMqClient::RecvMessage (this=0x2aaaad62ce60) at /afs/cern.ch/work/a/apeters/eos-master/eos/mq/XrdMqClient.cc:304
#19 0x00002aaaad083f58 in eos::mgm::Messaging::Listen (this=0x2aaaac1c9600) at /afs/cern.ch/work/a/apeters/eos-master/eos/mgm/Messaging.cc:185
#20 0x00002aaaad2e1669 in XrdMqMessaging::Start (pp=<value optimized out>) at /afs/cern.ch/work/a/apeters/eos-master/eos/mq/XrdMqMessaging.cc:35
#21 0x000000326f21fc2f in XrdSysThread_Xeq (myargs=<value optimized out>) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysPthread.cc:86
#22 0x0000003da6a0673d in start_thread () from /lib64/libpthread.so.0
#23 0x0000003da62d44bd in clone () from /lib64/libc.so.6


Reply to this email directly or view it on GitHub.



Use REPLY-ALL to reply to list

To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1