URL: <http://savannah.cern.ch/bugs/?87507> Summary: Segfault in Xrootd startup Project: XROOTD Submitted by: bbockelm Submitted on: 2011-10-06 15:46 Severity: 3 - Normal Priority: 5 - Normal Status: None Privacy: Public Assigned to: None Originator Email: Open/Closed: Open Discussion Lock: Any Fixed by commit(s): _______________________________________________________ Details: It seems the race condition with the XrdCmsFinder is still around. I get the following stack trace for 3.1.0-rc1, which includes Andy's fix from Sept 19. Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x40908940 (LWP 1369)] XrdNet::Trim (this=0x0, hname=0x64209c "xrootd-itb") at /usr/src/debug/xrootd/xrootd/src/XrdNet/XrdNet.cc:258 258 if (Domlen && k > Domlen) (gdb) thread apply all bt [New Thread 0x40a09940 (LWP 1370)] Thread 11 (Thread 0x40a09940 (LWP 1370)): #0 0x0000003d29ad3f31 in clone () from /lib64/libc.so.6 #1 0x0000003d2a206660 in ?? () from /lib64/libpthread.so.0 #2 0x0000000040a09940 in ?? () #3 0x0000000000000000 in ?? () Thread 10 (Thread 0x40908940 (LWP 1369)): #0 XrdNet::Trim (this=0x0, hname=0x64209c "xrootd-itb") at /usr/src/debug/xrootd/xrootd/src/XrdNet/XrdNet.cc:258 #1 0x00002aaaab181a64 in XrdLink::Alloc (Peer=..., opts=0) at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdLink.cc:232 #2 0x00002aaaab17ef3e in XrdInet::Connect (this=0x63d220, host=<value optimized out>, port=1213, opts=<value optimized out>, tmo=<value optimized out>) at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdInet.cc:84 #3 0x00002aaaaaaf2164 in XrdCmsClientMan::Hookup (this=0x640140) at /usr/src/debug/xrootd/xrootd/src/XrdCms/XrdCmsClientMan.cc:300 #4 0x00002aaaaaaf264a in XrdCmsClientMan::Start (this=0x640140) at /usr/src/debug/xrootd/xrootd/src/XrdCms/XrdCmsClientMan.cc:202 #5 0x00002aaaab156ec7 in XrdSysThread_Xeq (myargs=<value optimized out>) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysPthread.cc:87 #6 0x0000003d2a20673d in start_thread () from /lib64/libpthread.so.0 #7 0x0000003d29ad3f6d in clone () from /lib64/libc.so.6 Thread 9 (Thread 0x40807940 (LWP 1368)): #0 0x0000003d2a20e1c1 in nanosleep () from /lib64/libpthread.so.0 #1 0x00002aaaaaaecdf1 in XrdAccConfig_Refresh (start_data=<value optimized out>) at /usr/src/debug/xrootd/xrootd/src/XrdAcc/XrdAccConfig.cc:85 #2 0x00002aaaab156ec7 in XrdSysThread_Xeq (myargs=<value optimized out>) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysPthread.cc:87 #3 0x0000003d2a20673d in start_thread () from /lib64/libpthread.so.0 #4 0x0000003d29ad3f6d in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x40706940 (LWP 1367)): #0 0x0000003d29acb186 in poll () from /lib64/libc.so.6 #1 0x00002aaaab18370e in XrdPollPoll::Start (this=0x63c330, syncsem=<value optimized out>, retcode=<value optimized out>) at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdPollPoll.icc:261 #2 0x00002aaaab1822d8 in XrdStartPolling (parg=<value optimized out>) at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdPoll.cc:87 #3 0x00002aaaab156ec7 in XrdSysThread_Xeq (myargs=<value optimized out>) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysPthread.cc:87 #4 0x0000003d2a20673d in start_thread () from /lib64/libpthread.so.0 #5 0x0000003d29ad3f6d in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x40605940 (LWP 1366)): #0 0x0000003d29acb186 in poll () from /lib64/libc.so.6 #1 0x00002aaaab18370e in XrdPollPoll::Start (this=0x63b730, syncsem=<value optimized out>, retcode=<value optimized out>) at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdPollPoll.icc:261 #2 0x00002aaaab1822d8 in XrdStartPolling (parg=<value optimized out>) at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdPoll.cc:87 #3 0x00002aaaab156ec7 in XrdSysThread_Xeq (myargs=<value optimized out>) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysPthread.cc:87 #4 0x0000003d2a20673d in start_thread () from /lib64/libpthread.so.0 #5 0x0000003d29ad3f6d in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x40504940 (LWP 1365)): #0 0x0000003d29acb186 in poll () from /lib64/libc.so.6 #1 0x00002aaaab18370e in XrdPollPoll::Start (this=0x6371d0, syncsem=<value optimized out>, retcode=<value optimized out>) at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdPollPoll.icc:261 #2 0x00002aaaab1822d8 in XrdStartPolling (parg=<value optimized out>) at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdPoll.cc:87 #3 0x00002aaaab156ec7 in XrdSysThread_Xeq (myargs=<value optimized out>) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysPthread.cc:87 #4 0x0000003d2a20673d in start_thread () from /lib64/libpthread.so.0 #5 0x0000003d29ad3f6d in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x40403940 (LWP 1364)): ---Type <return> to continue, or q <return> to quit--- #0 0x0000003d2a20cd01 in sem_wait () from /lib64/libpthread.so.0 #1 0x00002aaaab185b66 in Wait (this=0x2aaaaaf1b860) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysPthread.hh:317 #2 XrdScheduler::Run (this=0x2aaaaaf1b860) at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdScheduler.cc:260 #3 0x00002aaaab185d79 in XrdStartWorking (carg=0x2aaaaaf1b900) at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdScheduler.cc:65 #4 0x00002aaaab156ec7 in XrdSysThread_Xeq (myargs=<value optimized out>) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysPthread.cc:87 #5 0x0000003d2a20673d in start_thread () from /lib64/libpthread.so.0 #6 0x0000003d29ad3f6d in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x40302940 (LWP 1363)): #0 0x0000003d2a20cd01 in sem_wait () from /lib64/libpthread.so.0 #1 0x00002aaaab185b66 in Wait (this=0x2aaaaaf1b860) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysPthread.hh:317 #2 XrdScheduler::Run (this=0x2aaaaaf1b860) at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdScheduler.cc:260 #3 0x00002aaaab185d79 in XrdStartWorking (carg=0x2aaaaaf1b900) at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdScheduler.cc:65 #4 0x00002aaaab156ec7 in XrdSysThread_Xeq (myargs=<value optimized out>) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysPthread.cc:87 #5 0x0000003d2a20673d in start_thread () from /lib64/libpthread.so.0 #6 0x0000003d29ad3f6d in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x40201940 (LWP 1362)): #0 0x0000003d2a20b150 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002aaaab157062 in XrdSysCondVar::Wait (this=0x2aaaaaf1b950, sec=780) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysPthread.cc:130 #2 0x00002aaaab186332 in XrdScheduler::TimeSched (this=0x2aaaaaf1b860) at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdScheduler.cc:525 #3 0x00002aaaab186399 in XrdStartTSched (carg=0x2aaaaaf1b954) at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdScheduler.cc:59 #4 0x00002aaaab156ec7 in XrdSysThread_Xeq (myargs=<value optimized out>) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysPthread.cc:87 #5 0x0000003d2a20673d in start_thread () from /lib64/libpthread.so.0 #6 0x0000003d29ad3f6d in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x40100940 (LWP 1361)): #0 0x0000003d2a20b150 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002aaaab157062 in XrdSysCondVar::Wait (this=0x2aaaaaf1bb18, sec=1200) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysPthread.cc:130 #2 0x00002aaaab17e883 in XrdBuffManager::Reshape (this=0x2aaaaaf1ba10) at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdBuffer.cc:205 #3 0x00002aaaab17ebb9 in XrdReshaper (pp=0x2aaaaaf1bb1c) at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdBuffer.cc:35 #4 0x00002aaaab156ec7 in XrdSysThread_Xeq (myargs=<value optimized out>) at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysPthread.cc:87 #5 0x0000003d2a20673d in start_thread () from /lib64/libpthread.so.0 #6 0x0000003d29ad3f6d in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x2aaaab39d4a0 (LWP 1358)): #0 0x0000003d29ad3f31 in clone () from /lib64/libc.so.6 #1 0x0000003d2a20719f in pthread_create@@GLIBC_2.2.5 () from /lib64/libpthread.so.0 #2 0x00002aaaab156dc4 in XrdSysThread::Run (tid=0x7fffffffb688, proc=0x2aaaaaaf3a10 <XrdCmsStartResp(void*)>, arg=0x0, opts=0, tDesc=0x2aaaaaafd313 "async callback") at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysPthread.cc:283 #3 0x00002aaaaaaf3bd0 in XrdCmsFinderRMT::StartManagers (this=<value optimized out>, myManList=<value optimized out>) at /usr/src/debug/xrootd/xrootd/src/XrdCms/XrdCmsFinder.cc:652 #4 0x00002aaaaaaf5fc8 in XrdCmsFinderRMT::Configure (this=0x63ff10, cfn=0x63eb60 "/etc/xrootd/xrootd.cf", envP=0x7fffffffcf80) at /usr/src/debug/xrootd/xrootd/src/XrdCms/XrdCmsFinder.cc:153 #5 0x00002aaaaaf2fbae in XrdOfs::ConfigRedir (this=0x2aaaab13b940, Eroute=..., EnvInfo=0x7fffffffcf80) at /usr/src/debug/xrootd/xrootd/src/XrdOfs/XrdOfsConfig.cc:454 #6 0x00002aaaaaf309a1 in XrdOfs::Configure (this=0x2aaaab13b940, Eroute=..., EnvInfo=0x7fffffffcf80) at /usr/src/debug/xrootd/xrootd/src/XrdOfs/XrdOfsConfig.cc:199 #7 0x00002aaaaaf2d8d8 in XrdSfsGetDefaultFileSystem (native_fs=<value optimized out>, lp=<value optimized out>, configfn=0x62c1e0 "/etc/xrootd/xrootd.cf", EnvInfo=0x7fffffffcf80) at /usr/src/debug/xrootd/xrootd/src/XrdOfs/XrdOfsFS.cc:45 #8 0x00000000004112c6 in XrdXrootdProtocol::Configure (parms=<value optimized out>, pi=0x2aaaaaf1b6c0) at /usr/src/debug/xrootd/xrootd/src/XrdXrootd/XrdXrootdConfig.cc:234 #9 0x0000000000418519 in XrdgetProtocol (pname=<value optimized out>, parms=0x3d0f00 <Address 0x3d0f00 out of bounds>, pi=0x2aaaaaf1b6c0) ---Type <return> to continue, or q <return> to quit--- at /usr/src/debug/xrootd/xrootd/src/XrdXrootd/XrdXrootdProtocol.cc:131 #10 0x00002aaaab184993 in XrdProtLoad::Load (lname=0x0, pname=0x636d60 "xrootd", parms=0x0, pi=0x2aaaaaf1b6c0) at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdProtLoad.cc:78 #11 0x00002aaaaad17ded in XrdConfig::Setup (this=0x2aaaaaf1b6c0, dfltp=<value optimized out>) at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdConfig.cc:743 #12 0x00002aaaaad18b76 in XrdConfig::Configure (this=0x2aaaaaf1b6c0, argc=8, argv=<value optimized out>) at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdConfig.cc:385 #13 0x00002aaaaad19474 in main (argc=8, argv=0x7fffffffd598) at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdMain.cc:154 #14 0x0000003d29a1d994 in __libc_start_main () from /lib64/libc.so.6 #15 0x000000000040b329 in _start () (gdb) q _______________________________________________________ Reply to this item at: <http://savannah.cern.ch/bugs/?87507> _______________________________________________ Message sent via/by LCG Savannah http://savannah.cern.ch/