Could you also post the logfile with about 10 minutes of context right before the crash? Hopefully, it won’t be huge. If it is, just send it to me directly – Andy<br>
<br>
From: jthiltges <br>
Sent: Thursday, November 09, 2017 2:17 PM<br>
To: xrootd/xrootd <br>
Cc: Subscribed <br>
Subject: [xrootd/xrootd] segfault in XrdXrootdMonFile::DoXFR (#618)<br>
<br>
Dear XRootD developers,<br>
<br>
In testing the OSG build of xrootd v4.7.1 on our StashCache server at Nebraska, we encountered a segfault in XrdXrootdMonFile::DoXFR(). @bbockelm suggested sending the backtrace upstream:<br>
<br>
Core was generated by `/usr/bin/xrootd -l /var/log/xrootd/xrootd.log -c /etc/xrootd/xrootd-stashcache-&#39;.<br>
Program terminated with signal 11, Segmentation fault.<br>
#0  XrdXrootdMonFile::DoXFR () at /usr/src/debug/xrootd-4.7.1/src/XrdXrootd/XrdXrootdMonFile.cc:271<br>
271	                 {if (fsP-&gt;xfrXeq) DoXFR(fsP);<br>
Missing separate debuginfos, ...<br>
(gdb) bt<br>
#0  XrdXrootdMonFile::DoXFR () at /usr/src/debug/xrootd-4.7.1/src/XrdXrootd/XrdXrootdMonFile.cc:271<br>
#1  0x00007f09b61d03a5 in XrdXrootdMonFile::DoIt (this=0x1983fa0) at /usr/src/debug/xrootd-4.7.1/src/XrdXrootd/XrdXrootdMonFile.cc:230<br>
#2  0x00007f09b5f5ecff in XrdScheduler::Run (this=0x610e98 &lt;XrdMain::Config+440&gt;) at /usr/src/debug/xrootd-4.7.1/src/Xrd/XrdScheduler.cc:357<br>
#3  0x00007f09b5f5ee49 in XrdStartWorking (carg=&lt;optimized out&gt;) at /usr/src/debug/xrootd-4.7.1/src/Xrd/XrdScheduler.cc:87<br>
#4  0x00007f09b5f1b4d7 in XrdSysThread_Xeq (myargs=0x7f0834513270) at /usr/src/debug/xrootd-4.7.1/src/XrdSys/XrdSysPthread.cc:86<br>
#5  0x00007f09b5ad7e25 in start_thread (arg=0x7f0412ded700) at pthread_create.c:308<br>
#6  0x00007f09b4ddd34d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113<br>
And here&#39;s a bit more info on the loop it&#39;s running<br>
<br>
(gdb) # fsP doesn&#39;t point to allocated memory<br>
(gdb) print fsP<br>
$1 = (XrdXrootdFileStats *) 0x7f036c700d98<br>
(gdb) print *fsP<br>
Cannot access memory at address 0x7f036c700d98<br>
<br>
(gdb) # Loop variables<br>
(gdb) print i<br>
$2 = 124<br>
(gdb) print n<br>
$3 = 232<br>
<br>
(gdb) # Bad fMap entry and adjacent good entries<br>
(gdb) print XrdXrootdMonFile::fmMap[i].fMap[n-2]<br>
$4 = {{cVal = 139672740028968, cPtr = 0x7f08180de228, vPtr = 0x7f08180de228}}<br>
(gdb) print XrdXrootdMonFile::fmMap[i].fMap[n-1]<br>
$5 = {{cVal = 139652680912280, cPtr = 0x7f036c700d98, vPtr = 0x7f036c700d98}}<br>
(gdb) print XrdXrootdMonFile::fmMap[i].fMap[n]<br>
$6 = {{cVal = 139675760573464, cPtr = 0x7f08cc17bc18, vPtr = 0x7f08cc17bc18}}<br>
<br>
(gdb) # Bad fMap.vPtr entry and adjacent good entries<br>
(gdb) print *(XrdXrootdMonFile::fmMap[i].fMap[n-2].vPtr)<br>
$7 = {FileID = 1276444928, MonEnt = -1818, monLvl = 2 &#39;\002&#39;, xfrXeq = 0 &#39;\000&#39;, fSize = 180785925, xfr = {read = 12931845, readv = 0, write = 0}, ops = {read = 13, readv = 0, write = 0, rsMin = 32767, rsMax = 0, rsegs = 0, rdMin = 348933, rdMax = 1048576, <br>
    rvMin = 2147483647, rvMax = 0, wrMin = 2147483647, wrMax = 0}, ssq = {read = 0, readv = 0, rsegs = 0, write = 0}}<br>
(gdb) print *(XrdXrootdMonFile::fmMap[i].fMap[n-1].vPtr)<br>
Cannot access memory at address 0x7f036c700d98<br>
(gdb) print *(XrdXrootdMonFile::fmMap[i].fMap[n].vPtr)<br>
$8 = {FileID = 1309999360, MonEnt = -1816, monLvl = 2 &#39;\002&#39;, xfrXeq = 0 &#39;\000&#39;, fSize = 178914241, xfr = {read = 33570816, readv = 0, write = 0}, ops = {read = 33, readv = 0, write = 0, rsMin = 32767, rsMax = 0, rsegs = 0, rdMin = 16384, rdMax = 1048576, <br>
    rvMin = 2147483647, rvMax = 0, wrMin = 2147483647, wrMax = 0}, ssq = {read = 0, readv = 0, rsegs = 0, write = 0}}<br>
—<br>
You are receiving this because you are subscribed to this thread.<br>
Reply to this email directly, view it on GitHub, or mute the thread.<br>


<p style="font-size:small;-webkit-text-size-adjust:none;color:#666;">&mdash;<br />You are receiving this because you are subscribed to this thread.<br />Reply to this email directly, <a href="https://github.com/xrootd/xrootd/issues/618#issuecomment-343316950">view it on GitHub</a>, or <a href="https://github.com/notifications/unsubscribe-auth/AD7YjnGK7g3584Gv1JvRGuFFf9h7A2Dbks5s04C0gaJpZM4QYv4w">mute the thread</a>.<img alt="" height="1" src="https://github.com/notifications/beacon/AD7YjjI8jXsEA_7gQzS8fliKia1aCsthks5s04C0gaJpZM4QYv4w.gif" width="1" /></p>
<div itemscope itemtype="http://schema.org/EmailMessage">
<div itemprop="action" itemscope itemtype="http://schema.org/ViewAction">
  <link itemprop="url" href="https://github.com/xrootd/xrootd/issues/618#issuecomment-343316950"></link>
  <meta itemprop="name" content="View Issue"></meta>
</div>
<meta itemprop="description" content="View this Issue on GitHub"></meta>
</div>

<!cript type="application/json" data-scope="inboxmarkup">{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/xrootd/xrootd","title":"xrootd/xrootd","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/xrootd/xrootd"}},"updates":{"snippets":[{"icon":"PERSON","message":"@abh3 in #618: Could you also post the logfile with about 10 minutes of context right before the crash? Hopefully, it won’t be huge. If it is, just send it to me directly – Andy\n\nFrom: jthiltges \nSent: Thursday, November 09, 2017 2:17 PM\nTo: xrootd/xrootd \nCc: Subscribed \nSubject: [xrootd/xrootd] segfault in XrdXrootdMonFile::DoXFR (#618)\n\nDear XRootD developers,\n\nIn testing the OSG build of xrootd v4.7.1 on our StashCache server at Nebraska, we encountered a segfault in XrdXrootdMonFile::DoXFR(). @bbockelm suggested sending the backtrace upstream:\n\nCore was generated by `/usr/bin/xrootd -l /var/log/xrootd/xrootd.log -c /etc/xrootd/xrootd-stashcache-'.\nProgram terminated with signal 11, Segmentation fault.\n#0  XrdXrootdMonFile::DoXFR () at /usr/src/debug/xrootd-4.7.1/src/XrdXrootd/XrdXrootdMonFile.cc:271\n271\t                 {if (fsP-\u003exfrXeq) DoXFR(fsP);\nMissing separate debuginfos, ...\n(gdb) bt\n#0  XrdXrootdMonFile::DoXFR () at /usr/src/debug/xrootd-4.7.1/src/XrdXrootd/XrdXrootdMonFile.cc:271\n#1  0x00007f09b61d03a5 in XrdXrootdMonFile::DoIt (this=0x1983fa0) at /usr/src/debug/xrootd-4.7.1/src/XrdXrootd/XrdXrootdMonFile.cc:230\n#2  0x00007f09b5f5ecff in XrdScheduler::Run (this=0x610e98 \u003cXrdMain::Config+440\u003e) at /usr/src/debug/xrootd-4.7.1/src/Xrd/XrdScheduler.cc:357\n#3  0x00007f09b5f5ee49 in XrdStartWorking (carg=\u003coptimized out\u003e) at /usr/src/debug/xrootd-4.7.1/src/Xrd/XrdScheduler.cc:87\n#4  0x00007f09b5f1b4d7 in XrdSysThread_Xeq (myargs=0x7f0834513270) at /usr/src/debug/xrootd-4.7.1/src/XrdSys/XrdSysPthread.cc:86\n#5  0x00007f09b5ad7e25 in start_thread (arg=0x7f0412ded700) at pthread_create.c:308\n#6  0x00007f09b4ddd34d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113\nAnd here's a bit more info on the loop it's running\n\n(gdb) # fsP doesn't point to allocated memory\n(gdb) print fsP\n$1 = (XrdXrootdFileStats *) 0x7f036c700d98\n(gdb) print *fsP\nCannot access memory at address 0x7f036c700d98\n\n(gdb) # Loop variables\n(gdb) print i\n$2 = 124\n(gdb) print n\n$3 = 232\n\n(gdb) # Bad fMap entry and adjacent good entries\n(gdb) print XrdXrootdMonFile::fmMap[i].fMap[n-2]\n$4 = {{cVal = 139672740028968, cPtr = 0x7f08180de228, vPtr = 0x7f08180de228}}\n(gdb) print XrdXrootdMonFile::fmMap[i].fMap[n-1]\n$5 = {{cVal = 139652680912280, cPtr = 0x7f036c700d98, vPtr = 0x7f036c700d98}}\n(gdb) print XrdXrootdMonFile::fmMap[i].fMap[n]\n$6 = {{cVal = 139675760573464, cPtr = 0x7f08cc17bc18, vPtr = 0x7f08cc17bc18}}\n\n(gdb) # Bad fMap.vPtr entry and adjacent good entries\n(gdb) print *(XrdXrootdMonFile::fmMap[i].fMap[n-2].vPtr)\n$7 = {FileID = 1276444928, MonEnt = -1818, monLvl = 2 '\\002', xfrXeq = 0 '\\000', fSize = 180785925, xfr = {read = 12931845, readv = 0, write = 0}, ops = {read = 13, readv = 0, write = 0, rsMin = 32767, rsMax = 0, rsegs = 0, rdMin = 348933, rdMax = 1048576, \n    rvMin = 2147483647, rvMax = 0, wrMin = 2147483647, wrMax = 0}, ssq = {read = 0, readv = 0, rsegs = 0, write = 0}}\n(gdb) print *(XrdXrootdMonFile::fmMap[i].fMap[n-1].vPtr)\nCannot access memory at address 0x7f036c700d98\n(gdb) print *(XrdXrootdMonFile::fmMap[i].fMap[n].vPtr)\n$8 = {FileID = 1309999360, MonEnt = -1816, monLvl = 2 '\\002', xfrXeq = 0 '\\000', fSize = 178914241, xfr = {read = 33570816, readv = 0, write = 0}, ops = {read = 33, readv = 0, write = 0, rsMin = 32767, rsMax = 0, rsegs = 0, rdMin = 16384, rdMax = 1048576, \n    rvMin = 2147483647, rvMax = 0, wrMin = 2147483647, wrMax = 0}, ssq = {read = 0, readv = 0, rsegs = 0, write = 0}}\n—\nYou are receiving this because you are subscribed to this thread.\nReply to this email directly, view it on GitHub, or mute the thread.\n"}],"action":{"name":"View Issue","url":"https://github.com/xrootd/xrootd/issues/618#issuecomment-343316950"}}}</script><br>
<hr>
<p align="left">
Use REPLY-ALL to reply to list</p>
<p align="center">To unsubscribe from the XROOTD-DEV list, click the following link:<br>
<a href="https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1" target="_blank">https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1</a>
</p>