Hi Matevz,
Thanks a lot for looking into this.
- The crash seems to happen always when i make a request
- Currently prefetching is disabled
- Yes, i think it is direct proxy mode
- stack trace is attached
A similar setup seems to work for Ilija without issues with the xcaches
using slate - i tried to mimic that setup closely. Running xrootd from
this container image:
https://gitlab.physik.uni-muenchen.de/Nikolai.Hartmann/xcache-singularity-lrz/-/blob/51d2da52829eb6d8ea377539884f337208141aca/xcache.singularity.def
using this config
https://gitlab.physik.uni-muenchen.de/Nikolai.Hartmann/xcache-singularity-lrz/-/blob/51d2da52829eb6d8ea377539884f337208141aca/etc/xrootd/xcache.cfg
Cheers,
Nikolai
On 7/7/20 1:38 AM, Matevz Tadel wrote:
> Hi Nikolai,
>
> I tried to reproduce it with current master in nearly all ways,
> with/without prefetching and with direct/forwarding mode. Also, with std
> malloc and tcmalloc. No luck :(
>
> Backtrace or core would help a lot at this point.
>
> Cheers,
> Matevz
>
> On 2020-07-03 00:54, Nikolai Hartmann wrote:
>> Hi,
>>
>> I'm trying to upgrade to xrootd5 rc4 for our xcache server to mitigate a
>> problem with dCache.
>>
>> Now when i try to read a file through xcache it crashes with "Attempt to
>> free invalid pointer". I attached the corresponding part of the log.
>> Any ideas?
>>
>> Thanks,
>> Nikolai
>>
>> ########################################################################
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the XROOTD-L list, click the following link:
>> https://urldefense.com/v3/__https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1__;!!Mih3wA!Xzk53aW-mEg2pavzme9Hd49MPmno8frpbkh2YetRsquNyAt5jiVsDB91pTNUHA$
>>
>>
>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
#0 0x00007ffff6266387 in __GI_raise (sig=sig@entry=6)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:55
#1 0x00007ffff6267a78 in __GI_abort () at abort.c:90
#2 0x00007ffff7a08a66 in tcmalloc::Log(tcmalloc::LogMode, char const*, int, tcmalloc::LogItem, tcmalloc::LogItem, tcmalloc::LogItem, tcmalloc::LogItem) () from /usr/lib64/libtcmalloc.so
#3 0x00007ffff79fd644 in (anonymous namespace)::InvalidFree(void*) ()
from /usr/lib64/libtcmalloc.so
#4 0x00007fffefb5369b in XrdPfc::File::free_block (this=0x23bd680, b=0x23c46c0)
at /usr/src/debug/xrootd/xrootd/src/XrdPfc/XrdPfcFile.cc:1061
#5 0x00007fffefb53754 in XrdPfc::File::dec_ref_count (this=this@entry=0x23bd680,
b=<optimized out>) at /usr/src/debug/xrootd/xrootd/src/XrdPfc/XrdPfcFile.cc:1044
#6 0x00007fffefb577e3 in XrdPfc::File::Read (this=0x23bd680, io=io@entry=0xe4ec40,
iUserBuff=iUserBuff@entry=0x2834000 "", iUserOff=iUserOff@entry=0,
iUserSize=iUserSize@entry=262144)
at /usr/src/debug/xrootd/xrootd/src/XrdPfc/XrdPfcFile.cc:879
#7 0x00007fffefb5f1eb in XrdPfc::IOEntireFile::Read (this=0xe4ec40, buff=0x2834000 "",
off=0, size=262144) at /usr/src/debug/xrootd/xrootd/src/XrdPfc/XrdPfcIOEntireFile.cc:167
#8 0x00007ffff748843e in XrdOucCacheIO::Read (this=<optimized out>, iocb=...,
buff=<optimized out>, offs=<optimized out>, rlen=<optimized out>)
at /usr/src/debug/xrootd/xrootd/src/XrdOuc/XrdOucCache.hh:330
#9 0x00007ffff0c663c0 in XrdPssFile::Read (this=0x23c4630, aiop=0x24a98c8)
at /usr/src/debug/xrootd/xrootd/src/XrdPss/XrdPssAio.cc:83
#10 0x00007ffff7766fa3 in XrdOfsFile::read (this=0x25f8000, aiop=0x24a98c8)
at /usr/src/debug/xrootd/xrootd/src/XrdOfs/XrdOfs.cc:1031
#11 0x00007ffff77409a0 in XrdXrootdAioReq::Read (this=0x24a8ee8)
at /usr/src/debug/xrootd/xrootd/src/XrdXrootd/XrdXrootdAio.cc:395
#12 0x00007ffff7764440 in XrdXrootdProtocol::aio_Read (this=this@entry=0xe1f400)
at /usr/src/debug/xrootd/xrootd/src/XrdXrootd/XrdXrootdXeqAio.cc:107
#13 0x00007ffff775aadb in XrdXrootdProtocol::do_ReadAll (this=this@entry=0xe1f400,
asyncOK=asyncOK@entry=1)
at /usr/src/debug/xrootd/xrootd/src/XrdXrootd/XrdXrootdXeq.cc:2315
#14 0x00007ffff775acc9 in XrdXrootdProtocol::do_Read (this=0xe1f400)
at /usr/src/debug/xrootd/xrootd/src/XrdXrootd/XrdXrootdXeq.cc:2267
#15 0x00007ffff74b92d9 in XrdLinkXeq::DoIt (this=0x2535e00)
at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdLinkXeq.cc:308
#16 0x00007ffff74b5ca9 in XrdLink::setProtocol (this=0x2535e00, pp=<optimized out>,
runit=<optimized out>, push=<optimized out>)
at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdLink.cc:421
#17 0x00007ffff74bc4cf in XrdScheduler::Run (this=0x614c20 <XrdGlobal::Sched>)
at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdScheduler.cc:382
#18 0x00007ffff74bc619 in XrdStartWorking (carg=<optimized out>)
at /usr/src/debug/xrootd/xrootd/src/Xrd/XrdScheduler.cc:88
#19 0x00007ffff747f917 in XrdSysThread_Xeq (myargs=0xe1b2c0)
at /usr/src/debug/xrootd/xrootd/src/XrdSys/XrdSysPthread.cc:86
#20 0x00007ffff702cea5 in start_thread (arg=0x7ffff2594700) at pthread_create.c:307
#21 0x00007ffff632e8dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
|