 |
 |
Hi Horst, Please make sure to install the debug RPM so that we can get actual statement numbers. It's called xrootd-debuginfo-4.11.2-1.el7.x86_64.rpm (assuming you are running Cent7). They should be readily available from OSG (though we have them too). Andy On Sat, 21 Mar 2020, Horst Severini wrote: > Thanks Andy, > > will do! > > Horst > > Andrew Hanushevsky <[log in to unmask]> wrote: > >> Hi Horst, >> >> Indeed, one should never see a core dump and if one does appear we >> definitely want to know about it. When you do see one, here is the first >> dump of information that would be helpful before we start digging deeper: >> >> gdb >executable> <corefile> >> where >> quit >> >> Cut and past the output into a mail file or posting. We may ask for a >> detailed traceback of every thread. I think I'll put that process on the >> xroot web page. For now, at least we will know where it went bonkers. >> >> Andy >> >> >> On Sat, 21 Mar 2020, Horst Severini wrote: >> >>> Hi Wei, >>> >>> yes, we have enough space for a few core dumps in /var/. It's just that there >>> were 5 or 6 in the last week or two, and that filled /var/ up completely. >>> I'll keep a closer eye on it for now. >>> >>> Thanks, >>> >>> Horst >>> >>> On 3/21/20 4:39 PM, Yang, Wei wrote: >>>> Also make sure you have enough space to hold a core dump. It can something >>>> be 10GB+ >>>> >>>> -- >>>> Wei Yang [log in to unmask] | 650-926-3338(O) >>>> >>>> ???-----Original Message----- >>>> From:<[log in to unmask]> on behalf of Horst >>>> Severini<[log in to unmask]> >>>> Date: Saturday, March 21, 2020 at 1:30 PM >>>> To: xrootd-dev<[log in to unmask]>,"[log in to unmask]" >>>> <[log in to unmask]> >>>> Subject: Re: XrootD smoke test report for 2020-03-21 10:01:46 GMT >>>> >>>> Thanks Wei, >>>> I'll send you the next one I get!:) >>>> Cheers, >>>> Horst >>>> On 3/21/20 2:55 PM, Yang, Wei wrote: >>>> > Indeed a core dump is usually the thing we need. >>>> > >>>> > -- >>>> > Wei Yang [log in to unmask] | 650-926-3338(O) >>>> > >>>> > On 3/21/20, 12:19 PM,"[log in to unmask] on behalf of >>>> Horst Severini" <[log in to unmask] on behalf of [log in to unmask]> >>>> wrote: >>>> > >>>> > We're running 4.11.1 here at OU. >>>> > >>>> > Cheers, >>>> > >>>> > Horst >>>> > >>>> > Albert Rossi<[log in to unmask]> wrote: >>>> > >>>> > > Hi Horst, >>>> > > >>>> > > actually, if you notice, all endpoints failed on today's >>>> test. So it was not just OU. >>>> > > >>>> > > The Stanford developers may want you to run a few commands >>>> over the core file from gdb once you have it in hand. >>>> > > >>>> > > What version of xrootd are you running, just out of >>>> curiosity? Is it bleeding-edge, or a stable release? >>>> > > >>>> > > Cheers, Al >>>> > > >>>> > > ________________________________________________ >>>> > > Albert L. Rossi >>>> > > Application Developer & Systems Analyst III >>>> > > Scientific Computing Division, Data Movement Development >>>> > > FCC 229A >>>> > > Mail Station 369 (FCC 2W) >>>> > > Fermi National Accelerator Laboratory >>>> > > Batavia, IL 60510 >>>> > > (630) 840-3023 >>>> > > ________________________________ >>>> > > From: Horst Severini<[log in to unmask]> >>>> > > Sent: Saturday, March 21, 2020 1:18 PM >>>> > >To:[log in to unmask] >>>> <[log in to unmask]>;[log in to unmask]<[log in to unmask]>; >>>> Albert Rossi<[log in to unmask]> >>>> > > Subject: Re: XrootD smoke test report for 2020-03-21 10:01:46 >>>> GMT >>>> > > >>>> > > Hi Al, >>>> > > >>>> > > thanks, good idea. I'll save the next core file. >>>> > > >>>> > > I'm pretty sure the authentication failures simply came >>>> because >>>> > > that partition was full and no new proxies or what not could >>>> be >>>> > > created, so I wouldn't worry about that. >>>> > > >>>> > > Thanks, >>>> > > >>>> > > Horst >>>> > > >>>> > > Albert Rossi<[log in to unmask]> wrote: >>>> > > >>>> > >> Hi Horst, >>>> > >> >>>> > >> I would definitely report >>>> [log in to unmask] >>>> > >> >>>> > >> As for why the massive authentication failure, I've seen >>>> this before, it might have to do with CA cert issues. >>>> > >> >>>> > >> Cheers, Al >>>> > >> >>>> > >> ________________________________________________ >>>> > >> Albert L. Rossi >>>> > >> Application Developer & Systems Analyst III >>>> > >> Scientific Computing Division, Data Movement Development >>>> > >> FCC 229A >>>> > >> Mail Station 369 (FCC 2W) >>>> > >> Fermi National Accelerator Laboratory >>>> > >> Batavia, IL 60510 >>>> > >> (630) 840-3023 >>>> > >> ________________________________ >>>> > >> From: Horst Severini<[log in to unmask]> >>>> > >> Sent: Saturday, March 21, 2020 11:19 AM >>>> > >>To:[log in to unmask] <[log in to unmask]> >>>> > >> Subject: Re: XrootD smoke test report for 2020-03-21 >>>> 10:01:46 GMT >>>> > >> >>>> > >> Hi all, >>>> > >> >>>> > >> our/var/ partition had filled up because of too many >>>> xrootd core dumps. >>>> > >> I cleared those up and restarted xrootd, and things look >>>> better again now. >>>> > >> >>>> > >> Not sure why we keep getting core dumps, though. >>>> > >> >>>> > >> Cheers, >>>> > >> >>>> > >> Horst >>>> > >> >>>> > >> On 3/21/20 5:01 AM,[log in to unmask] wrote: >>>> > >>> XROOTD SMOKE TEST SUMMARY >>>> > >>> 2020-03-21 10:01:46 GMT >>>> > >>> >>>> > >>> Client: bogus6.fnal.gov >>>> > >>> >>>> > >>> XrootD version: v4.11.2 >>>> > >>> >>>> > >>> Reference server: CERN-TRUNK >>>> > >>> >>>> > >>> Credential delegation: ON >>>> > >>> >>>> > >>> Checksum: -C adler32 >>>> > >>> >>>> > >>> Total number of round-trip tests: 21 >>>> > >>> >>>> > >>> --------------------------------SOUND >>>> ENDPOINTS--------------------------------- >>>> > >>> >>>> > >>> SCORE ENDPT TYPE UP SRC >>>> DST DN >>>> > >>> >>>> -------------------------------------------------------------------------------- >>>> > >>> >>>> > >>> -----------------------------PROBLEMATIC >>>> ENDPOINTS------------------------------ >>>> > >>> >>>> > >>> SCORE ENDPT TYPE UP SRC >>>> DST DN >>>> > >>> >>>> -------------------------------------------------------------------------------- >>>> > >>> 19 BRUSSELS dCache F - >>>> F F 0/4 >>>> > >>> 19 CERN-EOS EOS F - >>>> F F 0/4 >>>> > >>> 19 CERN-TRUNK DPM - F >>>> - F 0/2 >>>> > >>> 19 DESY-PROM dCache F - >>>> F F 0/4 >>>> > >>> 19 FNAL dCache F - >>>> F F 0/4 >>>> > >>> 19 IN2P3-DOMA xrootd F - >>>> F F 0/4 >>>> > >>> 19 PRAGUE DPM F - >>>> F F 0/4 >>>> > >>> 19 RAL-CEPH CEPH F - >>>> F F 0/4 >>>> > >>> 19 RAL-LCG2 Echo F - >>>> F F 0/4 >>>> > >>> 19 SLAC XrootD F - >>>> F F 0/4 >>>> > >>> 19 TRIUMF dCache F - >>>> F F 0/4 >>>> > >>> 19 UKI-BRUNEL DPM F - >>>> F F 0/4 >>>> > >>> 19 UKI-LANC DPM F - >>>> F F 0/4 >>>> > >>> 19 UKI-MAN1 DPM F - >>>> F F 0/4 >>>> > >>> 19 UKI-MAN2 DPM F - >>>> F F 0/4 >>>> > >>> 19 UNI-BONN CephFS F - >>>> F F 0/4 >>>> > >>> 18 OU XrootD F - >>>> F F 0/4 >>>> > >>> 13 BNL dCache F - >>>> F F 0/4 >>>> > >>> 7 AGLT2 dCache F - >>>> F F 0/4 >>>> > >>> 0 CALTECH HDFS F - >>>> F F 0/4 >>>> > >>> 0 TRIUMF-PROD dCache F - >>>> F F 0/4 >>> > ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1