Okay, sounds good!
Yes, we'll definitely upgrade whenever that would be advisable.
Thanks,
Horst
Andrew Hanushevsky <[log in to unmask]> wrote:
> Hi Horst,
>
> Other thna those core files nothing that would interest you. Once we see
> what the problem is maybe an upgrade would be a good thing.
>
> Andy
>
>
> On Sat, 21 Mar 2020, Horst Severini wrote:
>
> > Hi Andy,
> >
> > thanks. I just checked, and that's already installed on se1.
> > We probably did that a while ago when we were looking at other issues.
> >
> > We're running 4.11.1 everywhere on those storage nodes. Should we upgrade
> > to 4.11.2, or is there nothing in the .2 update that's applicable for us
> > right now?
> >
> > Thanks,
> >
> > Horst
> >
> > Andrew Hanushevsky <[log in to unmask]> wrote:
> >
> >> Hi Horst,
> >>
> >> Please make sure to install the debug RPM so that we can get actual
> >> statement numbers. It's called xrootd-debuginfo-4.11.2-1.el7.x86_64.rpm
> >> (assuming you are running Cent7). They should be readily available from
> >> OSG (though we have them too).
> >>
> >> Andy
> >>
> >> On Sat, 21 Mar 2020, Horst Severini wrote:
> >>
> >>> Thanks Andy,
> >>>
> >>> will do!
> >>>
> >>> Horst
> >>>
> >>> Andrew Hanushevsky <[log in to unmask]> wrote:
> >>>
> >>>> Hi Horst,
> >>>>
> >>>> Indeed, one should never see a core dump and if one does appear we
> >>>> definitely want to know about it. When you do see one, here is the first
> >>>> dump of information that would be helpful before we start digging deeper:
> >>>>
> >>>> gdb >executable> <corefile>
> >>>> where
> >>>> quit
> >>>>
> >>>> Cut and past the output into a mail file or posting. We may ask for a
> >>>> detailed traceback of every thread. I think I'll put that process on the
> >>>> xroot web page. For now, at least we will know where it went bonkers.
> >>>>
> >>>> Andy
> >>>>
> >>>>
> >>>> On Sat, 21 Mar 2020, Horst Severini wrote:
> >>>>
> >>>>> Hi Wei,
> >>>>>
> >>>>> yes, we have enough space for a few core dumps in /var/. It's just that there
> >>>>> were 5 or 6 in the last week or two, and that filled /var/ up completely.
> >>>>> I'll keep a closer eye on it for now.
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>> Horst
> >>>>>
> >>>>> On 3/21/20 4:39 PM, Yang, Wei wrote:
> >>>>>> Also make sure you have enough space to hold a core dump. It can something
> >>>>>> be 10GB+
> >>>>>>
> >>>>>> --
> >>>>>> Wei Yang [log in to unmask] | 650-926-3338(O)
> >>>>>>
> >>>>>> ???-----Original Message-----
> >>>>>> From:<[log in to unmask]> on behalf of Horst
> >>>>>> Severini<[log in to unmask]>
> >>>>>> Date: Saturday, March 21, 2020 at 1:30 PM
> >>>>>> To: xrootd-dev<[log in to unmask]>,"[log in to unmask]"
> >>>>>> <[log in to unmask]>
> >>>>>> Subject: Re: XrootD smoke test report for 2020-03-21 10:01:46 GMT
> >>>>>>
> >>>>>> Thanks Wei,
> >>>>>> I'll send you the next one I get!:)
> >>>>>> Cheers,
> >>>>>> Horst
> >>>>>> On 3/21/20 2:55 PM, Yang, Wei wrote:
> >>>>>> > Indeed a core dump is usually the thing we need.
> >>>>>> >
> >>>>>> > --
> >>>>>> > Wei Yang [log in to unmask] | 650-926-3338(O)
> >>>>>> >
> >>>>>> > On 3/21/20, 12:19 PM,"[log in to unmask] on behalf of
> >>>>>> Horst Severini" <[log in to unmask] on behalf of [log in to unmask]>
> >>>>>> wrote:
> >>>>>> >
> >>>>>> > We're running 4.11.1 here at OU.
> >>>>>> >
> >>>>>> > Cheers,
> >>>>>> >
> >>>>>> > Horst
> >>>>>> >
> >>>>>> > Albert Rossi<[log in to unmask]> wrote:
> >>>>>> >
> >>>>>> > > Hi Horst,
> >>>>>> > >
> >>>>>> > > actually, if you notice, all endpoints failed on today's
> >>>>>> test. So it was not just OU.
> >>>>>> > >
> >>>>>> > > The Stanford developers may want you to run a few commands
> >>>>>> over the core file from gdb once you have it in hand.
> >>>>>> > >
> >>>>>> > > What version of xrootd are you running, just out of
> >>>>>> curiosity? Is it bleeding-edge, or a stable release?
> >>>>>> > >
> >>>>>> > > Cheers, Al
> >>>>>> > >
> >>>>>> > > ________________________________________________
> >>>>>> > > Albert L. Rossi
> >>>>>> > > Application Developer & Systems Analyst III
> >>>>>> > > Scientific Computing Division, Data Movement Development
> >>>>>> > > FCC 229A
> >>>>>> > > Mail Station 369 (FCC 2W)
> >>>>>> > > Fermi National Accelerator Laboratory
> >>>>>> > > Batavia, IL 60510
> >>>>>> > > (630) 840-3023
> >>>>>> > > ________________________________
> >>>>>> > > From: Horst Severini<[log in to unmask]>
> >>>>>> > > Sent: Saturday, March 21, 2020 1:18 PM
> >>>>>> > >To:[log in to unmask]
> >>>>>> <[log in to unmask]>;[log in to unmask]<[log in to unmask]>;
> >>>>>> Albert Rossi<[log in to unmask]>
> >>>>>> > > Subject: Re: XrootD smoke test report for 2020-03-21 10:01:46
> >>>>>> GMT
> >>>>>> > >
> >>>>>> > > Hi Al,
> >>>>>> > >
> >>>>>> > > thanks, good idea. I'll save the next core file.
> >>>>>> > >
> >>>>>> > > I'm pretty sure the authentication failures simply came
> >>>>>> because
> >>>>>> > > that partition was full and no new proxies or what not could
> >>>>>> be
> >>>>>> > > created, so I wouldn't worry about that.
> >>>>>> > >
> >>>>>> > > Thanks,
> >>>>>> > >
> >>>>>> > > Horst
> >>>>>> > >
> >>>>>> > > Albert Rossi<[log in to unmask]> wrote:
> >>>>>> > >
> >>>>>> > >> Hi Horst,
> >>>>>> > >>
> >>>>>> > >> I would definitely report
> >>>>>> [log in to unmask]
> >>>>>> > >>
> >>>>>> > >> As for why the massive authentication failure, I've seen
> >>>>>> this before, it might have to do with CA cert issues.
> >>>>>> > >>
> >>>>>> > >> Cheers, Al
> >>>>>> > >>
> >>>>>> > >> ________________________________________________
> >>>>>> > >> Albert L. Rossi
> >>>>>> > >> Application Developer & Systems Analyst III
> >>>>>> > >> Scientific Computing Division, Data Movement Development
> >>>>>> > >> FCC 229A
> >>>>>> > >> Mail Station 369 (FCC 2W)
> >>>>>> > >> Fermi National Accelerator Laboratory
> >>>>>> > >> Batavia, IL 60510
> >>>>>> > >> (630) 840-3023
> >>>>>> > >> ________________________________
> >>>>>> > >> From: Horst Severini<[log in to unmask]>
> >>>>>> > >> Sent: Saturday, March 21, 2020 11:19 AM
> >>>>>> > >>To:[log in to unmask] <[log in to unmask]>
> >>>>>> > >> Subject: Re: XrootD smoke test report for 2020-03-21
> >>>>>> 10:01:46 GMT
> >>>>>> > >>
> >>>>>> > >> Hi all,
> >>>>>> > >>
> >>>>>> > >> our/var/ partition had filled up because of too many
> >>>>>> xrootd core dumps.
> >>>>>> > >> I cleared those up and restarted xrootd, and things look
> >>>>>> better again now.
> >>>>>> > >>
> >>>>>> > >> Not sure why we keep getting core dumps, though.
> >>>>>> > >>
> >>>>>> > >> Cheers,
> >>>>>> > >>
> >>>>>> > >> Horst
> >>>>>> > >>
> >>>>>> > >> On 3/21/20 5:01 AM,[log in to unmask] wrote:
> >>>>>> > >>> XROOTD SMOKE TEST SUMMARY
> >>>>>> > >>> 2020-03-21 10:01:46 GMT
> >>>>>> > >>>
> >>>>>> > >>> Client: bogus6.fnal.gov
> >>>>>> > >>>
> >>>>>> > >>> XrootD version: v4.11.2
> >>>>>> > >>>
> >>>>>> > >>> Reference server: CERN-TRUNK
> >>>>>> > >>>
> >>>>>> > >>> Credential delegation: ON
> >>>>>> > >>>
> >>>>>> > >>> Checksum: -C adler32
> >>>>>> > >>>
> >>>>>> > >>> Total number of round-trip tests: 21
> >>>>>> > >>>
> >>>>>> > >>> --------------------------------SOUND
> >>>>>> ENDPOINTS---------------------------------
> >>>>>> > >>>
> >>>>>> > >>> SCORE ENDPT TYPE UP SRC
> >>>>>> DST DN
> >>>>>> > >>>
> >>>>>> --------------------------------------------------------------------------------
> >>>>>> > >>>
> >>>>>> > >>> -----------------------------PROBLEMATIC
> >>>>>> ENDPOINTS------------------------------
> >>>>>> > >>>
> >>>>>> > >>> SCORE ENDPT TYPE UP SRC
> >>>>>> DST DN
> >>>>>> > >>>
> >>>>>> --------------------------------------------------------------------------------
> >>>>>> > >>> 19 BRUSSELS dCache F -
> >>>>>> F F 0/4
> >>>>>> > >>> 19 CERN-EOS EOS F -
> >>>>>> F F 0/4
> >>>>>> > >>> 19 CERN-TRUNK DPM - F
> >>>>>> - F 0/2
> >>>>>> > >>> 19 DESY-PROM dCache F -
> >>>>>> F F 0/4
> >>>>>> > >>> 19 FNAL dCache F -
> >>>>>> F F 0/4
> >>>>>> > >>> 19 IN2P3-DOMA xrootd F -
> >>>>>> F F 0/4
> >>>>>> > >>> 19 PRAGUE DPM F -
> >>>>>> F F 0/4
> >>>>>> > >>> 19 RAL-CEPH CEPH F -
> >>>>>> F F 0/4
> >>>>>> > >>> 19 RAL-LCG2 Echo F -
> >>>>>> F F 0/4
> >>>>>> > >>> 19 SLAC XrootD F -
> >>>>>> F F 0/4
> >>>>>> > >>> 19 TRIUMF dCache F -
> >>>>>> F F 0/4
> >>>>>> > >>> 19 UKI-BRUNEL DPM F -
> >>>>>> F F 0/4
> >>>>>> > >>> 19 UKI-LANC DPM F -
> >>>>>> F F 0/4
> >>>>>> > >>> 19 UKI-MAN1 DPM F -
> >>>>>> F F 0/4
> >>>>>> > >>> 19 UKI-MAN2 DPM F -
> >>>>>> F F 0/4
> >>>>>> > >>> 19 UNI-BONN CephFS F -
> >>>>>> F F 0/4
> >>>>>> > >>> 18 OU XrootD F -
> >>>>>> F F 0/4
> >>>>>> > >>> 13 BNL dCache F -
> >>>>>> F F 0/4
> >>>>>> > >>> 7 AGLT2 dCache F -
> >>>>>> F F 0/4
> >>>>>> > >>> 0 CALTECH HDFS F -
> >>>>>> F F 0/4
> >>>>>> > >>> 0 TRIUMF-PROD dCache F -
> >>>>>> F F 0/4
> >>>>>
> >>>
> >
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
|