Print

Print


Yes, but I won't manage today. Is Monday fine with you?

Lukasz

On Friday, August 31, 2012 07:34:44 AM Brian Bockelman wrote:
> Can we cut a 3.2.3 patch release with these two fixes?
> 
> Brian
> 
> On Aug 31, 2012, at 12:14 AM, "Yang, Wei" <[log in to unmask]> wrote:
> > I tested the second. I didn't get a chance to test the 1st before I lost
> > the window of restarting the cluster. But I have sendfile() turned off
> > and I do get correct results, so it implicitly confirms the 1st one.
> > 
> > regards,
> > Wei Yang  |  [log in to unmask]  |  650-926-3338(O)
> > 
> > On Aug 30, 2012, at 9:29 PM, Wilko Kroeger wrote:
> >> Hello Brian
> >> 
> >> Yes, we also noticed that the detailed monitoring is not working in
> >> v3.2.2. We build a version on top of v3.2.2 adding the two commits:
> >> 
> >> commit e0ad3459c89a163e600070a15936b8fd5d26ff35
> >> Author: Andrew Hanushevsky <[log in to unmask]>
> >> Date:   Wed Aug 22 18:56:19 2012 -0700
> >> 
> >>     Make sure read statistics are updated for sendfile() and mmap I/O.
> >> 
> >> commit e51db4bb0178a21bbe87ccf7c9349b079c2d7455
> >> Author: Andrew Hanushevsky <[log in to unmask]>
> >> Date:   Mon Jul 30 16:52:56 2012 -0700
> >> 
> >>     Correct monitor initialization test to start monitor under all
> >>     configs.
> >> 
> >> As far as I can tell the detailed monitoring is now working. Wei might
> >> have done more testing.
> >> 
> >> Cheers,
> >> 
> >>  Wilko
> >> 
> >> On Thu, 30 Aug 2012, Brian Bockelman wrote:
> >>> Hi Andy,
> >>> 
> >>> The core wasn't interesting.  However, I tracked it down to this change
> >>> (line 334 in XrdXrootdConfig.cc):
> >>> 
> >>> if ((!isRedir || (RQList.Next() != 0 && XrdXrootdMonitor::Redirect())))
> >>> 
> >>> became:
> >>> 
> >>> if ((!isRedir || (RQList.Next() != 0)) && XrdXrootdMonitor::Redirect())
> >>> 
> >>> (in 3.2.2).  In master, it is this test:
> >>> 
> >>> if (!isRedir || XrdXrootdMonitor::Redirect())
> >>> 
> >>> Note that XrdXrootdMonitor::Redirect always returns 0 (I suspect the bug
> >>> is this).
> >>> 
> >>> So, basically, I think detailed monitoring is broken in the 3.2.2
> >>> release.  Matevz, take note...
> >>> 
> >>> What's the minimal patch?  I can ask OSG to push this out ASAP.
> >>> 
> >>> Brian
> >>> 
> >>> On Aug 28, 2012, at 9:26 PM, Andrew Hanushevsky <[log in to unmask]> 
wrote:
> >>>> Hi Brian,
> >>>> 
> >>>> Best to get a gcore on this one. Seems like the monitoring did not
> >>>> initialize correctly as it's trying to send to fd 0.
> >>>> 
> >>>> Andy
> >>>> 
> >>>> -----Original Message----- From: Brian Bockelman
> >>>> Sent: Tuesday, August 28, 2012 7:15 PM
> >>>> To: <[log in to unmask]>
> >>>> Subject: Strange detailed monitoring issue
> >>>> 
> >>>> After a power outage locally, Matevz noticed he is not receiving
> >>>> monitoring messages.
> >>>> 
> >>>> Sure enough, from strace:
> >>>> 
> >>>> [pid  1705] sendto(0,
> >>>> "t8\5\270\0\0\0\0\340\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\200\0\0\0\302^v#".
> >>>> .., 1464, 0, {sa_family=AF_UNSPEC,
> >>>> sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = -1 ENOTSOCK (Socket
> >>>> operation on non-socket)
> >>>> 
> >>>> Version info:
> >>>> 
> >>>> [root@red-gridftp3 ~]# rpm -q xrootd-server
> >>>> xrootd-server-3.2.2-1.osg.el5.xu
> >>>> 
> >>>> Log startup is below.  Config file snippet is:
> >>>> 
> >>>> xrootd.monitor all auth flush 30s mbuff 1472 window 5s dest files io
> >>>> info user xrootd.t2.ucsd.edu:9930 xrd.report xrootd.t2.ucsd.edu:9931
> >>>> every 30s all sync
> >>>> 
> >>>> Any ideas?  We are at a loss as to what might be happening.
> >>>> 
> >>>> Brian
> >>>> 
> >>>> 120828 21:07:13 1663 Scalla is starting. . .
> >>>> Copr.  2010 Stanford University, xrd version v3.2.2
> >>>> ++++++ xrootd [log in to unmask] initialization started.
> >>>> Config using configuration file /etc/xrootd/xrootd-clustered.cfg
> >>>> =====> xrd.port 1094
> >>>> =====> xrd.trace conn
> >>>> =====> all.adminpath /var/run/xrootd
> >>>> =====> xrd.report xrootd.t2.ucsd.edu:9931 every 30s all sync
> >>>> Config maximum number of connections restricted to 65536
> >>>> Copr.  2007 Stanford University, xrootd version 2.9.7 build v3.2.2
> >>>> ++++++ xrootd protocol initialization started.
> >>>> =====> all.export / nostage
> >>>> =====> xrootd.trace emsg login stall redirect
> >>>> =====> xrootd.seclib /usr/lib64/libXrdSec.so
> >>>> Config warning: ignoring fslib; libXrdOfs.so is built-in.
> >>>> =====> xrootd.fslib /usr/lib64/libXrdOfs.so
> >>>> =====> all.pidpath /var/run/xrootd
> >>>> =====> xrootd.monitor all auth flush 30s mbuff 1472 window 5s dest
> >>>> files io info user xrootd.t2.ucsd.edu:9930 Config exporting /
> >>>> ++++++ Authentication system initialization started.
> >>>> 120828 21:07:13 1663 secgsi_InitOpts: ***
> >>>> ------------------------------------------------------------ ***
> >>>> 120828 21:07:13 1663 secgsi_InitOpts:  Mode: server
> >>>> 120828 21:07:13 1663 secgsi_InitOpts:  Debug: -1
> >>>> 120828 21:07:13 1663 secgsi_InitOpts:  CA dir:
> >>>> /etc/grid-security/certificates 120828 21:07:13 1663 secgsi_InitOpts: 
> >>>> CA verification level: 1
> >>>> 120828 21:07:13 1663 secgsi_InitOpts:  CRL dir:
> >>>> /etc/grid-security/certificates/ 120828 21:07:13 1663 secgsi_InitOpts:
> >>>>  CRL extension: .r0
> >>>> 120828 21:07:13 1663 secgsi_InitOpts:  CRL check level: 1
> >>>> 120828 21:07:13 1663 secgsi_InitOpts:  CRL refresh time: 86400
> >>>> 120828 21:07:13 1663 secgsi_InitOpts:  Certificate:
> >>>> /etc/grid-security/xrd/xrdcert.pem 120828 21:07:13 1663
> >>>> secgsi_InitOpts:  Key: /etc/grid-security/xrd/xrdkey.pem 120828
> >>>> 21:07:13 1663 secgsi_InitOpts:  Proxy delegation option: 0
> >>>> 120828 21:07:13 1663 secgsi_InitOpts:  GRIDmap file:
> >>>> /etc/grid-security/grid-mapfile 120828 21:07:13 1663 secgsi_InitOpts: 
> >>>> GRIDmap option: 10
> >>>> 120828 21:07:13 1663 secgsi_InitOpts:  GRIDmap cache entries expiration
> >>>> (secs): 0 120828 21:07:13 1663 secgsi_InitOpts:  Authorization
> >>>> function: libXrdLcmaps.so 120828 21:07:13 1663 secgsi_InitOpts: 
> >>>> Authorization function parms:
> >>>> --osg,--lcmapscfg,/etc/xrootd/lcmaps.cfg,--loglevel,0|useglobals
> >>>> 120828 21:07:13 1663 secgsi_InitOpts:  Authorization cache entries
> >>>> expiration (secs): -1 120828 21:07:13 1663 secgsi_InitOpts:  Client
> >>>> proxy availability in XrdSecEntity.endorsement: 0 120828 21:07:13 1663
> >>>> secgsi_InitOpts:  VOMS option: 1
> >>>> 120828 21:07:13 1663 secgsi_InitOpts:  MonInfo option: 0
> >>>> 120828 21:07:13 1663 secgsi_InitOpts:  Crypto modules: ssl
> >>>> 120828 21:07:13 1663 secgsi_InitOpts:  Ciphers:
> >>>> aes-128-cbc:bf-cbc:des-ede3-cbc 120828 21:07:13 1663 secgsi_InitOpts: 
> >>>> MDigests: sha1:md5
> >>>> 120828 21:07:13 1663 secgsi_InitOpts: ***
> >>>> ------------------------------------------------------------ ***
> >>>> 120828 21:07:13 1663 secgsi_LoadAuthzFun: using 'XrdSecgsiAuthzFun()'
> >>>> from libXrdLcmaps.so =====> sec.protocol /usr/lib64 gsi
> >>>> -certdir:/etc/grid-security/certificates
> >>>> -cert:/etc/grid-security/xrd/xrdcert.pem
> >>>> -key:/etc/grid-security/xrd/xrdkey.pem -crl:1
> >>>> -authzfun:libXrdLcmaps.so -authzfunparms:--osg,--lcmapscfg,/et Config
> >>>> 1 authentication directives processed in
> >>>> /etc/xrootd/xrootd-clustered.cfg ------ Authentication system
> >>>> initialization completed.
> >>>> ++++++ File system initialization started.
> >>>> =====> all.role server
> >>>> Config warning: ignoring invalid trace option 'none'.
> >>>> =====> ofs.trace none
> >>>> =====> ofs.authorize
> >>>> =====> ofs.osslib /usr/lib64/libXrdHdfs.so
> >>>> ++++++ Authorization system initialization started.
> >>>> 120828 21:07:13 1663 acc_Config: Authorization system using
> >>>> configuration in /etc/xrootd/xrootd-clustered.cfg =====> acc.authdb
> >>>> /etc/xrootd/Authfile
> >>>> =====> acc.audit deny grant
> >>>> Config 2 authorization directives processed in
> >>>> /etc/xrootd/xrootd-clustered.cfg Config 1 auth entries processed in
> >>>> /etc/xrootd/Authfile
> >>>> ------ Authorization system initialization completed.
> >>>> Copr. 2009, Brian Bockelman, Hdfs Version
> >>>> 120828 21:07:13 1663 hdfs_Config: Copr. 2009, Brian Bockelman, Hdfs
> >>>> Version
> >>>> 120828 21:07:13 1663 hdfs_Config: Configuring HDFS.
> >>>> =====> oss.namelib /usr/lib64/libXrdCmsTfc.so
> >>>> file:/etc/xrootd/storage.xml?protocol=hadoop Copr. 2009 University of
> >>>> Nebraska-Lincoln TFC plugin v 1.0
> >>>> Params: file:/etc/xrootd/storage.xml?protocol=hadoop
> >>>> Xerces-c has been initialized.
> >>>> Connecting to the catalog file:/etc/xrootd/storage.xml?protocol=hadoop
> >>>> Using catalog file /etc/xrootd/storage.xml
> >>>> ------ HDFS storage system initialization completed.
> >>>> 120828 21:07:13 1663 hdfs_HDFS storage system initialization.:
> >>>> completed.
> >>>> ++++++ Configuring server role. . .
> >>>> =====> all.manager srm.unl.edu:1213
> >>>> =====> cms.trace all
> >>>> =====> all.adminpath /var/run/xrootd
> >>>> 120828 21:07:13 1663 Configure Global System Identification: anon-s
> >>>> 1213srm.unl.edu>>>> 
> >>>> Config effective /etc/xrootd/xrootd-clustered.cfg ofs configuration:
> >>>>    ofs.role server
> >>>>    ofs.authorize
> >>>>    ofs.maxdelay   60
> >>>>    ofs.osslib /usr/lib64/libXrdHdfs.so
> >>>>    ofs.persist    manual hold 600 logdir /var/run/xrootd/.ofs/posc.log
> >>>>    ofs.trace      0
> >>>> 
> >>>> ------ File system server initialization completed.
> >>>> Config warning: 'xrootd.prepare logdir' not specified; prepare tracking
> >>>> disabled. 120828 21:07:13 1675 cms_Finder: Connected to cmsd via
> >>>> /var/run/xrootd/.olb/olbd.admin ------ xrootd protocol initialization
> >>>> completed.
> >>>> ------ xrootd [log in to unmask]:1094 initialization completed.
> >>>> 
> >>>> 
> >>>> #######################################################################
> >>>> #
> >>>> Use REPLY-ALL to reply to list
> >>>> 
> >>>> To unsubscribe from the XROOTD-DEV list, click the following link:
> >>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
> >>> 
> >>> ########################################################################
> >>> Use REPLY-ALL to reply to list
> >>> 
> >>> To unsubscribe from the XROOTD-DEV list, click the following link:
> >>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
> >> 
> >> ########################################################################
> >> Use REPLY-ALL to reply to list
> >> 
> >> To unsubscribe from the XROOTD-DEV list, click the following link:
> >> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
> > 
> > ########################################################################
> > Use REPLY-ALL to reply to list
> > 
> > To unsubscribe from the XROOTD-DEV list, click the following link:
> > https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
> > <winmail.dat>
> 
> ########################################################################
> Use REPLY-ALL to reply to list
> 
> To unsubscribe from the XROOTD-DEV list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1

########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1