Hi Andrew, That problem existed in the early 4.x releases. The reason is that unlimited set the nproc value to -1 and xrootd din't know how to handle it. It got fixed in 4.1.1 though I'd sugggest going to 4.2.3. Andy On Wed, 16 Sep 2015, Andrew Melo wrote: > It appears that if the xrootd daemon is started with the nprocess limit set to UNLIMITED, it gets upside down: > > ``` > Config maximum number of threads restricted to -1 > <snip> > 150916 13:04:40 12143 XrdScheduler: Thread limit has been reached! > ``` > (full log at bottom) > (shoutout to @bbockelm who noticed what I had just skimmed right past. :raising_hand:) > > The behavior changed somehow between 3.x and 4.2.2 - the same box worked fine until xrootd was upgraded. > > > Copr. 2004-2012 Stanford University, xrd version v4.2.2 > ++++++ cmsd [log in to unmask] initialization started. > Config using configuration file /etc/xrootd/xrootd-lfs.cfg > =====> xrd.port 1094 > =====> xrd.trace conn > =====> all.adminpath /var/run/xrootd > Config maximum number of connections restricted to 65536 > Config maximum number of threads restricted to -1 > 150916 13:04:40 12143 XrdConfig: sendfile enabled. > 150916 13:04:40 12145 XrdXeq: Buffer Manager reshaper thread started > 150916 13:04:40 12143 XrdSched: scheduling underused thread monitor in 780 seconds > 150916 13:04:40 12146 XrdXeq: Time scheduler thread started > 150916 13:04:40 12143 XrdScheduler: Thread limit has been reached! > 150916 13:04:40 12143 XrdSched: Starting with 0 workers > 150916 13:04:40 12143 XrdLink: Allocating 8 link objects at a time > 150916 13:04:40 12143 XrdPoll: Starting poller 0 > 150916 13:04:40 12147 XrdXeq: Poller thread started > 150916 13:04:40 12143 XrdPoll: Starting poller 1 > 150916 13:04:40 12148 XrdXeq: Poller thread started > 150916 13:04:40 12143 XrdPoll: Starting poller 2 > 150916 13:04:40 12149 XrdXeq: Poller thread started > 150916 13:04:40 12143 XrdProtocol: getting port from protocol cmsd > Copr. 2007 Stanford University/SLAC cmsd. > ++++++ [log in to unmask] phase 1 initialization started. > =====> all.role server > =====> all.manager se1.accre.vanderbilt.edu:1214 > Config cms-xrd-transit.cern.ch -> all.manager vocms027.cern.ch > =====> all.manager cms-xrd-transit.cern.ch+ 1213 > =====> oss.namelib /usr/lib64/libXrdCmsTfc.so file:/etc/xrootd/storage-lfs.xml?protocol=direct > =====> all.export / nostage > =====> cms.trace all debug > =====> all.adminpath /var/run/xrootd > =====> all.pidpath /var/run/xrootd > The following paths are available to the redirector: > w / > > ``` > ------ [log in to unmask] phase 1 server initialization completed. > 150916 13:04:40 12143 XrdConfig: LCL port 55165 wsz=10000000 (10000000) > 150916 13:04:40 12143 XrdProtocol: getting protocol object cmsd > ++++++ [log in to unmask] phase 2 server initialization started. > 150916 13:04:40 12143 Configure2 Global System Identification: lfs-s 1214se1.accre.vanderbilt.edu1213vocms027.cern.ch > ++++++ Storage system initialization started. > =====> oss.namelib /usr/lib64/libXrdCmsTfc.so file:/etc/xrootd/storage-lfs.xml?protocol=direct > =====> all.export / nostage > Plugin No such file or directory loading namelib /usr/lib64/libXrdCmsTfc-4.so > Config Falling back to using /usr/lib64/libXrdCmsTfc.so > Plugin loaded unversioned XrdOucgetName2Name from namelib /usr/lib64/libXrdCmsTfc.so > Copr. 2009 University of Nebraska-Lincoln TFC plugin v 1.0 > Params: file:/etc/xrootd/storage-lfs.xml?protocol=direct > Xerces-c has been initialized. > Connecting to the catalog file:/etc/xrootd/storage-lfs.xml?protocol=direct > Using catalog file /etc/xrootd/storage-lfs.xml > ++++++ Configuring standalone mode . . . > 150916 13:04:40 12143 oss_AioInit: started AIO read signal thread; tid=889157376 > 150916 13:04:40 12143 oss_AioInit: started AIO write signal thread; tid=888104704 > No lfn2pfn mapping for / > No lfn2pfn mapping for / > No lfn2pfn mapping for / > Config effective /etc/xrootd/xrootd-lfs.cfg oss configuration: > oss.alloc 0 0 0 > oss.cachescan 600 > oss.fdlimit 32768 65536 > oss.maxsize 0 > oss.namelib /usr/lib64/libXrdCmsTfc.so > oss.trace fff > oss.xfr 1 deny 10800 keep 1200 > oss.memfile off max 33751543808 > oss.defaults r/w nocheck nodread nomig norcreate nopurge nostage xattr > oss.path / r/w nocheck nodread nomig norcreate nopurge nostage xattr > ------ Storage system initialization completed. > 150916 13:04:40 12143 Start Srv=0 dfs=0 lcl=0 Pre=1 dmLife=0 0 > 150916 13:04:40 12143 Start Lim=0 0 fix=0 Qmax=1 > 150916 13:04:40 12143 calcSpace New fs info; maxfree=819722MB utilized=7% > 150916 13:04:40 12143 Meter: Found 1 filesystem(s); 854GB total (7% util); 800GB free (800GB max) > 150916 13:04:40 12220 XrdXeq: FS meter thread started > ------ [log in to unmask] phase 2 server initialization completed. > ------ cmsd [log in to unmask]:55165 initialization completed. > 150916 13:24:40 12145 XrdBuffManager: Reshaper has 0K; target 6592098K > 150916 13:44:40 12145 XrdBuffManager: Reshaper has 0K; target 6592098K > 150916 14:04:40 12145 XrdBuffManager: Reshaper has 0K; target 6592098K > 150916 14:24:40 12145 XrdBuffManager: Reshaper has 0K; target 6592098K > 150916 14:44:40 12145 XrdBuffManager: Reshaper has 0K; target 6592098K > ``` > > --- > Reply to this email directly or view it on GitHub: > https://github.com/xrootd/xrootd/issues/288 --- Reply to this email directly or view it on GitHub: https://github.com/xrootd/xrootd/issues/288#issuecomment-140902978 ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1