Hi Andrew,
That problem existed in the early 4.x releases. The reason is that
unlimited set the nproc value to -1 and xrootd din't know how to handle
it. It got fixed in 4.1.1 though I'd sugggest going to 4.2.3.
Andy
On Wed, 16 Sep 2015, Andrew Melo wrote:
> It appears that if the xrootd daemon is started with the nprocess limit set to UNLIMITED, it gets upside down:
>
> ```
> Config maximum number of threads restricted to -1
> <snip>
> 150916 13:04:40 12143 XrdScheduler: Thread limit has been reached!
> ```
> (full log at bottom)
> (shoutout to @bbockelm who noticed what I had just skimmed right past. :raising_hand:)
>
> The behavior changed somehow between 3.x and 4.2.2 - the same box worked fine until xrootd was upgraded.
>
>
> Copr. 2004-2012 Stanford University, xrd version v4.2.2
> ++++++ cmsd [log in to unmask] initialization started.
> Config using configuration file /etc/xrootd/xrootd-lfs.cfg
> =====> xrd.port 1094
> =====> xrd.trace conn
> =====> all.adminpath /var/run/xrootd
> Config maximum number of connections restricted to 65536
> Config maximum number of threads restricted to -1
> 150916 13:04:40 12143 XrdConfig: sendfile enabled.
> 150916 13:04:40 12145 XrdXeq: Buffer Manager reshaper thread started
> 150916 13:04:40 12143 XrdSched: scheduling underused thread monitor in 780 seconds
> 150916 13:04:40 12146 XrdXeq: Time scheduler thread started
> 150916 13:04:40 12143 XrdScheduler: Thread limit has been reached!
> 150916 13:04:40 12143 XrdSched: Starting with 0 workers
> 150916 13:04:40 12143 XrdLink: Allocating 8 link objects at a time
> 150916 13:04:40 12143 XrdPoll: Starting poller 0
> 150916 13:04:40 12147 XrdXeq: Poller thread started
> 150916 13:04:40 12143 XrdPoll: Starting poller 1
> 150916 13:04:40 12148 XrdXeq: Poller thread started
> 150916 13:04:40 12143 XrdPoll: Starting poller 2
> 150916 13:04:40 12149 XrdXeq: Poller thread started
> 150916 13:04:40 12143 XrdProtocol: getting port from protocol cmsd
> Copr. 2007 Stanford University/SLAC cmsd.
> ++++++ [log in to unmask] phase 1 initialization started.
> =====> all.role server
> =====> all.manager se1.accre.vanderbilt.edu:1214
> Config cms-xrd-transit.cern.ch -> all.manager vocms027.cern.ch
> =====> all.manager cms-xrd-transit.cern.ch+ 1213
> =====> oss.namelib /usr/lib64/libXrdCmsTfc.so file:/etc/xrootd/storage-lfs.xml?protocol=direct
> =====> all.export / nostage
> =====> cms.trace all debug
> =====> all.adminpath /var/run/xrootd
> =====> all.pidpath /var/run/xrootd
> The following paths are available to the redirector:
> w /
>
> ```
> ------ [log in to unmask] phase 1 server initialization completed.
> 150916 13:04:40 12143 XrdConfig: LCL port 55165 wsz=10000000 (10000000)
> 150916 13:04:40 12143 XrdProtocol: getting protocol object cmsd
> ++++++ [log in to unmask] phase 2 server initialization started.
> 150916 13:04:40 12143 Configure2 Global System Identification: lfs-s 1214se1.accre.vanderbilt.edu1213vocms027.cern.ch
> ++++++ Storage system initialization started.
> =====> oss.namelib /usr/lib64/libXrdCmsTfc.so file:/etc/xrootd/storage-lfs.xml?protocol=direct
> =====> all.export / nostage
> Plugin No such file or directory loading namelib /usr/lib64/libXrdCmsTfc-4.so
> Config Falling back to using /usr/lib64/libXrdCmsTfc.so
> Plugin loaded unversioned XrdOucgetName2Name from namelib /usr/lib64/libXrdCmsTfc.so
> Copr. 2009 University of Nebraska-Lincoln TFC plugin v 1.0
> Params: file:/etc/xrootd/storage-lfs.xml?protocol=direct
> Xerces-c has been initialized.
> Connecting to the catalog file:/etc/xrootd/storage-lfs.xml?protocol=direct
> Using catalog file /etc/xrootd/storage-lfs.xml
> ++++++ Configuring standalone mode . . .
> 150916 13:04:40 12143 oss_AioInit: started AIO read signal thread; tid=889157376
> 150916 13:04:40 12143 oss_AioInit: started AIO write signal thread; tid=888104704
> No lfn2pfn mapping for /
> No lfn2pfn mapping for /
> No lfn2pfn mapping for /
> Config effective /etc/xrootd/xrootd-lfs.cfg oss configuration:
> oss.alloc 0 0 0
> oss.cachescan 600
> oss.fdlimit 32768 65536
> oss.maxsize 0
> oss.namelib /usr/lib64/libXrdCmsTfc.so
> oss.trace fff
> oss.xfr 1 deny 10800 keep 1200
> oss.memfile off max 33751543808
> oss.defaults r/w nocheck nodread nomig norcreate nopurge nostage xattr
> oss.path / r/w nocheck nodread nomig norcreate nopurge nostage xattr
> ------ Storage system initialization completed.
> 150916 13:04:40 12143 Start Srv=0 dfs=0 lcl=0 Pre=1 dmLife=0 0
> 150916 13:04:40 12143 Start Lim=0 0 fix=0 Qmax=1
> 150916 13:04:40 12143 calcSpace New fs info; maxfree=819722MB utilized=7%
> 150916 13:04:40 12143 Meter: Found 1 filesystem(s); 854GB total (7% util); 800GB free (800GB max)
> 150916 13:04:40 12220 XrdXeq: FS meter thread started
> ------ [log in to unmask] phase 2 server initialization completed.
> ------ cmsd [log in to unmask]:55165 initialization completed.
> 150916 13:24:40 12145 XrdBuffManager: Reshaper has 0K; target 6592098K
> 150916 13:44:40 12145 XrdBuffManager: Reshaper has 0K; target 6592098K
> 150916 14:04:40 12145 XrdBuffManager: Reshaper has 0K; target 6592098K
> 150916 14:24:40 12145 XrdBuffManager: Reshaper has 0K; target 6592098K
> 150916 14:44:40 12145 XrdBuffManager: Reshaper has 0K; target 6592098K
> ```
>
> ---
> Reply to this email directly or view it on GitHub:
> https://github.com/xrootd/xrootd/issues/288
—
Reply to this email directly or view it on GitHub.
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1