 |
 |
Hi Brian, The release numbers should sync between OSG and us. It definitely was fixed in 4.2.2. That said, I regressed as well. I misread nproc as the file descriptor limit. Please start xrootd (you can do this by hand on your own account as long maxproc is set to unlimited) with the -d option and send me the log file. Just do: xrootd -d no need for any config file to catch this problem. As for the nproc, 1024 is way too small. It should usually be set to about 4K. Andy On Wed, 16 Sep 2015, xrootd-dev wrote: > Might have regressed - as you can see in his log snippet, this was on 4.2.2. > > Brian > > Sent from my iPhone > >> On Sep 16, 2015, at 4:44 PM, Andrew Hanushevsky <[log in to unmask]> wrote: >> >> Hi Andrew, >> >> That problem existed in the early 4.x releases. The reason is that >> unlimited set the nproc value to -1 and xrootd din't know how to handle >> it. It got fixed in 4.1.1 though I'd sugggest going to 4.2.3. >> >> Andy >> >> On Wed, 16 Sep 2015, Andrew Melo wrote: >> >>> It appears that if the xrootd daemon is started with the nprocess limit set to UNLIMITED, it gets upside down: >>> >>> ``` >>> Config maximum number of threads restricted to -1 >>> <snip> >>> 150916 13:04:40 12143 XrdScheduler: Thread limit has been reached! >>> ``` >>> (full log at bottom) >>> (shoutout to @bbockelm who noticed what I had just skimmed right past. :raising_hand:) >>> >>> The behavior changed somehow between 3.x and 4.2.2 - the same box worked fine until xrootd was upgraded. >>> >>> >>> Copr. 2004-2012 Stanford University, xrd version v4.2.2 >>> ++++++ cmsd [log in to unmask] initialization started. >>> Config using configuration file /etc/xrootd/xrootd-lfs.cfg >>> =====> xrd.port 1094 >>> =====> xrd.trace conn >>> =====> all.adminpath /var/run/xrootd >>> Config maximum number of connections restricted to 65536 >>> Config maximum number of threads restricted to -1 >>> 150916 13:04:40 12143 XrdConfig: sendfile enabled. >>> 150916 13:04:40 12145 XrdXeq: Buffer Manager reshaper thread started >>> 150916 13:04:40 12143 XrdSched: scheduling underused thread monitor in 780 seconds >>> 150916 13:04:40 12146 XrdXeq: Time scheduler thread started >>> 150916 13:04:40 12143 XrdScheduler: Thread limit has been reached! >>> 150916 13:04:40 12143 XrdSched: Starting with 0 workers >>> 150916 13:04:40 12143 XrdLink: Allocating 8 link objects at a time >>> 150916 13:04:40 12143 XrdPoll: Starting poller 0 >>> 150916 13:04:40 12147 XrdXeq: Poller thread started >>> 150916 13:04:40 12143 XrdPoll: Starting poller 1 >>> 150916 13:04:40 12148 XrdXeq: Poller thread started >>> 150916 13:04:40 12143 XrdPoll: Starting poller 2 >>> 150916 13:04:40 12149 XrdXeq: Poller thread started >>> 150916 13:04:40 12143 XrdProtocol: getting port from protocol cmsd >>> Copr. 2007 Stanford University/SLAC cmsd. >>> ++++++ [log in to unmask] phase 1 initialization started. >>> =====> all.role server >>> =====> all.manager se1.accre.vanderbilt.edu:1214 >>> Config cms-xrd-transit.cern.ch -> all.manager vocms027.cern.ch >>> =====> all.manager cms-xrd-transit.cern.ch+ 1213 >>> =====> oss.namelib /usr/lib64/libXrdCmsTfc.so file:/etc/xrootd/storage-lfs.xml?protocol=direct >>> =====> all.export / nostage >>> =====> cms.trace all debug >>> =====> all.adminpath /var/run/xrootd >>> =====> all.pidpath /var/run/xrootd >>> The following paths are available to the redirector: >>> w / >>> >>> ``` >>> ------ [log in to unmask] phase 1 server initialization completed. >>> 150916 13:04:40 12143 XrdConfig: LCL port 55165 wsz=10000000 (10000000) >>> 150916 13:04:40 12143 XrdProtocol: getting protocol object cmsd >>> ++++++ [log in to unmask] phase 2 server initialization started. >>> 150916 13:04:40 12143 Configure2 Global System Identification: lfs-s 1214se1.accre.vanderbilt.edu1213vocms027.cern.ch >>> ++++++ Storage system initialization started. >>> =====> oss.namelib /usr/lib64/libXrdCmsTfc.so file:/etc/xrootd/storage-lfs.xml?protocol=direct >>> =====> all.export / nostage >>> Plugin No such file or directory loading namelib /usr/lib64/libXrdCmsTfc-4.so >>> Config Falling back to using /usr/lib64/libXrdCmsTfc.so >>> Plugin loaded unversioned XrdOucgetName2Name from namelib /usr/lib64/libXrdCmsTfc.so >>> Copr. 2009 University of Nebraska-Lincoln TFC plugin v 1.0 >>> Params: file:/etc/xrootd/storage-lfs.xml?protocol=direct >>> Xerces-c has been initialized. >>> Connecting to the catalog file:/etc/xrootd/storage-lfs.xml?protocol=direct >>> Using catalog file /etc/xrootd/storage-lfs.xml >>> ++++++ Configuring standalone mode . . . >>> 150916 13:04:40 12143 oss_AioInit: started AIO read signal thread; tid=889157376 >>> 150916 13:04:40 12143 oss_AioInit: started AIO write signal thread; tid=888104704 >>> No lfn2pfn mapping for / >>> No lfn2pfn mapping for / >>> No lfn2pfn mapping for / >>> Config effective /etc/xrootd/xrootd-lfs.cfg oss configuration: >>> oss.alloc 0 0 0 >>> oss.cachescan 600 >>> oss.fdlimit 32768 65536 >>> oss.maxsize 0 >>> oss.namelib /usr/lib64/libXrdCmsTfc.so >>> oss.trace fff >>> oss.xfr 1 deny 10800 keep 1200 >>> oss.memfile off max 33751543808 >>> oss.defaults r/w nocheck nodread nomig norcreate nopurge nostage xattr >>> oss.path / r/w nocheck nodread nomig norcreate nopurge nostage xattr >>> ------ Storage system initialization completed. >>> 150916 13:04:40 12143 Start Srv=0 dfs=0 lcl=0 Pre=1 dmLife=0 0 >>> 150916 13:04:40 12143 Start Lim=0 0 fix=0 Qmax=1 >>> 150916 13:04:40 12143 calcSpace New fs info; maxfree=819722MB utilized=7% >>> 150916 13:04:40 12143 Meter: Found 1 filesystem(s); 854GB total (7% util); 800GB free (800GB max) >>> 150916 13:04:40 12220 XrdXeq: FS meter thread started >>> ------ [log in to unmask] phase 2 server initialization completed. >>> ------ cmsd [log in to unmask]:55165 initialization completed. >>> 150916 13:24:40 12145 XrdBuffManager: Reshaper has 0K; target 6592098K >>> 150916 13:44:40 12145 XrdBuffManager: Reshaper has 0K; target 6592098K >>> 150916 14:04:40 12145 XrdBuffManager: Reshaper has 0K; target 6592098K >>> 150916 14:24:40 12145 XrdBuffManager: Reshaper has 0K; target 6592098K >>> 150916 14:44:40 12145 XrdBuffManager: Reshaper has 0K; target 6592098K >>> ``` >>> >>> --- >>> Reply to this email directly or view it on GitHub: >>> https://github.com/xrootd/xrootd/issues/288 >> ÿÿ >> Reply to this email directly or view it on GitHub. >> >> >> Use REPLY-ALL to reply to list >> >> To unsubscribe from the XROOTD-DEV list, click the following link: >> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 > > > --- > Reply to this email directly or view it on GitHub: > https://github.com/xrootd/xrootd/issues/288#issuecomment-140907285 --- Reply to this email directly or view it on GitHub: https://github.com/xrootd/xrootd/issues/288#issuecomment-140989432 ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1