Hi Matevz,
I just tried this in the official 4.1.0. Without specifying any thread
limit, I get the expected
Config maximum number of threads restricted to 12740
because that what we set the thread hard limit to. Specifying the maximum
threads also works but I see it is not optimal in that the scheduler uses
the specification to precreate a lot of threads that will eventually be
disposed of. I will see what I can do about that but it's not a critical
problem unless you specifying some rediculously large number. The whole
point of the 4.1.0 fix was to get people from specifying the xrd.sched
parameter as it creates more problems than it solves. So, what is it that
you do? What version of Linux do you run?
Andy
On Mon, 1 Dec 2014, Matevz Tadel wrote:
> Thanks Wei! I'll try it tomorrow.
>
> This is happening with stable-4.1.x built today.
>
> Matevz
>
> On 12/01/14 15:20, Yang, Wei wrote:
>> You may need something like:
>>
>> xrd.sched maxt 16384
>>
>> according to Andy, this was fixed on 11/17
>>
>> Wei Yang | [log in to unmask] | 650-926-3338(O)
>>
>>
>>
>> On Dec 1, 2014, at 3:13 PM, Matevz Tadel <[log in to unmask]> wrote:
>>
>>> Hi,
>>>
>>> I'm running into a funny problem, this shows up in the logs:
>>>
>>> Config maximum number of threads restricted to -1
>>> 141201 14:39:20 5315 XrdScheduler: Thread limit has been reached!
>>>
>>> There's a funny thing with max num of processes:
>>>
>>> [1509] root@cabinet-10-10-11 /var/log/xrootd# ps aux|grep xroot
>>> xrootd 5299 0.0 0.1 240240 2440 ? Sl 14:39 0:00 xrootd -I v4 -c /opt/klus.cfg -b -l /var/log/xrootd/xrootd.log -s /var/run/xrootd/xrootd.pid
>>> xrootd 5315 0.0 0.1 105464 2500 ? Sl 14:39 0:00 cmsd -I v4 -c /opt/klus.cfg -b -l /var/log/xrootd/cmsd.log -s /var/run/xrootd/cmsd.pid
>>> [1509] root@cabinet-10-10-11 /var/log/xrootd# cat /proc/5299/limits
>>> Limit Soft Limit Hard Limit Units
>>> Max cpu time unlimited unlimited seconds
>>> Max file size unlimited unlimited bytes
>>> Max data size unlimited unlimited bytes
>>> Max stack size 10485760 unlimited bytes
>>> Max core file size unlimited unlimited bytes
>>> Max resident set unlimited unlimited bytes
>>> Max processes unlimited unlimited processes ### <------ this
>>> Max open files 4096 4096 files
>>> Max locked memory 65536 65536 bytes
>>> Max address space unlimited unlimited bytes
>>> Max file locks unlimited unlimited locks
>>> Max pending signals 14866 14866 signals
>>> Max msgqueue size 819200 819200 bytes
>>> Max nice priority 0 0
>>> Max realtime priority 0 0
>>> Max realtime timeout unlimited unlimited us
>>>
>>> So it seems that unlimited gets turned into -1 by xrootd. After that the server does not connect to the redirector.
>>>
>>>
>>> I was a bit annoyed by this so I set the nproc limit to 8192 in security/limits. This is apparently honored in shell but somehow does not get propagated to xrootd.
>>>
>>> [1509] root@cabinet-10-10-11 /var/log/xrootd# sudo -u xrootd bash -c "ulimit -a"
>>> core file size (blocks, -c) 0
>>> data seg size (kbytes, -d) unlimited
>>> scheduling priority (-e) 0
>>> file size (blocks, -f) unlimited
>>> pending signals (-i) 14866
>>> max locked memory (kbytes, -l) 64
>>> max memory size (kbytes, -m) unlimited
>>> open files (-n) 1024
>>> pipe size (512 bytes, -p) 8
>>> POSIX message queues (bytes, -q) 819200
>>> real-time priority (-r) 0
>>> stack size (kbytes, -s) 10240
>>> cpu time (seconds, -t) unlimited
>>> max user processes (-u) 8192
>>> virtual memory (kbytes, -v) unlimited
>>> file locks (-x) unlimited
>>>
>>> Any ideas what else to check / change? This is the code that does this check:
>>>
>>> https://github.com/xrootd/xrootd/blob/master/src/Xrd/XrdConfig.cc#L855
>>>
>>> Best,
>>> Matevz
>>>
>>> ########################################################################
>>> Use REPLY-ALL to reply to list
>>>
>>> To unsubscribe from the XROOTD-DEV list, click the following link:
>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>>
>>
>> ########################################################################
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the XROOTD-DEV list, click the following link:
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>>
>
> ########################################################################
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the XROOTD-DEV list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
|