 |
 |
Hi all, I got the following on the UNL redirector: 110706 15:20:29 3409 XrdScheduler: Thread limit has been reached! Sure enough, if you look below, there were a lot of threads - but not that many active users of the cmsd. There were about 500 users connected to the redirector at the time. Ideas? Should I just bump it and restart? Brian [root@xrootd ~]# ps -C cmsd -L | wc -l 2066 [root@xrootd ~]# lsof -p 3409 COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME cmsd 3409 xrootd cwd DIR 202,3 4096 1335851 /etc/xrootd/MAIN cmsd 3409 xrootd rtd DIR 202,3 4096 2 / cmsd 3409 xrootd txt REG 202,3 590160 1265174 /usr/bin/cmsd cmsd 3409 xrootd mem REG 202,3 139416 484802 /lib64/ld-2.5.so cmsd 3409 xrootd mem REG 202,3 1718120 484819 /lib64/libc-2.5.so cmsd 3409 xrootd mem REG 202,3 23360 485018 /lib64/libdl-2.5.so cmsd 3409 xrootd mem REG 202,3 145824 485021 /lib64/libpthread-2.5.so cmsd 3409 xrootd mem REG 202,3 615136 484821 /lib64/libm-2.5.so cmsd 3409 xrootd mem REG 202,3 53448 485022 /lib64/librt-2.5.so cmsd 3409 xrootd mem REG 202,3 58400 484823 /lib64/libgcc_s-4.1.2-20080825.so.1 cmsd 3409 xrootd mem REG 202,3 114352 484869 /lib64/libnsl-2.5.so cmsd 3409 xrootd mem REG 202,3 976312 910653 /usr/lib64/libstdc++.so.6.0.8 cmsd 3409 xrootd mem REG 202,3 53880 484850 /lib64/libnss_files-2.5.so cmsd 3409 xrootd mem REG 202,3 217016 1132028 /var/db/nscd/hosts cmsd 3409 xrootd 0u CHR 1,3 926 /dev/null cmsd 3409 xrootd 1u CHR 1,3 926 /dev/null cmsd 3409 xrootd 2w REG 202,3 3913587 1423082 /var/log/xrootd/MAIN/cmsd.log cmsd 3409 xrootd 3r 0000 0,11 0 9650817 eventpoll cmsd 3409 xrootd 4r FIFO 0,6 9650818 pipe cmsd 3409 xrootd 5w FIFO 0,6 9650818 pipe cmsd 3409 xrootd 6r 0000 0,11 0 9650819 eventpoll cmsd 3409 xrootd 7u IPv4 9650827 TCP xrootd.unl.edu:mpc-lifenet->xrootd.unl.edu:33154 (CLOSE_WAIT) cmsd 3409 xrootd 8r FIFO 0,6 9650820 pipe cmsd 3409 xrootd 9w FIFO 0,6 9650820 pipe cmsd 3409 xrootd 10r 0000 0,11 0 9650821 eventpoll cmsd 3409 xrootd 11r FIFO 0,6 9650822 pipe cmsd 3409 xrootd 12w FIFO 0,6 9650822 pipe cmsd 3409 xrootd 13u IPv4 9650823 TCP *:mpc-lifenet (LISTEN) cmsd 3409 xrootd 14u unix 0xffff880054caa980 9650824 /var/run/xrootd/MAIN/.olb/olbd.nimda cmsd 3409 xrootd 16u IPv4 10319192 TCP xrootd.unl.edu:mpc-lifenet->xrootd.t2.ucsd.edu:44711 (CLOSE_WAIT) cmsd 3409 xrootd 17u IPv4 9844692 TCP xrootd.unl.edu:mpc-lifenet->gridftp-16-23.ultralight.org:45031 (ESTABLISHED) cmsd 3409 xrootd 18u IPv4 10250440 TCP xrootd.unl.edu:mpc-lifenet->cithep172.ultralight.org:55049 (ESTABLISHED) cmsd 3409 xrootd 19u IPv4 9900239 TCP xrootd.unl.edu:mpc-lifenet->crabserver.rcac.purdue.edu:58973 (ESTABLISHED) cmsd 3409 xrootd 20u IPv4 9925712 TCP xrootd.unl.edu:mpc-lifenet->cithep230.ultralight.org:42783 (ESTABLISHED) cmsd 3409 xrootd 21u IPv4 10319223 TCP xrootd.unl.edu:mpc-lifenet->xrootd.t2.ucsd.edu:49612 (CLOSE_WAIT) cmsd 3409 xrootd 22u IPv4 9887691 TCP xrootd.unl.edu:mpc-lifenet->cmsdbs.rcac.purdue.edu:33708 (ESTABLISHED) cmsd 3409 xrootd 23u IPv4 9943439 TCP xrootd.unl.edu:mpc-lifenet->cithep251.ultralight.org:58624 (ESTABLISHED) cmsd 3409 xrootd 24u IPv4 9687047 TCP xrootd.unl.edu:mpc-lifenet->cmssrv32.fnal.gov:47378 (ESTABLISHED) cmsd 3409 xrootd 25u IPv4 10330951 TCP xrootd.unl.edu:mpc-lifenet->xrootd.unl.edu:50468 (ESTABLISHED) cmsd 3409 xrootd 26u IPv4 9844312 TCP xrootd.unl.edu:mpc-lifenet->cmshdfs02.hep.wisc.edu:36182 (ESTABLISHED) cmsd 3409 xrootd 27u IPv4 10002012 TCP xrootd.unl.edu:mpc-lifenet->srm.unl.edu:49894 (ESTABLISHED) cmsd 3409 xrootd 28u IPv4 9900246 TCP xrootd.unl.edu:mpc-lifenet->cithep160.ultralight.org:52890 (ESTABLISHED) cmsd 3409 xrootd 29u IPv4 10330984 TCP xrootd.unl.edu:mpc-lifenet->xrootd.t2.ucsd.edu:50437 (CLOSE_WAIT) cmsd 3409 xrootd 30u IPv4 10331018 TCP xrootd.unl.edu:mpc-lifenet->xrootd.t2.ucsd.edu:50442 (CLOSE_WAIT) cmsd 3409 xrootd 31u IPv4 10331054 TCP xrootd.unl.edu:mpc-lifenet->xrootd.t2.ucsd.edu:52384 (CLOSE_WAIT) cmsd 3409 xrootd 32u IPv4 10331099 TCP xrootd.unl.edu:mpc-lifenet->xrootd.t2.ucsd.edu:52388 (CLOSE_WAIT) cmsd 3409 xrootd 33u IPv4 10331129 TCP xrootd.unl.edu:mpc-lifenet->xrootd.t2.ucsd.edu:52390 (CLOSE_WAIT) cmsd 3409 xrootd 34u IPv4 10331162 TCP xrootd.unl.edu:mpc-lifenet->xrootd.t2.ucsd.edu:52396 (CLOSE_WAIT) cmsd 3409 xrootd 35u IPv4 10331186 TCP xrootd.unl.edu:mpc-lifenet->xrootd.t2.ucsd.edu:34861 (ESTABLISHED)