Hi Andy,

Thanks for helping.

On 11/11/2015 11:43 AM, Andrew Hanushevsky wrote:
> Hi John,
>
> A couple of questions...
>
> 1) Who is producing the following messages?
> global config file: [ERROR] OS Error: No such file or directory
> [2015-11-11 19:30:23.114653 +0000][Debug  ][Utility           ] Unable 
> to process
> user config file: [ERROR] OS Error: No such file or directory
This messages are in cmsd logs and are produced by xrootd:

qserv@clrinfoport09:~/src/xrootd (lsst-dev)⟫ grep -r "Unable to process 
global config file" *
src/XrdCl/XrdClDefaultEnv.cc:      log->Warning( UtilityMsg, "Unable to 
process global config file: %s",

>
> 2) The "statlib" uses the libXrdSsi.so because we packaged it there as 
> a convenience since we need to use the file registry. Do you have a 
> static initialization section that expects it will fire up all of 
> qserv? We don't want that.
I don't really understand this question, sorry. Here's our configuration 
file, it may help?

qserv@ccqserv126:/qserv$ cat /qserv/run/etc/lsp.cf
# Unified configuration for xrootd/cmsd for both manager and server 
instances
# "if"-block separates manager-only and server-only configuration.

# if regexp block.  Test whether the hostname matches the regexp,
# Example: tuson121 is the hostname of our manager node within our
# allocation at LLNL.

###################################################################################
# WARNING : for mono-node installation, comment manager configuration, and
# don't start cmsd
###################################################################################

############
# if manager
############
if ccqserv125.in2p3.fr*
# Use manager mode
all.role manager

# Enable support for multiple writable file copies
cms.space mwfiles linger 0

# Use standard filesystem plugin
# Newer xrootd uses embedded plugin by default, uncomment for older xrootd
#xrootd.fslib libXrdOfs.so

##########################
# else: non-manager nodes
##########################
else

# Use server mode
all.role server

# Use qserv worker filesystem plugin
# default path is /qserv/run/xrootd-run/
xrootd.fslib libXrdSsi.so
ssi.svclib libxrdsvc.so
oss.statlib -2 libXrdSsi.so

# Set pathname for file location resolution.
# i.e., pathname/somefile will be exported in the xroot URL:
# xroot://manager:0000/somefile
oss.localroot /qserv/run/xrootd-run

# Qserv uses the xrootd XrdSsi, which needs the configuration to
# force-disable asyncronous access.
xrootd.async off
fi

########################################
# Shared directives (manager and server)
########################################

# Writable paths for administration
# cmsd and xrootd paths for pid
all.pidpath /qserv/run/var/run
# path to write logging and other information
all.adminpath /qserv/run/tmp

# host:port of manager instance (2131 is default)
all.manager ccqserv125.in2p3.fr:2131

# Do not change. This specifies valid virtual paths that can be accessed.
# "nolock" directive prevents write-locking and is important for qserv
# qserv is hardcoded for these paths.
all.export / nolock

xrd.port 1094

# Optional: Prevent dns resolution in logs.
# This may speed up request processing.
xrd.network nodnr

# Uncomment the following line for detailed xrootd debugging
# xrootd.trace all debug


>
> 3) This is a container, right?
Yes. FYI, our previous cmsd version was running fine under the same sort 
of container with same network setting.
>
> 4) Login to the container and do a /sbin/ifconfig and send me the 
> output. The cmsd is complining that there is no network configuration.

root@ccqserv126:/qserv# ifconfig
eth0      Link encap:Ethernet  HWaddr 02:42:ac:11:00:07
           inet addr:172.17.0.7  Bcast:0.0.0.0  Mask:255.255.0.0
           inet6 addr: fe80::42:acff:fe11:7/64 Scope:Link
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:1558 errors:0 dropped:0 overruns:0 frame:0
           TX packets:89 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:0
           RX bytes:296356 (289.4 KiB)  TX bytes:6348 (6.1 KiB)

lo        Link encap:Local Loopback
           inet addr:127.0.0.1  Mask:255.0.0.0
           inet6 addr: ::1/128 Scope:Host
           UP LOOPBACK RUNNING  MTU:65536  Metric:1
           RX packets:14 errors:0 dropped:0 overruns:0 frame:0
           TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:0
           RX bytes:1176 (1.1 KiB)  TX bytes:1176 (1.1 KiB)

root@ccqserv126:/qserv# ping ccqserv126
PING ccqserv126.in2p3.fr (172.17.0.7): 56 data bytes
64 bytes from 172.17.0.7: icmp_seq=0 ttl=64 time=0.061 ms
64 bytes from 172.17.0.7: icmp_seq=1 ttl=64 time=0.049 ms
>
> 5) I assume things are registered in DNS or at least appear correctly 
> in /etc/hosts otherwise we will have a problem. The container has to 
> look like an actual machine.
# runned inside he container
root@ccqserv126:/qserv# ping ccqserv126
PING ccqserv126.in2p3.fr (172.17.0.7): 56 data bytes
64 bytes from 172.17.0.7: icmp_seq=0 ttl=64 time=0.061 ms
64 bytes from 172.17.0.7: icmp_seq=1 ttl=64 time=0.049 ms

please note that the bare-metal host has the same name but.
[qserv@ccqserv126 fjammes]$ ping ccqserv126
PING ccqserv126.in2p3.fr (10.158.37.126) 56(84) bytes of data.
64 bytes from ccqserv126.in2p3.fr (10.158.37.126): icmp_seq=1 ttl=64 
time=0.039 ms

>
> Andy
>
> On Wed, 11 Nov 2015, John Gates wrote:
>
>> We're getting an error with cmsd, and not sure what is wrong. The log 
>> shows:
>>
>>
>> =====> oss.localroot /qserv/run/xrootd-run
>> =====> all.pidpath /qserv/run/var/run
>> =====> all.adminpath /qserv/run/tmp
>> =====> all.manager ccqserv125.in2p3.fr:2131
>> =====> all.export / nolock
>> The following paths are available to the redirector:
>> w  /
>>
>> ------ [log in to unmask] phase 1 server initialization 
>> completed.
>> ++++++ [log in to unmask] phase 2 server initialization started.
>> ++++++ Storage system initialization started.
>> =====> oss.statlib -2 libXrdSsi.so
>> =====> oss.localroot /qserv/run/xrootd-run
>> =====> all.export / nolock
>> Plugin No such file or directory loading statlib libXrdSsi-4.so
>> Config Falling back to using libXrdSsi.so
>> [2015-11-11 19:30:23.114120 +0000][Warning][Utility           ] 
>> Unable to process global config file: [ERROR] OS Error: No such file 
>> or directory
>> [2015-11-11 19:30:23.114653 +0000][Debug  ][Utility           ] 
>> Unable to process user config file: [ERROR] OS Error: No such file or 
>> directory
>> [2015-11-11 19:30:23.114794 +0000][Info   ][Utility           ] Env: 
>> Importing from shell XRD_REQUESTTIMEOUT=64000 as RequestTimeout
>> [2015-11-11 19:30:23.114821 +0000][Info   ][Utility           ] Env: 
>> Importing from shell XRD_STREAMTIMEOUT=64000 as StreamTimeout
>> [2015-11-11 19:30:23.114844 +0000][Info   ][Utility           ] Env: 
>> Importing from shell XRD_TIMEOUTRESOLUTION=64000 as TimeoutResolution
>> [2015-11-11 19:30:23.114887 +0000][Info   ][Utility           ] Env: 
>> Importing from shell XRD_DATASERVERTTL=64000 as DataServerTTL
>> [2015-11-11 19:30:23.115001 +0000][Debug  ][PlugInMgr         ] 
>> Initializing plug-in manager...
>> [2015-11-11 19:30:23.115021 +0000][Debug  ][PlugInMgr         ] No 
>> default plug-in, loading plug-in configs...
>> [2015-11-11 19:30:23.115032 +0000][Debug  ][PlugInMgr         ] 
>> Processing plug-in definitions in /etc/xrootd/client.plugins.d...
>> [2015-11-11 19:30:23.115054 +0000][Debug  ][PlugInMgr         ] 
>> Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS 
>> Error: No such file or directory
>> [2015-11-11 19:30:23.115103 +0000][Debug  ][PlugInMgr         ] 
>> Processing plug-in definitions in 
>> /home/qserv/.xrootd/client.plugins.d...
>> [2015-11-11 19:30:23.115124 +0000][Debug  ][PlugInMgr         ] 
>> Unable to process directory /home/qserv/.xrootd/client.plugins.d: 
>> [ERROR] OS Error: No such file or directory
>> ++++++ ssi phase 1 initialization started.
>> =====> all.role server
>> =====> ssi.svclib libxrdsvc.so
>> ------ ssi phase 1 initialization completed.
>> ++++++ ssi phase 2 initialization started.
>> 151111 19:30:23 1143 sysFinder: Network i/f undefined; unable to 
>> self-locate.
>> ------ ssi phase 2 initialization failed.
>> ++++++ Configuring standalone mode . . .
>> ------ Storage system initialization failed.
>> ------ [log in to unmask] phase 2 server initialization failed.
>> 151111 19:30:23 1143 XrdProtocol: Protocol cmsd could not be loaded
>> ------ cmsd [log in to unmask]:1094 initialization failed.
>>
>> ########################################################################
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the QSERV-L list, click the following link:
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
>>
>
> ########################################################################
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the QSERV-L list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1

########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1