Hi Andy and John,
The docker issues have been solved:
1. When updating interactively a container, for example to change a
config file, the default startup command (i.e.
/qserv/scripts/start.sh) will be replaced with "bash", because the
updated container was running bash (interactively) as main command.
That's why ours containers weren't starting on the cluster, they
were running bash, instead of start.sh.
2. When updating interactively a container,services musn't be
started in the container because this can lead to inconsistent state
at the commit; especially if services doesn't start successfully in
interactive mode. That's why cmsd/xrootd weren't starting...
I'm not trained to interactively update containers, my scripts are
fully automated, but configuration step will have to be improved.
This issues are now fixed and log are enabled on all workers.
You can re-setup the cluster from scratch by running next
commands on ccqservbuild:
git clone https://github.com/fjammes/qserv-cluster.git
cd qserv-cluster/shmux
./run.sh
# here's some quick and dirty scripts to extract info from czar log:
cat ../log-ana/get-chk.sh
Regards,
Fabrice