Runnnig multiple docker daemons

Docker Engine General
1

We have been trying to run multiple (5-7) docker daemon processes in one host, in order to provide isolated docker environment to each developer. We create custom network bridges and run like

docker daemon -D -g /home/{user}/docker/g --exec-root=/home/{user}/docker/e -b br_{user} --dns=8.8.8.8 --iptables=false -H unix:///home/{user}/docker/docker.sock -p /home/{user}/docker/docker.pid

It runs for a day or so, but the issue occurs almost every day that network stop working suddenly. Digging into it, we’ve found that IP masquerade in the iptables are gone in those events, but we don’t see why iptables get broken. Also sometimes we see docker.pid file is also gone along with the issue, but the daemon process is up, and as far as the containerized process don’t try to send outgoing packets it is ok. The issue affects only the outgoing packet from the containerized processes.

My question is,

  • Do you think more than one daemon processes can run in one host? Any idea on things to keep in mind?
  • Do you have any suggestions where to look to do further investigation?
  • Any workaround?

Docker version is 1.8.2, the host OS as well as most dockerized process OS is Ubuntu14.04

Thanks,
Hitoshi

2

Are there any changes? Could you run multiple Docker instances?

Are these instances completely isolated from one another?
I am thinking of a way to share one docker installation on one host between multiple users.

Thanks,
Peter

3

We worked around by refreshing iptable masquerade setting every minute that time, and it’s been ok. Now I’m also not seeing the issue even without the refresh anymore, probably after upgrading to 1.9. Not sure exactly when the issue went away. Nevertheless, it would be nice if we have more support from docker core to run the multiple daemons easily.

4

Thank you very much! It sounds very promising!

Peter

5

Also refer
http://blog.alpaca.ai/run-multiple-docker-daemons-with-net-container/