I am building a gossip algorithm, that should make thousands of nodes communicate with each other (let’s call them containers, to disambiguate). I’m thinking of using docker swarm mode and an overlay network, since it will do a lot of cumbersome work on my behalf.
I need to provide my containers with IPs of existing containers to bootstrap my gossip algorithm. Say they are started sequentially, so the first node has no parameter IP, the second has the IP of the first one, the third has the IP of container #1 and #2, and so on.
Until then, I was passing these IPs as command line parameters, but it won’t be possible anymore using docker swarm services. I was thinking of using an external repository to store and retrieve IPs for the containers, but it seems dumb, since docker already ships one, and it seems to know the IP of anyone in the system, given the right command. I saw there that I could get all the IPs from a container using nslookup or dig, for instance.
My question is: can I use a Docker Remote API (like Go’s docker/client) to retrieve IPs of containers participating in a service, from inside a container?
Apart from that, thanks for all your work, Docker is really quite awesome.
Using etcd or Consul for this is pretty common. I think if you’re already using Docker Swarm then it will provide a DNS name that should resolve to at least some of the relevant container addresses.
No, unless you have a non-default setup that exposes the Docker socket into containers, and you are comfortable potentially giving software running inside one container unrestricted root-level access to everything on the host.
Thanks @dmaze, I already understand my setup better.
Ok, let’s not give my containers root powers, safety first. In a swarm mode context, I believe I will just have to create two services: my project’s one, and my kv store. I will look up whether the deployment of etcd, for instance, has been facilitated with the arrival of docker swarm mode; I was not very confident deploying it, since it’s my first time using a kv store.
So, since I’m so new at kv stores, can you please tell me how I’d ask etcd (or consul) to retrieve “only the last entries that were added to the store”? Sorry this is totally off-topic… Adding elements in-order is explained here, but I don’t understand the right approach to limit the results length. Should I put a TTL on the keys, to avoid keeping thousands of IPs? I won’t control the exact amount of kept keys, that way. Or should I send all the IPs to every client and then discard the first entries? But then I will flood the network. Or is there a better way to ask precisely for “the ten last entries of the directory”?
Anyway, let’s say the topic is closed, many thanks
