Docker Community Forums

Share and learn in the Docker community.

Swarm for batch jobs


(Ray Johnson) #1

With Docker 1.11 and the old swarm we built a nice integration with Rundeck to run batch jobs that were defined in containers. This was working very well for us.

But with swarm in docker 1.12 we have a real problem getting this use case to work:

For one “docker run” is not swarm aware like it was on docker 1.11. Rather it will only run the job in the node you are talking to - and not find a node to run it on.

Then the new “docker service” only supports long running jobs - meaning it will restart the container when it exists. Not exactly what you want for a batch job that is supposed to run once a day.

Our current work around is to query the swarm master for all the nodes and pick a machine to use to call “docker run” on. (Currently using the rand() strategy. :slight_smile: ) This is working for our immediate needs but is certainly not ideal.

Are there plans to either 1) allow docker run to let swarm pick which node to run the container, or 2) provide a new swarm mode or something to allow scheduling a task that we do not want to restart?

Ray