First, some background:
I have an in-house job scheduling system for some large (hundreds of jobs) data workflows. Broadly speaking, this is a MySQL database that contains job information, a JSP front end, and a Perl daemon that accesses the database to see what jobs it should run next. The jobs are called via Perl system calls, so what’s actually executing is pretty arbitrary. Could be as simple as creating a directory, or could be something like calling some script to extract data from a database. When these jobs finish, the Perl daemon looks at the database and runs the next job per the business rules we have defined.
I’m interested in turning this into something that can run in Docker, and in fact I did so a while back with a little bit of hacking on our codebase. Having a MySQL container was easy. Adding the JSP to a Tomcat container was pretty easy too. Getting the Perl daemon to run was ok as well. The wrinkle came about when I wanted each job in the workflow to be a Docker call itself. In order for the Perl daemon to invoke docker run correctly, it needed to be called with -v /var/run/docker.sock:/var/run/docker.sock.
The idea of having to mount the docker socket inside the container seems a little bit dirty to me. I’m not thrilled about having to run the Perl daemon container in privileged mode, especially in production. One of the great things about containerization is the security, and this flies in the teeth of that. So my question is, how else can i solve this problem? Is there any other way to start what are effectively sibling Docker processes other than privileged mode and mounting docker.sock?