Passing an argument with a large value (10k lines) to docker run

General Discussions
1

Hello.

I have some data that I want to make available to docker run. At first, I thought about doing docker run -e foo=<data-here> but I got an “argument list too long” error, since the actual data comprises of ~8k lines.

The big picture is this: I have a Go service that will at some point have to run a container and pass it the said data. Could I perhaps pass these data by using the Go SDK?

Thanks in advance.

2

Why not save the data to a file, and bind mount the file using:

docker run -v /path/on/host:/path/on/container
3

This was my initial thought, but it complicates the implementation. The simplest thing to do was that the user just passes some key/value pairs and I pass them to docker run as environmental variables. Using mounts instead would mean the user would now in addition have to specify a new type of variables (“file variables” maybe?) and the service would have to know about them too, as a special type of arguments.

I was looking if there’s a solution that doesn’t use mounts.

Thanks!

4

The amount of data you require makes either environment variables or command line arguments unsuitable. So it’s either a mount, or using the docker api to copy the data into the container directly.

Or maybe you could run a service inside the container which can receive the data from your Go service…this is getting complicated. Never mind. Go with mounts, I think.

5

It’s fundamentally a standard U*nx command line, and those are limited length. I think environment variables might have some limits too. If you’re passing ~8k lines of data, you have to do it via a file or network I/O.

Remember that being able to run a Docker command generally implies root access to the system (if you can docker run anything, it is trivial to give yourself root on the host if you don’t already have it). Running a service as root would be a little odd.

As you’re describing the problem, a more typical way to set this up would be to make the container be a longer-lived network service, and instead of trying to launch a new container (as root) every time you needed to do this action, make a network (HTTP?) call to it.

Be very very careful with escaping. And with what the allowed environment variables are. It seems like this approach has a lot of potential for large security issues.