Be able to access container filesystems on disk from the OS X host
Actual behavior
Containers’ files are stored in the qcow2 file which is difficult to access.
More Info
I have some containers built that include Python environments which work well to host my apps.
I want to be able to point my editor running on my Mac at these Python environments for code linting, tab-completion etc which requires access to the files within the python environment in the container, to see the modules installed within that environment. However I can’t access them as they are stored in the qcow2 copy on write file used by the docker for mac VM. My editor doesn’t run in a container, so I can’t access them via share volume etc.
Some Options:
Use docker cp to copy the environment out of the built container- bad because this is a manual step that has to be done every time the docker env changes, doesn’t guarantee the env your dev tools are using is the one the container is
Somehow mount the qcow2 file locally? This sounds like a bad idea and seems non-trivial
Mount a local dir at container build time that is used to store the environment, which is then used by the dev tools and the container via a mount. Bad because this is requires 3rd party tools like rocker, and mounting things at build time seems to be unpopular for predictability reasons which is fair.
Run a network file system server within the container or another one with access to a shared volume and share it over NFS / SSHFS etc. This feels pretty horrible, adds another layer, requires mounting the FS everytime the container starts etc.
Do I have any other options other than these? If not is there any work to be done to allow native file access to the container file systems from a host, via a ‘reverse mount’ option at container runtime or moving away from the qcow2 file and just storing files on disk etc?
If not is there any work to be done to allow native file access to the container file systems from a host, via a ‘reverse mount’ option at container runtime or moving away from the qcow2 file and just storing files on disk etc?
The easiest way to do this is via volume sharing at docker run time. In the case of my build environments, I volume mount the source and build directories into the container (thus sharing them with the OSX or Windows host), and then point the editor at the directory on the Mac. As long as there is no architecture mismatch between OSX and Linux, this works.
Note that docker run containers can be long-lasting, and you can also docker exec into the container to run more commands – this can be useful to execute IDE commands within the container, as stdin/stdout are captured by the host docker cli tool.
Does docker run -v volume sharing help with your dev setup?
run -v means I can mount in a local copy of the code which is great, and as you say, I can docker exec in and run say tests or whatever within that container, which is also great! However, things like code linting, code completion often run within the editor or from a small command by the editor. Some editors seem to be integrating Docker support to access the environment for just these things ( https://blog.jetbrains.com/pycharm/2015/12/using-docker-in-pycharm/ ) but i’m looking at things like Atom. This expects a path to a python interpreter / linting commands which are on the local machine. The linter could work by docker execing into a running container and running the lint command there, however not sure how it would find the correct container etc? The code completion is a bit more ingrained and so might be harder…
I guess my request for the ‘reverse -v’ option would quite make sense, and the whole point is those commands are run within the container itself, so the exec method is better, though a little tricky…
The problem with the reverse mount option is that the build artefacts (e.g. the pyc files) are Linux-format, whereas the Python interpreter running on OSX will expect OSX-format ones. So the editor plugins really ought to be wrapped with a docker exec so that the python call also runs with in the container.
Several languages do support this remote mode for their editor tools; for Python the Jupyter/iPython protocol can be run over TCP into a container. I’ve not looked in detail at this though – would be most interested if you have time to see if it could be made to work with D4Mac.
Seems like some docker support is needed to use a container environment, though not unlike virtualenv support and the like. Time to open some Pull Requests and learn coffee script maybe for my Atom Linters etc!