Hi Docker Comunity,
I am working a project which requires large amount of applications running in Docker container.
I am already running out of disk space with few apps. My machine was having 30GB of disk space and there is further space to run container.
What I like to know that is it a good idea to have one machine running Docker and then having several containers running on different port. There will be quite a few base images like tomcat, lamp etc. Each of this will be having one or more applications, running on different ports.
For e.g. I will have a base image of LAMP stack and then spin up one or two apps may be more on it. Ofcourse I will be committing my changes so I can re-run the container with apps whenever needed.
If anyone from community can express their thoughts on having one machine to run multiple containers, having few images and impact on disk space that would be really helpful.
Sure 1 host can handle multiple containers, if its hardware is big enough to handle it.
“The Docker Way”, would not be to have a LAMP stack in 1 container, but split it out, so you have a mysql container and a apache/php container.
There’s nothing wrong with this, so long as it fits on the system you have. It sounds like your current system is filling up, so you might consider getting a bigger system (easier if it’s a cloud system) or splitting workloads up into multiple systems.
One easy first step could be to move the databases on to dedicated hosts (or hosted cloud services). This can be a good idea because they’re often the “bottom” of the dependency stack, their data is actually critical, and updates to their software are infrequent: they have a fundamentally different lifecycle than most applications you run under Docker. That might help free up enough space to get you unstuck.
Otherwise, you’re starting to look at more complicated paths like Docker Swarm or Kubernetes, which will come with some more discipline around using Docker. Speaking of…
I’d recommend you forget docker commit exists. Start using docker build if you haven’t already and check your Dockerfiles into source control alongside your applications. If you do start doing things across multiple hosts, you can have a sequence where you check in application code, an image gets built out of it and pushed into a registry, and then that image gets deployed to your cluster. Then every host will have consistent software, and when /var/lib/docker manages to break you won’t actually have lost your hand-tweaked image.
As others have pointed out, you are adding too much to your containers. Each application should be deployed it its own app container separate from any database or web container. It is not advisable, for example, to have a Tomcat container and deploy several applications into it. This defeats the purpose of using containers to isolate workloads. Instead, create multiple images from your base Tomcat image, each with a single application inside. For example:
Create a Dockerfile that contains the first app:
FROM tomcat:9-jre8-alpine
COPY FirstApplication.war /usr/local/tomcat/webapps/
Create another Dockerfile that contains the second app:
FROM tomcat:9-jre8-alpine
COPY SecondApplication.war /usr/local/tomcat/webapps/
Then use docker build to create separate images from which you will instantiate your containers. Docker uses a layered filesystem. Layers are reusable. Both of these images will share the common layers from tomcat:9-jre8-alpine so that the only additional disk space will be the apps themselves. This should take up exactly the same amount of disk space as placing multiple apps into one container but it has the added benefit that you can now redeploy a single application without taking any of the others down. I also selected the Alpine version of Tomcat which is only 108MB in size instead of 463MB in size. A significant savings in disk space.
Your databases should be in their own containers as well with external volumes mapped to persistent storage on your VM so that your data will (1) survive container restarts and (2) be easily backed up. Likewise for any web front ends like Apache or Nginx being in separate containers. This has the added advantage of being able to separately scale the web, app, and db containers as needed instead of scaling the entire stack. This will maximize the limited resources that you have.
Thank you all for the response. I think i have got some great points : Basically separate out db container and apache/webserver container to reduce the disk usage. And use docker file with base configuration and then on top of it build images. Just to confirm I don’t need to have separate database really if 10 different apps can rely on single container running with required database? I am thinking if i would put too much on single database if something happens then I will not be able to connect to all 10 apps. Perhaps creating 2 different containers running mysql db makes sense.
Also, as far as storage driver is concerned my docker is using devicemapper. Should I move it to overlay2? I read that overlay2 could cause inode exhaustion and that might not be the way to go.
