Quick preface: We currently provision all of our servers from simple bash scripts. Everything from yum removing bloat, installing needed dependencies, compiling our services, etc. I’m almost certain Docker can make my life so much simpler in all aspects but I’m still really confused…
TLDR; Frankly, the entirety of this post is almost simply “how would we make a container that contains our current configuration work identically across anyone who deploys the image”. The only real things we tweak on the host OS is maybe ~4 config files for things like network stack, ulimits, firewall config depending on the service(s) on the machine, etc.
I’m think I’m slowly understanding what Docker “does” in relation to isolating what’s inside of the container, etc. However, due to that nature and from various answers of ‘containers are meant to be ephemeral’ from a few different people, I’m beginning to question if Docker is even a tool that I’d be looking for.
So I know that I could have a container that just contains a webserver (nginx/httpd/etc), another container that contains php-fpm if I wanted to put that in a separate container, another container for your mysql, etc.
One of the first things that’s throwing me for a loop is compiling those apps and then building the image that just contains them. Is it correct that compiling apps such as those (versus say… node.js in terms of an ‘app’) would inherently require a slim OS in the container as well, purely from the stand-point that I always compile these apps myself instead of just pulling from yum/apt?
We also do things on the host OS such as changing those few config tweaks like network stack, ulimits, and a few other things, so how exactly would that work in relation to having a slim OS in the container? Taking my above nginx example, I have nginx run as it’s own user, I change the ulimits on the nginx user does having nginx run inside of a container (let alone inside of a container that also contains an OS) affect that? If the host OS doesn’t already have the nginx owner account created I presume nginx won’t start?
I don’t want to drone off too much on tons of intricacy type questions like that straight from the get-go, but when you think “identical development and product environments” that kinda starts getting thrown out the window when you alter actual OS file, no? Unless you put the ENTIRE system inside of a container and your host OS become a generic OS with no modifications? At which point, I’ll openly admit I’m not a linux guru, I don’t even think that would be logical/possible since the OS inside the container could have all sorts of crazy sysctl and ulimit and whatever other parameters you set, but the host OS wouldn’t care about you making nginx’s nofile 15000 in sysctl, the host OS would still restrict it to it’s default, no?
Sorry for essentially writing a novel with such simple questions for you experience guys, but it seems like if you have such a custom environment (I personally don’t think it’s any different than the majority of servers, if not less actually heh) you’re in the realm of imaging instead of simple containers and that’s what I don’t believe is true since I’m almost positive Docker works in such basic setups like that. To put it another way, the aformentioned bash script literally compiles nginx with modules, cat’s in our config scripts for our vhost(s), wget’s the latest tar’d vhost(s) directories from the hourly backup, etc. Same with mysql, pulls 5.7 Percona from yum, wget’s the latest DB snapshot, and good to go.
Finally, with a few comments I’ve received from people saying ‘containers are meant to be ephemeral’ is it actually NOT the point/ability of containers to be able to ensure that identical config so when we spin up a new server we instantly brought another webserver online as soon as we pulled the image? Perhaps I’m leaning too heavily on the definition but that to me makes it sound like it’s almost predominatently meant to allow developers to quickly work on some items, verify they work locally, which then means they will work identical on production and you shouldn’t be using containers for things like your stack (I know has to be wrong heh) or your DB’s, etc.
I truly appreciate any comments, and again I apologize for droning on.