We have an application written in Java. It has two interfaces: a web interface for users, and a CLI interface for admin operations.
Some of my colleagues are using the CLI interface, and they are used to SSH into a server, and execute a command.
The CLI interface is a traditional interactive shell script, with options presented as a list, and the user must specify a number, then press Enter, to specify an option. Most of the times further menu items are presented and input is required from the user.
The CLI interface and the Web interface share and execute the same codebase.
We recently moved our application into Docker. Until then it was running directly on an AWS EC2 instance. Now it runs inside a docker container on an AWS EC2 instance.
I was asked to preserve the way my colleagues are using the CLI tool, so they don’t have to understand what docker is and how to figure stuff out. They are mostly non technical people.
So I created a wrapper script that executes the desired CLI application inside docker. My colleagues, SSH into the EC2 instance (same as before), then the execute the same command as before. The command is now the script I mentioned and it does a docker exec -it <container_name> <command_inside_docker>. So far so good. The application works, all interactions are correct.
Now, the problem.
It happens, quite frequently, that people using this tool are on spotty Internet connection, or they just close their terminal with the SSH session, or SSH session times out. So, basically, their SSH session gets closed, but without exiting from the application’s CLI interface.
This unfortunately leaves the application’s CLI interface running inside the docker image, and the next time they use the tool, a new process is started with the CLI interface. As it is a Java application and it loads various classes, sometimes a single instance can use up to 300MB of RAM when running. After a while, too many such instances will remain hanging, and the EC2 instance will remain without memory.
Is there any way to kill an interactive shell inside Docker when the SSH session that started the command is killed?
So far I tried:
- Running the tool without the
-ioption, but this won’t allow user inputs - Running the tool without the
-toption, this generally works, but it doesn’t fix the problem. The process still remains running inside Docker after closing the SSH connection to the host.