How to keep a container running?

General
,
1

I know this isn’t an actual docker way. But, I need to convert a custom distro into a container. So, far I was able to run the container and bash into the container and execute a script. As long as I left the console container terminates. Here is the process for the container creation from a Raspbx .img file.
After mounting the image:

mkdir rootfs raspbxfdisk -l -u=sectors /home/pi/raspbx-10-10-2020.img
mount -o loop,offset=272630784 /home/pi/raspbx-10-10-2020.img /rootfs

Then,

cp -a /rootfs/. /raspbx/
nano /raspbx/run/startup.sh
chmod +x /run/*

tar -C raspbx -c . | docker import - raspbx

After this I was able to run the container using:

docker run --name=raspbx -it --privileged --restart unless-stopped raspbx bash
cd /run
./startup.sh

At this point, script runs fine and I can access the webui. But, the container stops if I exit the shell.So, I tried

So, I tried

docker run --name=raspbx --net=macvlan_network --ip=192.168.188.27 -itd --privileged --restart no --entrypoint=/bin/bash raspbx

This time the script executes , but as soon as the container finishes executing the script it stops working. Same result with Dockerfile:

FROM raspbx

ENTRYPOINT ["sh", "/run/startup.sh"]

EXPOSE 80 3306 5060 5061 5160 5161 4569 10000-20000/udp

And

FROM raspbx

ENTRYPOINT ["bash", "/run/startup.sh"]

EXPOSE 80 3306 5060 5061 5160 5161 4569 10000-20000/udp

The script is:

#!/bin/bash -x

#https://docs.docker.com/engine/admin/multi-service_container/

/etc/init.d/mysql start
status=$?
if [ $status -ne 0 ]; then
  echo "Failed to start mysql: $status"
  exit $status
fi

fwconsole start
if [ $status -ne 0 ]; then
  echo "Failed to start fwconsole: $status"
  exit $status
fi


#restore backup if exists
if [ -f /backup/new.tgz ]; then
  echo "Restoring backup from /backup/new.tgz"
  php /var/www/html/admin/modules/backup/bin/restore.php --items=all --restore=/backup/new.tgz
  echo "Done"
fi
#restart freepbx to load everything fine after restoring backup
fwconsole stop
if [ $status -ne 0 ]; then
  echo "Failed to stop fwconsole: $status"
  exit $status
fi
fwconsole start
if [ $status -ne 0 ]; then
  echo "Failed to start fwconsole: $status"
  exit $status
fi


/etc/init.d/apache2 start
status=$?
if [ $status -ne 0 ]; then
  echo "Failed to start apache2: $status"
  exit $status
fi

/run/backup.sh &
status=$?
if [ $status -ne 0 ]; then
  echo "Failed to start backup.sh: $status"
  exit $status
fi

/run/delete-old-recordings.sh &
status=$?
if [ $status -ne 0 ]; then
  echo "Failed to start delete-old-recordings: $status"
  exit $status
fi

while /bin/true; do
  ps aux |grep mysqld |grep -q -v grep
  MYSQL_STATUS=$?
  ps aux |grep asterisk |grep -q -v grep
  ASTERISK_STATUS=$?
  ps aux |grep '/run/backup.sh' |grep -q -v grep
  BACKUPSCRIPT_STATUS=$?

  echo "Checking running processes..."
  if [ $MYSQL_STATUS -ne 0 -o $ASTERISK_STATUS -ne 0 -o $BACKUPSCRIPT_STATUS -ne 0 ]; then
    echo "One of the processes has already exited."
    exit -1
  fi
  echo "OK"
  sleep 60
done

The only way I was able to keep it alive more then 24hrs is :

docker run --name=raspbx --net=macvlan_network --ip=192.168.188.27 -itd --privileged --restart no --entrypoint=/bin/bash raspbx -c 'sleep inf'

So, how do I keep the container keep alive, any help will be highly appreciated.

2

When you use

docker run ... raspbx bash

then bash will have PID 1 and the container is alive as long as bash is running. When you exit bash, then bash stops, so the container does not have any process and stops too even if you run another process with docker exec.

With your second solution

docker run ... -itd --entrypoint=/bin/bash raspbx

Bash should have run indefinitely because of the interactive (-i) and detached (-d) mode without executing the startup script. So my question is how did you execute the startup script in this case? What do you mean by “as soon as the container finishes the script”? What script?

Changing the entrypoint won’t help if the final command resulted by the SHELL, ENTRYPOINT and CMD directives does not keep the container running. In your case it is “/run/startup.sh”. If I am not mistaken startup.sh runs some process in the background and then as a final step runs process status checking in a loop in every minute. It terminates only if one of the background processes exited so grep returnes an error code. Since one terminated process is enough to stop the loop if not all of the background processes run forever, the startup script will not be able to run forever. This means the process with PID 1 will disappear and the container stops.

Using “sleep inf” as a command can help but then you won’t be able to handle signals with the “trap” command to terminate the container properly since signals would be cought only when sleep inf terminates. Which will not happen. Adding “–init” to the docker run command might help. Then “tini” will function as the init process with PID 1 and executes the startup script and forwards the stop signal (SIGTERM) to the sleep command.

Although, it will not solve your original problem if you run sleep as a docker command. You would have to use docker exec to exeute the startup script and that process would not be under “tini”. Stopping the container could damage your database.

I have a demo script to show how you can handle signals and waiting for a process in the background, but It is only for one process.

#!/bin/bash

set -m

echo "HELLO" > "/usr/local/apache2/htdocs/index.html"

httpd -D FOREGROUND & pid=$!

for signal in TERM USR1 HUP INT; do
  trap "echo SIGNAL: $signal; kill -s $signal $pid" $signal
done

# USR2 converted to WINCH
trap "kill -s WINCH $pid" USR2

status=999
while true; do
  if (( $status <= 128 )); then
    # Status codes larger than 128 indicates a trapped signal terminated the wait command (128 + SIGNAL).
    # In any other case we can stop the loop. 
    break
  fi
  wait -f
  status=$?
  echo exit status: $status
done

The key is the wait command running in a loop.

You could modify it to handle multiple processes but I recommend using supervisor instead. You can read about it in the documentation:
Run multiple services in a container

You could also use systemd if that is absolutely necessary but It is not recommended and much more difficult to do it right. I use that only for testing installations in containers instead of using virtual machines.

So my final advise is to use supervisor and you don’t have to deal with catching and forwarding signals, which is important if you don’t like loosing data :slight_smile:

3

Thank you very much for tip, I will try to implement “supervisor” and get back as soon I can. :grinning:. I did tried to implement bash job control. But, failed due to scripting error, seems like more work is needed.

For this, I set -it --entrypoint=/bin/bash container start executing the bash script. Then as you mentioned the container fails because of an error code.

4

OK, but it is just setting the entrypoint and I didn’t see setting any command. Even if the command was specified in the Dockerfile, overriding the entrypoint in command line would empty the CMD directive.

5

Sorry, that was a mistake. It was supposed to be --entrypoint=/run/startup.sh for the final container.

6

Sorry, for the delay. The startup.sh worked. It failed previously because it was missing backup.sh, delete-old-recordings.sh . After I add the scripts to the image during the image, it worked. Then I tried to run the container using supervisors. But, it failed,

[supervisord]
nodaemon=true

[program:mariadb]
command=/etc/init.d/mysql start
autorestart=true
user=root
stdout_logfile=/dev/fd/1
stdout_logfile_maxbytes=0
redirect_stderr=true

[program:freePBX]
supervisorctl start fwconsole
autorestart=true
stdout_logfile=/dev/fd/1
stdout_logfile_maxbytes=0
redirect_stderr=true

[program:apache]
command=apache2ctl -DFOREGROUND
autorestart=true
killasgroup=true
stopasgroup=true

Apache works, I can access the webui.But, fwconsole (FreePBX) shows parshing errors and MySQL shows:

2021-10-18 19:24:18,245 INFO success: mariadb entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
,2021-10-18 19:24:20,337 INFO exited: mariadb (exit status 2; expected)
,2021-10-18 19:24:21,345 INFO spawned: 'mariadb' with pid 1706
,2021-10-18 19:24:21 0 [Note] /usr/sbin/mysqld (mysqld 10.3.23-MariaDB-0+deb10u1) starting as process 1706 ...
,2021-10-18 19:24:22,433 INFO success: mariadb entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
,2021-10-18 19:24:24,378 INFO exited: mariadb (exit status 2; expected)
7

Well, the parsing error is obvious. You need to use “command=/your/command/path” but you used “supervisorctl …” which does not make sense since the whole configuration file is for the supervisord and supervisorctl.
And mysql command is wrong. I am not sure what causes teh error code but the init.d scripts are to start a daemon and you don’t want it with supervisor. Use “/usr/sbin/mysqld” and you probably need to change the user from root to mysql. At least I had to when I tried in a mariadb Docker container. You can also try to use the entrypoint script from the source code of the MariadB Docker image. It might not work but if it does, this is the supervisor command you need:

command=/usr/local/bin/docker-entrypoint.sh mysqld