Crontab inside of Docker container - start vpn (IPsec) connection and jupyter notebook

General
1

Hi all,
I do realize this is more linux-related question, but maybe (just maybe) it is docker-specific. The reason I can’t test it outside of the docker container is that IPsec connection is limited to external and internal IP addresses of the docker container linux instance.

So, I have manual/crontab.txt file:

* * * * * /usr/sbin/ipsec status | /bin/grep -q 'ESTABLISHED' || /usr/sbin/ipsec restart
@reboot /usr/sbin/ipsec restart
@reboot /usr/local/bin/python3.10 -m pip install requests requests_ntlm
@reboot /usr/local/bin/python3.10 -m pip install --upgrade pip
@reboot /bin/sleep 60 && /usr/sbin/ipsec status | /bin/grep -q 'ESTABLISHED' || /usr/sbin/ipsec restart

Dockerfile:

RUN if [ ! -f "/app/manual/crontab.txt" ]; then echo && echo && echo && echo "File /manual/crontab.txt does NOT exist: the file must be created by user before running docker build -- check git repository README file, build aborted." && echo && echo && echo && exit 1; fi

RUN if [ ! -s "/app/manual/crontab.txt" ]; then echo && echo && echo && echo "File /manual/crontab.txt is empty: the file must be populated by the user before running docker build -- check git repository README file, build aborted." && echo && echo && echo && exit 1; fi

RUN apt update

RUN apt install -y cron

COPY ./manual/crontab.txt /etc/cron.d/ipsec-restart-cron
RUN chmod 0644 /etc/cron.d/ipsec-restart-cron
RUN crontab /etc/cron.d/ipsec-restart-cron
RUN touch /var/log/cron.log

I do realize that installing pip packages in crontab inside of docker container is not according to the best practise and there is a Dockerfile for that, but this is to demonstrate how I try to set it and it does work for me with pip, but somehow it does not with IPsec. The same ipsec command shown above works for me when running it manually.

Ok, actually I figured out how to workaround that in Python:

import requests
from requests_ntlm import HttpNtlmAuth
import subprocess

ipsec_status = subprocess.run(['/usr/sbin/ipsec', 'status'], capture_output=True, text=True) 
if 'ESTABLISHED' not in ipsec_status.stdout: 
    subprocess.run(['/usr/sbin/ipsec', 'restart'])
...
[the rest of my Python code]

But hey, it should not be this hard to do it properly in docker, right? It is always great to have a workaroound instead of nothing, but I do believe there are gurus here to help.

Another thing is, I need a jupyter notebook running as a service inside of my docker container. I can run it anytime manually like this:

/usr/local/bin/jupyter notebook --NotebookApp.iopub_data_rate_limit=1.0e10 --ip 0.0.0.0 --no-browser --allow-root --NotebookApp.token='some_token1'

But my less advanced in docker team mates need this as well, I don’t want to be a bottleneck in case I forgot to run it manually, so here is my attempt to do it in crontab:

@reboot /usr/local/bin/jupyter notebook --NotebookApp.iopub_data_rate_limit=1.0e10 --ip 0.0.0.0 --no-browser --allow-root --NotebookApp.token='some_token1'
*/5 * * * * /usr/bin/pgrep -f "jupyter notebook" > /dev/null || /usr/local/bin/jupyter notebook --NotebookApp.iopub_data_rate_limit=1.0e10 --ip 0.0.0.0 --no-browser --allow-root --NotebookApp.token='some_token1'

None of those crontab record worked for me.

Please advise how to do it properly.

2

Why so complicated with crontab?

Why not create a simple script that you execute with CMD when the container is started, that includes your setup commands and the “real” application to run inside the container?

3

Here is my current CMD and ipsec command doesn’t work for me. Tried to put jupyter notebook run after ipsec, it didn’t work for me either. If you have idea how to do it properly, please advise, this is why I brought it here.

CMD /bin/bash -c '\
    /usr/sbin/service cron start & \
    /usr/sbin/ipsec status | /bin/grep -q 'ESTABLISHED' || /usr/sbin/ipsec restart & \
    if [ -z "$MYSQL_HOST" ]; then \
        echo "MYSQL_HOST is not set. Exiting."; \
        exit 1; \
    elif [ ${#MYSQL_HOST} -le 1 ]; then \
        echo "Length of MYSQL_HOST is 1 or less. Exiting."; \
        exit 1; \
    else \
        echo "Length of MYSQL_HOST is ${#MYSQL_HOST} and value is $MYSQL_HOST."; \
    fi; \
    if [ -z "$MYSQL_PORT" ]; then \
        echo "MYSQL_PORT is not set. Exiting."; \
        exit 1; \
    fi; \
    if [ -z "$MYSQL_ROOT_PASSWORD" ]; then \
        echo "MYSQL_ROOT_PASSWORD is not set. Exiting."; \
        exit 1; \
    fi; \
    count=0; \
    echo "Length of MYSQL_HOST is ${#MYSQL_HOST} and value is $MYSQL_HOST."; \
    while ! mysql -h$MYSQL_HOST -P$MYSQL_PORT -uroot -p$MYSQL_ROOT_PASSWORD -e "SELECT 1" && [[ $count -lt 60 ]]; do \
        count=$((count+1)); \
        echo "waiting 5_sec for backend mysql database server ($MYSQL_HOST $MYSQL_PORT) to become available - try $count "; \
        sleep 5; \
    done; \
    if [ $count -eq 60 ]; then \
        echo "MySQL server did not become available after 5 minutes. Exiting."; \
        exit 1; \
    fi; \
    /mysql-init/init.sh && \
    echo " "; \
    echo "Begin mandatory tests"; \
    echo " "; \
    /opt/venv/bin/pytest /var/www/app/test_app.py && \
    echo " "; \
    echo "End mandatory tests"; \
    echo " "; \
    /usr/bin/supervisord -c /etc/supervisor/conf.d/supervisord.conf'
4

You can use the container friendly supercronic instead of cron.

Though, I would not recommend running multiple processes in the same container.

You might be better off to have a dedicated vpn service/container that establishes the vpn connection, and other containers that use the network namespace of the vpn service by using network_mode: "service:[service name]" (see: https://docs.docker.com/compose/compose-file/05-services/#network_mode).

Note: attaching containers to networks and publishing ports for containers that use the vpn service network namespace must be declared on the vpn service, and not the containers themselves.

1 Like