Docker-in-Docker(DIND) Docker client is not initialized

General
,
1

Hello, how do I make docker in docker work?

Im currently trying to make docker work inside a docker deployment, the reason is that my structure looks like this:

meetingsd-headless

This is a C++ SDK that can be called using docker compose up to execute, it also has a Dockerfile and docker-compose

app.py

Responsible for endpoint which when triggered would call subprocess docker compose up

Dockerfile

Using this to wrap and deploy to pipeline

docker-compose.yml

Here is my current Dockerfile:

# Use the official Python image from the Docker Hub
FROM python:3.9

# Install Docker CLI
RUN apt-get update && \
    apt-get install -y docker.io && \
    rm -rf /var/lib/apt/lists/*

# Set the working directory
WORKDIR /app

# Copy the requirements.txt file and install the dependencies
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt

# Copy the rest of the application code
COPY . .

# Expose the port the app runs on
EXPOSE 5007

# Run the application
CMD ["python", "app_multi.py"]

here is my docker-compose:

version: '3.8'

services:
  dind:
    image: docker:24.0.9-dind  # Use the latest Docker-in-Docker image
    privileged: true
    environment:
      - DOCKER_TLS_CERTDIR=/certs
      - DOCKER_HOST=tcp://dind:2375  # Set the Docker host to the DinD service

    volumes:
      - dind-certs-ca:/certs/ca
      - dind-certs-client:/certs/client
      - /var/run/docker.sock:/var/run/docker.sock  # Share the Docker socket with the host

  flask_app:
    build: .
    ports:
      - "5007:5007"
    volumes:
      - .:/app
    depends_on:
      - dind
    environment:
      - GOOGLE_APPLICATION_CREDENTIALS=/app/credentials.json  # Adjust the path accordingly
      - DOCKER_HOST=tcp://dind:2375  # Set the Docker host to the DinD service
      - DOCKER_CERT_PATH=/certs/client
      - DOCKER_TLS_VERIFY=1
    volumes:
      - dind-certs-client:/certs/client


volumes:
  dind-certs-ca:
  dind-certs-client:

Heres some important parts of my app.py which when dockerized is supposed to create multiple docker instances:

  1. Heres where I setup the docker client:
print("DOCKER_HOST:", os.environ.get("DOCKER_HOST"))
docker_instantiate = None
# Debugging: Verify Docker client connection
try:
    client = docker.DockerClient(base_url=os.environ.get("DOCKER_HOST"))
    print("Docker client initialized successfully.")
except Exception as e:
    print(f"Error initializing Docker client: {e}")
    docker_instatniate = e
    client = None
  1. Here how I use it to docker compose up the C++ SDK that I mentioned:
            # Start the Docker container
            process = subprocess.Popen(
                ["docker", "compose", "-f", "meetingsdk-headless/compose.yaml", "up"],
            )
            container_name = 'meetingsdk-headless-zoomsdk-1'  # The name of your Docker container
            if wait_for_container_to_start(container_name):
                meetings[meeting_id]['status'] = 'running'
                thread = threading.Thread(target=monitor_docker_logs, args=(container_name, meeting_id))
                thread.start()
            process.wait()

But im currently stuck at Docker client is not initialized even when I already set DOCKER_HOST and even tried hardcoding the DOCKER_HOST string to base_url.

ANy ideas how I could make this work? I am a begginer at docker.

2

What did the exception say?

My guess:

Docker in Docker means Docker Daemon inside a Docker container. So you wouldn’t need to mount a socket file from the host into the container. That socket will be created inside the container.