How can I create base image(s) in docker-compose?

General Discussions
1

I have two services which rely on Machine Learning libraries. My services code add few MBs to docker image size but ML libraries add close to 1.4 GB to image size. Both services are using same set of ML libraries.
When I’m pushing the services to docker hub it is upload 1.4GB of layers two times. I’m wondering if I should create the base image with all ML libraries and my services are based on it? I’m not sure how to achieve it in docker-compose.

My current Docker compose file looks like below:

version: '3.8'
services:

   a_service:
    image : myuser/a_service:${TAG:-latest}
    build: 
      context: ./src
      dockerfile: a_service/Dockerfile

   b_service:
    image : myuser/b_service:${TAG:-latest}
    build: 
      context: ./src
      dockerfile: b_service/Dockerfile

Dockerfile content for a_service looks like below. Dockerfile of service- b_service is same except it add b_service code to image.

FROM python:3.8-slim-buster as base

RUN apt-get update \
    && apt-get install -y --no-install-recommends apt-utils gcc build-essential\
    && rm -rf /var/lib/apt/lists/*

WORKDIR /wheels
# Install pip requirements
ADD a_service/requirements.txt .
RUN python -m pip install --upgrade pip \
    && python -m pip install -r requirements.txt \
    && pip wheel --wheel-dir=/wheels -r requirements.txt

FROM python:3.8-slim-buster

RUN mkdir -p /usr/share/man/man1
RUN apt-get update \
    && apt-get install -y --no-install-recommends ca-certificates-java\ 
    default-jre\
    && rm -rf /var/lib/apt/lists/*

ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

RUN useradd appuser

WORKDIR /app
# Only following statement is different for b_service
ADD --chown=appuser a_service ./a_service
ADD --chown=appuser _common ./_common
COPY --from=base /wheels /app/wheels

RUN pip install -U pip \
        && pip install --no-index --find-links=/app/wheels -r /app/a_service/requirements.txt \
        && rm -rf /app/wheels \
        && rm -rf /root/.cache/pip/* 


RUN chown appuser /app
USER appuser

ENV PYTHONPATH=/app
CMD ["python", "/app/a_service/entry.py"]

I’m using following workflow:

docker-compose -f docker-compose.yml up
docker-compose -f docker-compose.yml push

And on target machine

docker stack deploy -f docker-compose.yml my_stack

3

If you put all of the common items at the beginning of the Dockerfiles, then the layers could be shared. The files need to be identical to that point.

But a better solution would be to create your own base image with the common items and then use that in your compose based Dockerfiles.

So the basic answer is no, you can’t create a base image in a docker compose file, but you could create it externally and then layer your apps on that.

2 Likes
4

Yes, thanks that is what I’m doing and it is working well.