How to speed up build time

Open Source Projects DockerEngine
1

Description

I only change the cmake version on the dockerfile, but the docker build will start build from cmake step.
And the opencv is also build, even the opencv step is not changed.
Do you have idead to pass opencv build?

original

FROM ubuntu20.04 AS base
RUN apt-get install XXX

FROM base AS cmake
ENV cmake_version=3.26
RUN apt-get install cmake==cmake_version

FROM cmake AS opencv
ENV opencv_version=4.6
RUN apt-get install XXX

change the cmake version

FROM ubuntu20.04 AS base
RUN apt-get install XXX

FROM base AS cmake
ENV cmake_version=3.27
RUN apt-get install cmake==cmake_version

FROM cmake AS opencv
ENV opencv_version=4.6
RUN apt-get install XXX
2

Hello and welcome,

it is similar to building a house. If you want to change the ground floor you have to rebuild this floor and the first floor above (and all other floors above, too).
If you want to change the houseā€™s foundation you have to rebuild the complete house including its foundation.

Every command within your docker-compose.yml Dockerfile results in a new layer within your image.
Docker can reuse unchanged layers during a (re-)build-process.
A layer is considered as unchanged if the layer it is based on is unchanged and the command itself is unchanged.

As you change a layer by modifying the cmake-version to be installed all layers based on this layer have to be rebuild.

Best regards
Matthias

3

Follow your describe.
What are the different between dockerfile1 and dockerfile2 ?
Are they the same behavior for (re-)build-process on the case ?

If they are the same behavior , the dockerfile1 is the redundant method?
FROM base AS cmake
FROM cmake AS opencv

dockerfile1

FROM ubuntu20.04 AS base
RUN apt-get install XXX

FROM base AS cmake
ENV cmake_version=3.26
RUN apt-get install cmake==cmake_version

FROM cmake AS opencv
ENV opencv_version=4.6
RUN apt-get install XXX

dockerfile2

FROM ubuntu20.04 
RUN apt-get install XXX

ENV cmake_version=3.26
RUN apt-get install cmake==cmake_version

ENV opencv_version=4.6
RUN apt-get install XXX
4

Since dockerfile1 is simply a chain of stages using the same instructions, yes, dockerfile1 and dockerfile2 should have the same result, but this is something you can just try and see.

In case of anyone reads the quoted post, Iā€™m pretty sure @matthiasradde wanted to write ā€œDockerfileā€, not ā€œdocker-compose.ymlā€. :slight_smile:

5

OK, so the dockerfile1 is redundant to use the command FROM xxx AS xxx.
But this improves readability by using a chain of stages, right?

6

@rimelek Uups - thank you for the hint - it is corrected now :slight_smile:

7

I donā€™t think so. At least not for me. It indicates that there is something more happening in the app than it is. I would use stages only when I need them and for readability I would add comments and/or use multiple line breaks to separate parts from each other.

If using stages helps you to read the Dockerfile, you can still use it, there is nothing wrong with it, just make sure you donā€™t use multiple RUN instructions just so you can put them into different stages for readability. These instructions create new layers and there is a limit how many layers can be in an image (128 in case of overlayFS) and these layers need to be merged when creating a container. I wouldnā€™t create more than necessary.

And since this topic is about speeding up build time, it is worth noting that

  • Each RUN instruction creates a new container, executes the commands and deletes the container. It is very quick compared to everything else, but still takes time, although I have never felt the need of measuering it. I create my Dockerfiles in a way that I need them so I donā€™t care how many seconds it takes to handle these containers.
  • Each apt-get install command could override some dependencies that was installed in the previous layers. I donā€™t think it happens frequently, but I helped someone who had a really large Docker image, because the second apt-get install first removed a dependency which was not compatible with another, and installed its own verson. In the end you could have a perfectly working Docker image but have different variants of the same package in multiple layers which is not only increasing the size of the image, but naturally it takes time to install, remove and install packages again.
  • apt-get installnon-interactively by default installs recommended depdendencies as well even if you donā€™t need them. Again, bigger size and and more time, so usually this is how an apt-get install command should look like:
    RUN apt-get update \
 && apt-get install -y --no-install-recommends PACKAGENAME
1 Like