Difference between Docker Desktop and Docker Engine

General
1

Hi,

Good day!

Docker newbie here, I just want to ask what is the difference between Docker Desktop and Docker Engine?

3 Likes
2

Interesting. Recently I wanted to talk about it, because I feel it is not communicated well enough.
Basically Docker Desktop is a virtual machine + Graphical user interface with some extra features like the new extensions and running a single-node Kubernetes ā€œclusterā€ easily. Inside the virtual machine there is Docker CE (Docker Community Edition) daemon. The Docker documentation says:

Docker Engine is an open source containerization technology for building and containerizing your applications. Docker Engine acts as a client-server application with:

  • A server with a long-running daemon process dockerd .
  • APIs which specify interfaces that programs can use to talk to and instruct the Docker daemon.
  • A command line interface (CLI) client docker .

So the client, the API and the deamon is also part of Docker Engine. In case of Docker Desktop, the daemon is inside the virtual machine, but the client is on your host machine.

Keep it in mind when you try to access container IP addresses directly, because the Docker network exists only inside the virtual machine, not on your host. This is also true for local volumes too. The other very important difference is related to the network is that even if you use the host network mode (docker run --net host) it will be the host network of the virtual machine, not your actual physical host.

I could continue but you really need to remember only one thing. Docker containers are running inside the virtual machine. Everything else is a consequence. Docker Desktop just tries to hide that fact from you so you donā€™t need to setup a virtual machine and the client-server connection.

Having a virtual machine is required on Windows and MacOS if you want to run Linux containers. On Linux, virtual machine is not necessary, but Docker wants to give you the same experience with the same Linux kernel on each platform. So if you install Docker Desktop for Linux, you will have a virtual machine there too.

8 Likes
3

Also a newbie, wondering if this is why PyCharm canā€™t see the images and containers I made in Docker Desktop. Because itā€™s siloed away in a VM? Thank for any help!

4

Not likely. It is more likely that you need to fix your configuration. Pycharm or other Docker clients donā€™t care where Docker stores its data. It will use the TCP socket or unix socket to communicate. IN case of PyCharm, you need to click on ā€œServicesā€ at the bottom of the IDE, then right-click on the whale (Docker logo), and choose Edit configuration. Then you can select Docker for Mac (if you are on Mac) or probably Docker for Windows in case of Windows. If you for example tried with an SSH connection or TCP connection or something that Windows has and Mac doesnā€™t, just correct the configuration and it should work.

Screenshot 2022-07-08 at 15.43.37
Screenshot 2022-07-08 at 15.43.371690Ɨ894 117 KB

1 Like
5

Thank you! Thatā€™s helpful. Iā€™m on Linux, and indeed Iā€™d configured PyCharm correctly as far as I can tell. However, Iā€™ve also had some IP address conflicts and was trying both Engine and Desktop, so Iā€™ll see if my fixes solve both issues.

6

Well, I had to uninstall Docker Desktop on my linux, because it changed my configuration and somehow broke the authentication and I also had some version problem about a month ago or something like that. So I canā€™t try it on Linux, but if it is not that you have Docker Desktop and the engine on your Linux at the same time, then it is a bug. PyCharm should be able to see your images.

1 Like
7

See here Frequently asked questions for Linux for the details

8

What do you mean by ā€œIn case of Docker Desktopā€? Did you mean:

  • if you install Docker via Docker Desktop Installer?
  • if you start Docker Engine via Docker Desktop?

And if itā€™s not the case of Docker Desktop, what would that statement be?

Iā€™m using Windows, if that matters. My purpose is to understand the difference between engine and daemon, and see if I can run the engine/daemon without opening Docker Desktop

9

The docker engine is the daemon. Docker Desktop comes with a convenient UI to allow developers to start/stop Docker Desktop (and with it the backend that runs inside the utility vm that runs the docker engine). It is not meant to be used for 24/7 operations.

To run Linux containers, the docker engine must be run on a Linux system, as docker depends on Linux kernel, libraries, and tools. You can install docker-ce on any supported Linux system / architecture. On Windows you would need to create a Linux vm, and install docke-ce following the installation instructions. It is possible to run docke-ce in a WSL2 distribution as long as the OS / architecture is in the support matrix and systemd is enabled for the WSL2 distribution.

2 Likes
10

Iā€™m confused. So is it that the engine and the daemon are identical, or is it that the engine has 3 parts: the CLI client, the API and the daemon?

Basically Docker Desktop is a virtual machine + Graphical user interface with some extra features like the new extensions and running a single-node Kubernetes ā€œclusterā€ easily

What is the technology behind that virtual machine? Is it the same with VMWare? What OS does it use?

11

Technically the engine can run without the cli-client being installed. Furthermore, the cli-client can be installed on a system without the engine and use it with a remote engine. The cli-client is nothing than a user interface that interacts with the api endpoints of a docker engine.

Since you are using Windows, the backend for Docker Desktop can either run on Hyper-V or WSL2 (prefered)

You are not supposed to work in the utility vm that runs the Docker Desktop backend. From the user perspective itā€™s an appliance controlled by the Docker Desktop frontend application.

1 Like
12

So is it correct that technically this statement is incorrect: ā€œThe client, the API and the daemon are parts of Docker Engineā€? That while the engine should be understood as the daemon only, people usually refer the engine as the larger thing?

You are not supposed to work in the utility vm that runs the Docker Desktop backend.

I know, but for the sake of learning docker I still want to know more. It will help answer questions like ā€œif I want to run a container, would it be better to run it via Docker Desktop, or via Docker CE via WSL?ā€

13

It was indeed my understanding that the engine is the daemon (including the api it implements). For me the client is not relevant for the docker engine. But apparently I was wrong: according to the docs the cli-client is part of the definition. The api seems to be mentioned to illustrate that the api version are aligned for the daemon (implements api provider) and cli-client (implements api consumer) for docker engine release versions. Daemon and cli-client support a range of api versions. The daemon by calling the versioned endpoint, the cli-client by setting an environment variable. 3rd party management tools, like Portain, just use the versioned api from daemon to interact with it - otherwise each daemon version would require a specific version of the 3rd party mangement tool.

With Docker Desktop on WLS2, the ā€œutility vmā€ is composed of the shared lightweight WSL2 vm (which provides the kernel) and an Alpine-based docker-desktop distribution running on the WSL2 vm. Additionally, there is a docker-desktop-data distribution to store persistent data.

1 Like
14

Most of your questions were answered by @meyay so some additional info.

The documentation of Docker Desktop contains information about the virtualization solutions, but I admit I couldnā€™t find all the info in one place. Since all operating systems are different and different vitualization technologies are supported on them Docker Desktop uses what it can. On Windows, it is either HyperV or WSL2, but on Linux it is qemu kvm. On macOS there are two ways, but on recent macOS the ā€œVirtualization frameworkā€ can be used and it is required when VirtioFS is enabled, which is recommended.

Because it is confusing indeed. I already noted it to some people at Docker. The documentation indeed says:

Docker Engine acts as a client-server application with:

  • A server with a long-running daemon process dockerd.
  • APIs which specify interfaces that programs can use to talk to and instruct the Docker daemon.
  • A command line interface (CLI) client docker.

So it includes the CLI, but if you check the Docker Desktop overview, it mentiones the Engine and the CLI although the CLI is not linked:

Whatā€™s included in Docker Desktop?

You can also install the client without daemon even though it is part of the Engine. I guess the reason is something historical.

As Metin wrote, it is Alpine-based. LinuxKit: GitHub - linuxkit/linuxkit: A toolkit for building secure, portable and lean operating systems for containers

If you want to know more about what is inside, you can read a blogpost about a video I made and also watch the video:

Since then the name of the container that runs the Docker daemon inside the virtual machine has changed. It is ā€œ02-dockerā€ now and not just ā€œdockerā€. The old version also had the docker client in that container, now it is just the daemon. You donā€™t really need the client inside but it was convinient to use it for debugging.

1 Like
15

Thank you for this. I have been struggling for hours.
we should have a place to upvote these answers.

1 Like
16

It is not an upvote as on stackoverflow, but you can like posts by clicking on the heart icon. Iā€™m glad I could help.

3 Likes
17

I have Docker Desktop running. How can i transform this into docker engine? Because i need Docker and my containers to start automatically.

18

It is like moving applications from a server to another. Export all the data and copy it to the new server. In your case it is your host machine. Then if you installed Docker CE you just run the same commands. If you use compose, you use the same compose files. If you have Docker Desktop related parameters like using the host.docker.internal, you replace those with something else compatible with your new environment. If you have enough space, donā€™t delete Docker Desktop until you are sure the new environment works correctly. Until you delete Docker Desktop, make sure you are using the default context when working with containers or run all commands as root (on linux).

19

Hi. Thanks for your help.
But I am a bit lostā€¦

I have, installed, the latest version of docker desktop (4.34.1). (Windows 11, linux, ubuntu, ā€¦)
So with that, docker engine is installed too. (In the lower left corner of the docker desktop ā€œcontainersā€ screen you can indeed see ā€œengine runningā€).

(NB: If I go to services.msc, Docker Desktop is there but not Docker Engine: why?).

The documentation, Forums and/or the docker HUB tells me that, to be able to work in production and thus have, (what I want), when the computer restarts, a container that starts automatically, you must not use Docker Desktop but Docker Engine.

And thatā€™s what I canā€™t figure out how to do: use Docker engine.

I donā€™t really understand the concept of ā€œcontextā€: currently, with the command ā€œdocker contect lsā€, my system tells me that the default is desktop-linux as it should be.

If I eventually do the command ā€œdocker context use defaultā€, then, apparently, I could interact with docker engine, but how?

Some documentations are talking about CLI (command line interface): is that what I should use? Where and how to access it? Do you have any examples?

Best regards,
Vincent.

20

On Windows you canā€™t run Linux containers natively. You always need a VM. In this case, Docker Desktop. That will have the engine inside.

Because it connects to the engineā€™s endpoint not Docker Desktop. But now you are talking about Linux againā€¦

You are using the CLI already. That is the docker command.

It seems you need a lot more info and learning to be able to understand us.

Recommended links to learn the basics and concepts: