New Docker User - What am I doing wrong?

Docker Desktop
1

Hello, I am a new Docker user. I’m trying to get Docker set up on an Ubuntu host and run an image that uses my NVIDIA GPU, but am getting errors that I can’t seem to find a way to resolve.

Host: Ubuntu 24.04.01 (edit - corrected version, typo before)

Installed docker desktop for Linux following these instructions Ubuntu | Docker Docs, and I get

Docker version 27.3.1, build ce12230

I also installed the NVIDIA Container Toolkit following Installing the NVIDIA Container Toolkit — NVIDIA Container Toolkit 1.16.2 documentation

If I then run

sudo docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

I get

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.107.02             Driver Version: 550.107.02     CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4060 Ti     Off |   00000000:01:00.0  On |                  N/A |
| 30%   42C    P2             36W /  165W |    1403MiB /  16380MiB |     56%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

But when I try to run a container image, I get the following error in the log

RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from…

The container image I’m trying to run is aidockorg/comfyui-cuda:latest

Any help for a new user would be very much appreciated! Thank you

2

Just making sure - Are you really running on old ubuntu 12.04.01, or is that a typo for 22.04.01?

Also, since you’re on Linux, you should probably install Docker Engine rather than Docker Desktop

Docker Desktop sets up a Linux VM with Docker Engine running on it, since you’re already on Linux, that would just impact your performance and limit your capabilities

3

Ah, yes that was a typo - corrected. Okay, I can try removing everything and trying Docker Engine, thanks for the suggestion

4

I quote the relevant part from the docs

  1. Set up Docker’s package repository. See step one of Install using the apt repository.

It mentions only “Step 1”. Not to install docker-ce or docker-ce-cli. Now you have Docker CE and Docker Desktop, and you access the Docker CE sockt when you use sudo So that is the second problem. you should nver use sudo when using Docker Desktop.

First of all, you run a container, not an image. For some reason even some documentation mentions running an image which must be very confusing.

And second, you already ran a container and you wrote it worked:

So I’m confused too. By the way as far as I know, you don’t need to specify --runtime=nvidia when using --gpus all

How exactly did you try to run the container based on the other image? My guess would be that you ran it without root and Docker Desktop will not support the GPU on Linux. As far as I know, it is only supported on Windows when using the WSL2 backend, since WSL2 supports the GPU.

5

or extend. :slight_smile: As I always say, it all depends on the use case. In this case, I agree, since the GPU is needed.

6

You’re right, I should clarify
It limits some things, by the nature of running within a VM, resource allocation can no longer be automatic, as-needed, but is instead fixed and adjusted manually.

It extends the capabilities by providing a large encasing toolset and a nice little GUI.
It can be started as a Kubernetes cluster, is the only current source of docker init, and other features I may not be aware of

I use it myself, unless you manually configure Docker Engine on a VM, it is mandatory for MacOS and Window (Unless using Windows containers)

I have not tried setting up the containerd store and multi-platform builder on Docker Engine, so I don’t know if it is simple too; but within Docker Desktop it is as simple as a checkbox in the settings

7

Thanks for the thoughts! I’ll give these suggestions a try soon and confirm if I’m able to get this resolved.