Can't start containers with GPU access on Linux Mint

Docker Desktop
1

When I run

docker run --runtime=nvidia --gpus all nvcr.io/nvidia/k8s/cuda-sample:nbody

I get this error

docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].

Environment:

  • OS: Linux Mint 22 Cinnamon:6.2.9
  • Linux Kernel: 6.8.0-47-generic
  • Graphics card: NVIDIA Corporation GA104 [GeForce RTX 3060 Ti Lite Hash Rate]
  • Docker Desktop version: 4.34.3 (170107)
  • Docker version: 27.3.1, build ce12230

Steps to reproduce:

  1. Install Docker Desktop (file docker-desktop-amd64.deb) using debian installer
  2. Follow the steps at Installing the NVIDIA Container Toolkit Using the apt method.
  3. Manually update the config because docker desktop doesn’t use the config at /etc/docker instead it uses the one at /home/username/.docker which the steps at NVidia - Configuring Docker modify.
  4. Run docker run --runtime=nvidia --gpus all nvcr.io/nvidia/k8s/cuda-sample:nbody to confirm functionality.
    (these steps are to the best of my memory…)

My docker config now looks like this:

{
  "builder": {
    "gc": {
      "defaultKeepStorage": "20GB",
      "enabled": true
    }
  },
  "experimental": false,
  "runtimes": {
    "nvidia": {
      "args": [],
      "path": "nvidia-container-runtime"
    }
  }
}

Expected:
Starts the container and I see gpu stats in the logs

Actual:
Get an error in the terminal saying docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].

I’ve searched high and low for an answer to this, but any advice or solutions are extremely welcome. Not even LLMs can answer my question :sunglasses: :sweat_smile:

3

Only Docker Desktop for Windows supports GPUs when the WSL2 backend is used, since WSL2 supports GPUs. Otherwise you have a virtual machine without GPUs. So Docker Desktop for Linux does not support it, but on Linux, you can install Docker CE which does.

By the way as far as I know, you don’t need the runtime option when you use the gpus option since Docker CE 19.03. The nvidia container runtime project is archived

4

Thanks for the feedback @rimelek . I’ve got it working re your advice that docker desktop on linux doesn’t work for GPU related workloads, and instead to use Docker CE (a.k.a Docker Engine). It’s a real shame because I kinda loved that UI. I’m now using VSCode and the Docker plugin to manage my containers. I had to add my user to the docker group, but it’s working and I’m happy.

To make a note of what I did to get this working, I’ll leave this for posterity:

  • Uninstalled Docker Desktop completely before installing Docker CE
sudo apt remove docker-desktop
sudo apt purge docker-desktop
sudo rm -rf ~/.docker #plus any other places there's docker config
  • Removed any leftover packages that weren’t used elsewhere, just in case
sudo apt remove docker docker-engine docker.io containerd runc
sudo apt update
  • Manually removed Docker’s network settings by removing the relevant configurations, basically any daemon.json files that were hanging around my system.
  • Installed Docker CE with these commands
sudo apt install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu noble stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io

(“noble” is my UBUNTU_CODENAME from /etc/os-release)
(The first apt install here might not be necessary but it was the steps I followed)

sudo docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

…and finally got the sexy output:

Fri Oct 25 22:53:04 2024       
------------------------------------------------------------------------------------------
| NVIDIA-SMI 550.107.02             Driver Version: 550.107.02     CUDA Version: 12.4     |
|-----------------------------------------------------------------------------------------
| Extra stuff went here
------------------------------------------------------------------------------------------

I’ll consider this a moderate success. I’m going to miss that UI xD. Thanks for the guidance @rimelek !

5

Thank you for providing detailed instructions. I was able to make it work with one adjustment

  1. sudo vim /etc/nvidia-container-runtime/config.toml, then changed no-cgroups = false, save
  2. Restart docker daemon: sudo systemctl restart docker, then you can test by running sudo docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi