Introducing Docker Model Runner

Announcements
1

Docker Model Runner is a beta feature in Docker Desktop that allows you to pull, run, and manage AI models directly from Docker Hub. It was introduced in Docker Desktop 4.40 and is currently available for Docker Desktop for Mac with Apple Silicon.

With Docker Model Runner, you can:

  • Pull AI models from Docker Hub (from the public Docker Hub namespace of ai)

  • Run AI models directly from the command line

  • Manage local models (add, list, remove)

  • Interact with models using a submitted prompt or in chat mode

Models are pulled from Docker Hub the first time they’re used and stored locally. They’re loaded into memory only at runtime when a request is made, and unloaded when not in use to optimize resources. Since models can be large, the initial pull may take some time, but after that, they’re cached locally for faster access.

You can interact with models using OpenAI-compatible APIs, making it easy to integrate with existing applications.

Basic Commands:

docker model status # Check if Model Runner is active
docker model pull # Download a model from Docker Hub
docker model list # List locally available models
docker model run # Run a model interactively or with a prompt
docker model rm # Remove a downloaded model

Getting Started

To enable Docker Model Runner, you need to:

  1. Navigate to the “Features in development” tab in Docker Desktop settings
  2. Under the “Experimental features” tab, select “Access experimental features”
  3. Select “Apply and restart”
  4. Reopen Docker Desktop
  5. Go to “Settings” > “Features in development”
  6. From the “Beta” tab, check “Enable Docker Model Runner”

Anouncement Blog: https://www.docker.com/blog/introducing-docker-model-runner/

Docs: Docker Model Runner | Docker Docs

YouTube: https://www.youtube.com/watch?v=zQi-8mCTNf8