Specify a path for Docker registry mirror

,

I would like to use AWS ECR’s pull-through cache to avoid unnecessary data ingress, and then have Docker configured on the CI runners in such a way as to use the pull-through cache while still allowing CI steps to run docker run postgres:16 without explicit reference to the ECR pull-through cache.

It seems to me that this requires two mirrors due to the special handling of short aliases (i.e., ‘official’ Docker Hub images):

  • docker pull $image:$tag needs to map to $ecr/docker-hub/library/$image:$ag
  • docker pull $repo/$image:$tag needs to map to $ecr/docker-hub/$repo/$image:$tag

Where $ecr refers to the $account_id.dkr.ecr.$region.amazonaws.com.

If Docker read of /etc/containers/registries.conf, this should be straightforward:

[[registry]]
location = "docker.io"
[[registry.mirror]]
location = "$ecr/docker-hub/library"
[[registry.mirror]]
location = "$ecr/docker-hub"

Unfortunately, Docker only allows for a global registry-mirrors option, and it doesn’t even support paths. So while I can point it directly to https://$ecr, this won’t work as it will miss the docker-hub scope. Docker also does not insert an automatic scope (e.g., registry-1.docker.io, docker.io). See for example these logs which show the wrong scope:

> docker pull node:23
Trying to pull node from https://$ecr/"
Attempting next endpoint for pull after error: Head "https://$ecr/v2/library/node/manifests/23": no basic auth credentials"
                                                                 ^ missing scope here

I’ve seen the registry-mirrors configuration set up for a local registry mirror, but this unfortunately doesn’t work for me as the CI runners are ephemeral. Hence the desire to use ECR as a long-lived cache.

Did I miss something obvious when trying to configure Docker to use AWS ECR’s pull-through cache?


PS: I did see a question from Jan 2024 (https://forums.docker.com/t/139261) but it remains unanswered.

If you think this is a bug or a relevant feature request, you could post it to Moby Github.