I would like to use AWS ECR’s pull-through cache to avoid unnecessary data ingress, and then have Docker configured on the CI runners in such a way as to use the pull-through cache while still allowing CI steps to run docker run postgres:16
without explicit reference to the ECR pull-through cache.
It seems to me that this requires two mirrors due to the special handling of short aliases (i.e., ‘official’ Docker Hub images):
docker pull $image:$tag
needs to map to$ecr/docker-hub/library/$image:$ag
docker pull $repo/$image:$tag
needs to map to$ecr/docker-hub/$repo/$image:$tag
Where $ecr
refers to the $account_id.dkr.ecr.$region.amazonaws.com
.
If Docker read of /etc/containers/registries.conf
, this should be straightforward:
[[registry]]
location = "docker.io"
[[registry.mirror]]
location = "$ecr/docker-hub/library"
[[registry.mirror]]
location = "$ecr/docker-hub"
Unfortunately, Docker only allows for a global registry-mirrors
option, and it doesn’t even support paths. So while I can point it directly to https://$ecr
, this won’t work as it will miss the docker-hub
scope. Docker also does not insert an automatic scope (e.g., registry-1.docker.io
, docker.io
). See for example these logs which show the wrong scope:
> docker pull node:23
Trying to pull node from https://$ecr/"
Attempting next endpoint for pull after error: Head "https://$ecr/v2/library/node/manifests/23": no basic auth credentials"
^ missing scope here
I’ve seen the registry-mirrors
configuration set up for a local registry mirror, but this unfortunately doesn’t work for me as the CI runners are ephemeral. Hence the desire to use ECR as a long-lived cache.
Did I miss something obvious when trying to configure Docker to use AWS ECR’s pull-through cache?
PS: I did see a question from Jan 2024 (https://forums.docker.com/t/139261
) but it remains unanswered.