Docker stack deploy "No such image..." when deploying using DOCKER_HOST from GH runner

Docker Engine Swarm
1

i’m trying to deploying a test application remotely from a gh action in a docker swarm single node enviroment (my local machine) using docker stack deploy with a docker-compose file, the problem is that if a simply run the gh action with only docker stack deploy --prune --with-registry-auth --resolve-image=always --compose-file docker-compose.yml coolapp the image ghcr.io/darkimage/web_demo that is on the ghcr.io private registry doesn’t get pulled from it even specifying --with-registry-auth (both the gh runner and my local machine have the credential to access it docker login) resulting in the container not starting with logs saying No such image: ghcr.io/darkimage/web_demo@latest:SHA.... i worked around it by running docker-compose -f docker-compose.yml pull before the docker stack deploy command, is this actually needed to get docker stack deploy working? i feel like it should do it by itself!

thank you in advance.

here are my files
docker-compose.yaml

version: "3.7"

services:
  db:
    image: postgres:alpine3.16
    volumes:
      - db:/var/lib/postgresql/data
    environment:
      - POSTGRES_DB=postgres
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
  web:
    image: "ghcr.io/darkimage/web_demo"
    command: python ./manage.py runserver 0.0.0.0:8080
    volumes:
      - code:/code
    ports:
      - target: 8080
        published: 8080
        mode: host
    environment:
      - POSTGRES_NAME=postgres
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
    depends_on:
      - db
volumes:
  db:
    external: true
  code:
    external: true

GH_action.yaml

name: publish
on: [push]
run-name: ${{ github.actor }} is deploying a stack to a docker swarm 🚀
jobs:
  update-stack-swarm:
    # needs: publish-docker-image
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Login to GitHub Container Registry
        uses: docker/login-action@v1
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Install SSH Key
        uses: shimataro/ssh-key-action@v2
        with:
          key: ${{ secrets.SSH_PRIVATE_KEY }}
          known_hosts: 'just-a-placeholder-so-we-dont-get-errors'
      - run: cd ${{env.GITHUB_WORKSPACE}}

      - name: Adding Known Hosts Variable
        run: ssh-keyscan -p ${{ secrets.SSH_PORT }} -H ${{ secrets.SSH_HOST }} > ~/.ssh/known_hosts

      - name: Adding private key
        run: eval $(ssh-agent) \
          ssh-add "~/.ssh/id_rsa"
      - name: Docker version
        run: docker version

      - name: swarm-deploy
        run: docker-compose -f docker-compose.yml pull && docker stack deploy --prune --with-registry-auth --resolve-image=always --compose-file docker-compose.yml coolapp
        env:
          DOCKER_HOST: 'ssh://${{secrets.SSH_USER}}@${{secrets.SSH_HOST}}:${{ secrets.SSH_PORT }}'

docker version gh runner

Client:
 Version:           20.10.18+azure-1
 API version:       1.41
 Go version:        go1.18.6
 Git commit:        b40c2f6b5deeb11ac6c485c940865ee40664f0f0
 Built:             Thu Sep  8 08:19:02 UTC 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.18+azure-1
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.6
  Git commit:       e42327a6d3c55ceda3bd5475be7aae6036d02db3
  Built:            Thu Sep  8 22:50:10 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.5.13+azure-1
  GitCommit:        a17ec496a95e55601607ca50828147e8ccaeebf1
 runc:
  Version:          1.1.4
  GitCommit:        5fd4c4d144137e991c4acebb2146ab1483a97925
 docker-init:
  Version:          0.19.0
  GitCommit:

docker version of my local machine (host)

Client:
 Cloud integration: v1.0.28
 Version:           20.10.17
 API version:       1.41
 Go version:        go1.17.11
 Git commit:        100c701
 Built:             Mon Jun  6 23:09:02 2022
 OS/Arch:           windows/amd64
 Context:           default
 Experimental:      true

Server: Docker Desktop 4.11.1 (84025)
 Engine:
  Version:          20.10.17
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.17.11
  Git commit:       a89b842
  Built:            Mon Jun  6 23:01:23 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.6
  GitCommit:        10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
 runc:
  Version:          1.1.2
  GitCommit:        v1.1.2-0-ga916309
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
2

I can not speak for the behavior inside the github action, but as long as a login to the registry was made and the argument --with-registry-auth was used, swarm will pull the images by itself. I have never used --resolve-image and swarm still deploys the most recent image for the specified tag.

You might want to check if setting an explicit tag for the web_demo changes things.

3

i tried also specifing :latest but it still doesn’t pull the image from the ghcr.io registry but i should also mention that the postgres image from dockerhub is pulled correctly each time.
I will try with a specific tag and report the result here.

4

tested with this changes

  • added
  publish-docker-image:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Login to GitHub Container Registry
        uses: docker/login-action@v1
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      - name: Build the web-demo Docker image 🏭
        run: |
          docker build . --tag ghcr.io/darkimage/web_demo:1.0
          docker push ghcr.io/darkimage/web_demo:1.0

to publish a docker image with 1.0 tag

  • removed docker-compose -f docker-compose.yml pull && from GH_action.yaml

  • added image: "ghcr.io/darkimage/web_demo:1.0" to docker-compose.yaml

but it still error with this (using docker service ps --no-trunc coolapp_web)

pyi5f2umlokeoo529al5lq54e   coolapp_web.1       ghcr.io/darkimage/web_demo:1.0@sha256:493081070f17be873064d8241fa50c080e26c8352720b59c097e9adc6158ed0b   docker-desktop   Ready           Rejected 2 seconds ago    "No such image: ghcr.io/darkimage/web_demo:1.0@sha256:493081070f17be873064d8241fa50c080e26c8352720b59c097e9adc6158ed0b"