Docker-compose not found when pushing to my server 😭

[ehsang]

I have been pushing to my server using my pipeline.yml for some time with no problem. Just a week ago without changing anything on my pipeline, docker file and docker-compose file, now I can not push to production server.
Here is my Pipeline: nest-server/.github/workflows/pipeline.yml at release · SAY-DAO/nest-server · GitHub
Here is my docker-compose file: same repo link as above

This is the error:

Run ENVIRONMENT=production DOMAIN=nest.saydao.org CONTAINER_IMAGE=${IMAGE_ID} STACK_NAME=say-nest1 docker-compose -f docker-compose.production.yml config > docker-stack-say-nest1-70c411d6d934858eb1c4eb5ee796d346470ac895.yml
  ENVIRONMENT=production DOMAIN=nest.saydao.org CONTAINER_IMAGE=${IMAGE_ID} STACK_NAME=say-nest1 docker-compose -f docker-compose.production.yml config > docker-stack-say-nest1-70c411d6d934858eb1c4eb5ee796d346470ac895.yml
  shell: /usr/bin/bash -e {0}
  env:
    REGISTRY: ghcr.io
    DEVELOPMENT: dev
    STAGING: staging
    PRODUCTION: production
    MAIN_BRANCH: main
    DEV_DOMAIN: d.nest.saydao.org
    STAGING_DOMAIN: s.nest.saydao.org
    PRODUCTION_DOMAIN: nest.saydao.org
    TARGET_LAYER: production
    TEST_TARGET_LAYER: development
    DEV_STACK_NAME: say-dev-nest1
    STAGING_STACK_NAME: say-staging-nest1
    PRODUCTION_STACK_NAME: say-nest1
    DOCKER_STACK_PATH: docker-compose.production.yml
    IMAGE_ID: ghcr.io/say-dao/nest-server:70c411d6d934858eb1c4eb5ee796d346470ac895
    STACK_FILE: docker-stack-say-nest1-70c411d6d934858eb1c4eb5ee796d346470ac895.yml
/home/runner/work/_temp/62631f51-5684-43ac-bc1a-ebecc566386e.sh: line 1: docker-compose: command not found
Error: Process completed with exit code 127.

[rimelek (Ákos Takács)]

Your build environment probably stopped supporting the outdated docker compose v1 and you need to use docker compose instead of docker-compose.

[ehsang]

@rimelek Thanks for your reply. I did change the docker-compose to docker compose but then I receive this:

======CMD======
docker stack deploy --prune --resolve-image=changed --with-registry-auth -c /tmp/docker-stack-say-nest1-d397a000dd75bdda0716962f26e1315fb36f4aff.yml say-nest1
======END======
err: unsupported Compose file version: 1.0
2024/08/11 14:35:26 Process exited with status 1

[rimelek (Ákos Takács)]

It was docker compose before and now docker stack? But the message seems pretty clear. You are using an unsupported compose file version.

Did you specify a compose file version in the compose file? Compose v2 does not require a version but Docker Swarm uses the “legacy compose file version 3” as mentioned in the documentation

Deploy a stack to a swarm

[ehsang]

@rimelek I am super noob in CI/CD and working on other developers code, how do I change that version?
Are you talking about the version: '3.6'?
this is my compose file:

SAY-DAO/nest-server/blob/release/docker-compose.production.yml

[ehsang]

I removed the Version:3.6 and pushed just to see what happens and now eventhough I put that back but I am getting a different error… I am so lost

======CMD======
docker stack deploy --prune --resolve-image=changed --with-registry-auth -c /tmp/docker-stack-say-nest1-06ae5e47fdc64a2586838e9b4d39e***8d7c745c7.yml say-nest1
======END======
err: WARNING: Error loading config file: open /home/***/.docker/config.json: permission denied
err: (root) Additional property name is not allowed
2024/08/11 19:12:49 Process exited with status 1

[rimelek (Ákos Takács)]

It seems to be a GitHub Actions which I don’t use yet. If everything you do gives you a new error, you will have to learn how the build server supports pipelines. You should have access to the client config json unless you once broke it and your non-root user can’t open it anymore or there is something special on GitHub like intentionally making that file unavailable. You also have an “Additional property name is not allowed” error message which means you are using a key in the yaml that is not supported by docker stack. Or since the invalid key is not mentioned but “root” is, your indentation could be incorrect. I don’t see a problem in your yaml, but are you sure the compose file you shared was what you actually used? I would add some debug line to the pipeline. For example something that reads the content of the compose file.

[ehsang]

Yeah same compose file. I am sure about the indentation, I also have the root access.
But still get this:

======CMD======
docker stack deploy --prune --resolve-image=changed --with-registry-auth -c /tmp/docker-stack-say-nest1-06ae5e47fdc64a2586838e9b4d39e***8d7c745c7.yml say-nest1
======END======
err: (root) Additional property name is not allowed

[meyay (Metin Y.)]

Seems like rending the compose file with docker compose config adds the project name as top level name: element into the compose file, which makes it an invalid compose file for stack deployments.

Update: you can remove the first line name: {project name} by piping the output to sed:

docker compose {args] config  |  sed '/^name:*/d' > /target

[ehsang]

@meyay

remove the first line

I am trying to implement what you said in my pipeline. Is this the right way? I think they way I am doing it is removing a line from my config

    steps:
      ...

      - name: Prepare Stack File
      - run: ...   docker compose -f ${{ env.DOCKER_STACK_PATH }}  config  > ${{ env.STACK_FILE }}
      
      - name: Remove the first line
        run: docker compose config | sed '/^name:*/d' >  ${{ env.STACK_FILE }}
          

error:

======CMD======
docker stack deploy --prune --resolve-image=changed --with-registry-auth -c /tmp/docker-stack-sy-net1-f386324459b2cb64f1d45e4929df298eb4f8bd1a.yml sy-net1
======END======
err: services.sy_net_db.ports.0.published must be a integer

I also Did the following:

    steps:
      ...
      - name: Prepare Stack File
      - run: ...   docker compose -f ${{ env.DOCKER_STACK_PATH }}  config  > ${{ env.STACK_FILE }} | sed '/^name:*/d' >  ${{ env.STACK_FILE }}
          

error:

======CMD======
docker stack deploy --prune --resolve-image=changed --with-registry-auth -c /tmp/docker-stack-sy-net1-f386324459b2cb64f1d45e4929df298eb4f8bd1a.yml sy-net1
======END======
err: (root) Additional property name is not allowed

[meyay (Metin Y.)]

it should look like this:

    steps:
      ...
      - name: Prepare Stack File
      - run: ...   docker compose -f ${{ env.DOCKER_STACK_PATH }}  config  | sed '/^name:*/d' >  ${{ env.STACK_FILE }}

The trick is to pipe the output of docker compose config to sed, so it can delete the line and create an output without it, which is finally written into a file.

[ehsang]

@meyay

docker compose -f ${{ env.DOCKER_STACK_PATH }}  config  | sed '/^name:*/d' >  ${{ env.STACK_FILE }}

Now my app is down …:((
It fixed the deployment but I think it removed more lines in my config because my docker logs of the deployed container returns this:

error: password authentication failed for user "postgres"

By the way I would also appreciate if you tell me how you figured out there is an extra line, need that for future debugging.

[meyay (Metin Y.)]

Quite easy: if the whole construct is not working, check every piece individually.

Even though you failed to mention in your first post that you use docker compose config to render your compose file to use it with docker stack deploy, it was obvious after your second post.

I remembered seeing the project name in the output of docker compose config recently, and I then I verified it with one of my compose files, and come up with the sed argument that deletes every line that starts with name: followed by whatever value. Since there can only be a single line in the compose file where name: actually starts at the beginning of the line, it deletes only this single line.

[meyay (Metin Y.)]

But it seems there is still another problem, because docker compose config removes the version as weill, which is required by docker stack deploy

It might be a better idea to just replace name: {project name} with version: "3.9":

steps:
      ...
      - name: Prepare Stack File
      - run: ...   docker compose -f ${{ env.DOCKER_STACK_PATH }}  config  | sed 's/^name:.*/version: "3.9"/' >  ${{ env.STACK_FILE }}

[ehsang]

@meyay

| sed 's/^name:.*/version: "3.9"/' > ${{ env.STACK_FILE }}

Thank alot for explaining and your support. Both fixed the deployment issue but, my postgres authentication failing which never was the case, is this related to what we did? I also did the following to resolve it:

docker stack rm <name>
docker config rm  <name>.env
nano prod.env
docker config create <name>.env prod.env

# then run all github jobs

The deployed container returns:

error: password authentication failed for user "postgres"
    at Parser.parseErrorMessage (/usr/src/app/node_modules/pg-protocol/dist/parser.js:283:98)
    at Parser.handlePacket (/usr/src/app/node_modules/pg-protocol/dist/parser.js:122:29)
    at Parser.parse (/usr/src/app/node_modules/pg-protocol/dist/parser.js:35:38)
    at Socket.<anonymous> (/usr/src/app/node_modules/pg-protocol/dist/index.js:11:42)
    at Socket.emit (node:events:520:28)
    at addChunk (node:internal/streams/readable:559:12)
    at readableAddChunkPushByteMode (node:internal/streams/readable:510:3)
    at Readable.push (node:internal/streams/readable:390:5)
    at TCP.onStreamRead (node:internal/stream_base_commons:191:23)

[meyay (Metin Y.)]

The information I provided should give you the details to test it. I also explained why it couldn’t be caused by the sed command. There is nothing more I can do. You need to test yourself and find the missing piece.