Database Connection Error on Ist run, not on subsequent runs

Open Source Projects Compose
1

I am using get-started sample application for compose understanding so written following dockerfile and compose file. I have NOT created any volume, network and anything extra. When composer is run first time, I get DB connection error, but subsequent run does not show any error. So my question is why first time it throws error but not on subsequent runs

Thank you in advance.

==>dockerfile

syntax=docker/dockerfile:1

FROM node:18-alpine
WORKDIR /app
COPY package.json yarn.lock ./
RUN yarn install --production
COPY . .
CMD [“node”, “src/index.js”]
EXPOSE 3000

==>docker-compose-dev
services:
sgweb:
build:
context: .
ports:
- 3000:3000
depends_on:
- sgdb
environment:
- MYSQL_HOST=sgdb
- MYSQL_USER=root
- MYSQL_PASSWORD=password
- MYSQL_DB=get-started-build

sgdb:
image: mysql:8.0
volumes:
- db-get-started:/var/lib/mysql
environment:
- MYSQL_ROOT_PASSWORD=password
- MYSQL_DATABASE=get-started-build

volumes:
db-get-started:

==> ERROR, when ran docker compose --file docker-compose-dev.yaml up, first time
app-sgweb-1 | Waiting for sgdb:3306…
app-sgweb-1 | Timeout
app-sgweb-1 | Error: connect ECONNREFUSED 172.18.0.2:3306
app-sgweb-1 | at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1494:16) {
app-sgweb-1 | errno: -111,
app-sgweb-1 | code: ‘ECONNREFUSED’,
app-sgweb-1 | syscall: ‘connect’,
app-sgweb-1 | address: ‘172.18.0.2’,
app-sgweb-1 | port: 3306,
app-sgweb-1 | fatal: true
app-sgweb-1 | }

==> NO error, when reran the same command
app-sgweb-1 | Waiting for sgdb:3306…
app-sgweb-1 | Connected!

2

Please format your code according these instructions: How to format your forum posts

3

Thank you so much Metin for your message. Here is formated code.

==>Dockerfile

#syntax=docker/dockerfile:1
FROM node:18-alpine
WORKDIR /app
COPY package.json yarn.lock ./
RUN yarn install --production
COPY . .
CMD ["node", "src/index.js"]
EXPOSE 3000

==>Docker-compose-dev

services:
  sgweb:
    build:
      context: .
    ports: 
    - 3000:3000
    depends_on:
    - sgdb
    environment:
    - MYSQL_HOST=sgdb
    - MYSQL_USER=root
    - MYSQL_PASSWORD=password
    - MYSQL_DB=get-started-build

  sgdb:
    image: mysql:8.0
    volumes: 
    - db-get-started:/var/lib/mysql
    environment:
    - MYSQL_ROOT_PASSWORD=password
    - MYSQL_DATABASE=get-started-build

volumes:
  db-get-started:
4

Probably because the first time the databse is not ready yet. The fact that the container is running doesn’t mean the database server finished the initialization.

5

Thank you so much for your inputs and sparing time on it. I may agree with your view that container is running, doesn’t mean the database server finished the initialization. But it should be ready after some time. But my observation is, even after waiting for few mins, DB was not ready. It was ready only when I brought down the compose and bring it up again.

6

Then I don’t know :frowning: Can you shar some log messages to show what you see when the db is not ready and after the compose down and up? Please, use code blocks for log messages as well.

I can only guess that the database is in a wrong state although the initialization was finished and removing the container and recreating it to mount the existing files can help. If this is the case, stopping the containers instead of removing them should help too.

PS.: I moved the topic from the Community category. That is for events and for asking about the community.

7

Thank you so much Akos for doing the needful for this issue.

Here is complete analysis, it has been observed: Script creates 4 objects: image, container, network and volume.

First time when script is executed <docker compose -f docker-compose-dev.yaml up --build>, it shows DB connection error, as it creates volume along with other 3 objects. When execution is terminated (not using --detach) with <CTRL+c> and rerun the command, log shows DB connected. Also note that after first run, command <docker compose ls -all> in a separate window, shows nothing under port of app container. But after second run, it shows: 0.0.0.0:3000->3000/tcp, :::3000->3000/tcp

Now reproduce the error. Terminate the excution, delete containers and volume. Rerun the same command. Again, app container throws DB connection error. Pressing <CTRL+c> and without deleting anything, rerunning same command, log message shows DB connected.

Docker-DBError-Analysis
Docker-DBError-Analysis1581×532 232 KB

So it seems, issue is related to container and volume creation.

Sorry to say I have not got you on: use code blocks. What is code blocks.

Also (separate ask) when command <docker compose logs -f> is executed on separate window, it throws error: no configuration file provided: not found

8

Of course, if when you delete the volume the database has to be initialized again. If you just delete the container the database server just starts.

I don’t know about that, but docker compose ls wouldn’t show containers but compose projects.

Exactly what you did after @meyay asked for formatting your message.

9

Thank you for your inputs. Sorry for typo, it should be <docker container ls -all>.

Agree that DB initialization is not happening. Now que is how to proceed, ie, what is the solution to resolve the issue. As I tried multiple options, but issue remains same. Even when I run script as it is given in (Part 8: Use Docker Compose) under (Get Started), it shows same error.

10

Start only the database first and never delete the volume or implement a healthcheck and use depends_on with the long syntax which depends on actual health status

https://docs.docker.com/compose/compose-file/05-services/#depends_on

11

As informed :smiley:, as you suggested, I already tried like placing DB first, using depends_on, even dived into syslog, but got nothing. On healthcheck part, yes, on this I hv to read, undertand and explore.

Just a wild guess, seems a bug either in MySQL image creation or somewhere during the container initialization part. May be someting like, DB is in initialization phase, but app tries to connect DB and fails, hence throwing DB connection error

12

Either I still don’t understand you completely or I don’t see any bug here. If generating the files takes time, apps trying to connect to the database will fail. If the apps don’t wait for the connection in a loop, they will fail. If you saw the example in a tutorial, that should have mentioned it and make the app wait for the connection.

By the way if you set the restart policy to always, even if the app fails and the container stops, it will be restarted and connect to the database when it is already initialized.

sgweb:
  restart: always

This works even without the health check.

13

Thank you Akos for your inputs and helping me out from this problem. By adding restart, it worked. But my question remains same, :slight_smile: why on first time its always failing. For the time being, I will move forward.

Once again thank you so much.

14

Unless you have some more information to add, my answer remains the same as well. Slow initialization, failing client and your client did not start again. So it is completely normal and expected. It was also expected that the web container didn’t have the port information in the list when it failed, because it failed… so by that time there was no port forward to a stopped container.

In both of your tests you just pressed CTRL+C to stop the container not delete it. So it was already initilized.

You wrote this before:

Yet the initialization was done in both cases. It was just the client application which did not start. If it were not true, the restart policy of the client contaniner wouldn’t help at all.

Regarding why the tutorial didn’t mention waiting for the connection properly, I don’t know. Maybe because the speed of the initialization can depend on the speed of the disk (HDD/SSD) and in some environments it could finish in time, while in other environments it is too slow. Or it is not impossible that the writer just forgot it because the database was already initialized on their machine.

Maybe I miss something but since you couldn’t make me realize it, all I can do is wish you luck and find someone who can see more in this issue than I was able to, but I still think that this issue is just about a .tutorial not mentioning waiting for the connection.