Nginx exits with error "No such file or directory" in Ubuntu 20.04

Hello all. I am learning Docker and I fell in love with how it works. So right now I am trying to deploy a test app to my Ubuntu VPS. The app is using 4 images: Nodejs, Mongodb, Nginx and Certbot. I was following this tutorial to set up Nginx and SSL.

So after the build is done with Docker Compose, I see that Nginx and Certbox exited with errors. Nginx logs says:

/docker-entrypoint.sh: Configuration complete; ready for start up
2022/01/31 13:42:28 [emerg] 1#1: cannot load certificate "/etc/nginx/ssl/live/test.example.dev/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/nginx/ssl/live/test.example.dev/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)
nginx: [emerg] cannot load certificate "/etc/nginx/ssl/live/test.example.dev/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/nginx/ssl/live/test.example.dev/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)

And Certbot logs says:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Certbot doesn't know how to automatically configure the web server on this system. However, it can still get a certificate for you. Please run "certbot certonly" to do so. You'll need to manually configure your web server to use the resulting certificate.

Now I tried countless methods from the internet and here to fix this but I am unable to do so. What am I doing wrong here?

Please note that port 443 is being used by something else in my server. Something Nginx related(not a docker container). That’s why I couldn’t add it in my docker-compose file. It throws an error that 443 is already in use.

Here’s my nginx/default.conf file:

server {
    listen 80;
    listen [::]:80;

    server_name test.example.dev;
    server_tokens off;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

    location / {
        return 301 https://test.example.dev$request_uri;
    }
}

server {
    listen 443 default_server ssl http2;
    listen [::]:443 ssl http2;

    server_name test.example.dev;

    ssl_certificate /etc/nginx/ssl/live/test.example.dev/fullchain.pem;
    ssl_certificate_key /etc/nginx/ssl/live/test.example.dev/privkey.pem;
    
    location /api {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        proxy_set_header Host $http_host;
        proxy_set_header X-NginX-Proxy true;
        proxy_pass http://practice-app:3050;
        proxy_redirect off;

    }
}

Here’s my shared docker-compose file for both dev and prod environment:

version: '3'
services:

  practice-app:
    build: .
    depends_on:
      - mongo
      
  nginx: 
    image: nginx:stable-alpine
    ports: 
      - "4088:80"
    volumes: 
      - ./nginx/default.conf:/etc/nginx/conf.d/default.conf:ro
    depends_on:
      - practice-app

  mongo:
    image: mongo:4.4.6
    environment:
      - MONGO_INITDB_ROOT_USERNAME=test
      - MONGO_INITDB_ROOT_PASSWORD=test
    volumes:
      - mongo-db:/data/db
      
volumes:
  mongo-db:

And here’s my docker-compose.prod.yml file:

version: "3"
services: 

  practice-app:
    build: 
      context: .
      args: 
        NODE_ENV: production
    environment:
      - NODE_ENV=production
    command: node index.js

  nginx: 
    ports: 
      - "4088:80"
    volumes:
      - ./nginx/default.conf:/etc/nginx/conf.d/default.conf:ro
      - ./certbot/www:/var/www/certbot/:ro
      - ./certbot/conf/:/etc/nginx/ssl/:ro

  certbot:
    image: certbot/certbot:latest
    volumes:
      - ./certbot/www/:/var/www/certbot/:rw
      - ./certbot/conf/:/etc/letsencrypt/:rw
    depends_on:
      - nginx
      
  mongo: 
    environment:
      - MONGO_INITDB_ROOT_USERNAME=test
      - MONGO_INITDB_ROOT_PASSWORD=test

Please forgive me if this question was asked before. I tried the solutions but it didn’t work. And also I am new to DevOps and all so these things are new to me. Please give me a beginner friendly solution.

The answer or at least a part of the answer is in the error message:

Please, run “certbot certonly”

certbot without command tries to the webserver you are using but you are running it in a container. I also use certonly because I don’t want anything to change my server configuration. The entrpoint of the image is “certbot”. It means you can set the command for the container to “certonly” so when the container starts, it will run “certbot certonly”

 certbot:
    image: certbot/certbot:latest
    command: certonly

But this would not be enough. Certbot also needs some configuration. IThis is what I used:

    certbot certonly \
      --expand \
      --email '${LE_EMAIL}' \
      --non-interactive \
      --agree-tos \
      --standalone \
      --preferred-challenges http-01 \
      --http-01-port 8080 \
      --cert-name 'mycertname' \
      -d 'my.domain.tld,my2.domain.tld'

You can notice that I used --http-01-port and set to 8080. Note that certbot can listen on any port but it doesn’t change the fact (at least as I know it) the remote letsencrypt server will not send you the request on any other ports than 80 or maybe 443. This parameter is useful only if you want to put your certbot client behind a reverse proxy (it will start a server process for you in the container) and configure that reverse proxy to forward the requests sent to any domain on /.well-known/acme-challenge/ to localhost:8080.

localhost:8080 works only when your certbot container uses the network namespace of your reverse proxy container

 certbot:
    image: certbot/certbot:latest
    command:
      - certonly
      - --expand
      - --email '${LE_EMAIL}'
      - --non-interactive
      - --agree-tos
      - --standalone
      - --preferred-challenges http-01
      - --http-01-port 8080
      - --cert-name 'mycertname'
      - -d 'my.domain.tld,my2.domain.tld'
    network_mode: container:proxy_nginx_1

Currently I don’t have the latest certbot so the actual parameters could be different from you.

One more thing. I run this certonly command occasionally and not constantly so you could use that in a cron job.

Hey thanks for your reply. However, it isn’t working. It says ‘unrecognized arguments’. Here’s what I added:

certbot:
    image: certbot/certbot:latest
    command:
      - certonly
      - --expand
      - --email example@gmail.com
      - --non-interactive
      - --agree-tos
      - --standalone
      - --preferred-challenges http-01
      - --http-01-port 8080
      - --cert-name 'test'
      - -d 'test.example.dev'

And this is the full error:

Certbot can obtain and install HTTPS/TLS/SSL certificates.  By default,
it will attempt to use a webserver both for obtaining and installing the
certificate.
certbot: error: unrecognized arguments: --email example@gmail.com --preferred-challenges http-01 --http-01-port 8080 --cert-name 'test'

Sorry, this is the right syntax

certbot:
    image: certbot/certbot:latest
    command:
      - certonly
      - --expand
      - --email
      - example@gmail.com
      - --non-interactive
      - --agree-tos
      - --standalone
      - --preferred-challenges
      - http-01
      - --http-01-port
      - 8080
      - --cert-name
      - 'test'
      - -d
      - 'test.example.dev'

but

  1. You won’t be able to generate certificates for test.example.dev
  2. --http-01-port with 8080 will probably not work for you.
  3. MAke sure certbot is available on port 80 from a remote server