Connection refused in request from container to container

I’m having quite a simple setup which works for the most part. In the console of my browser, I see the front-end app making a successful ajax request to the api app with the right response. However, if that request takes place from within the front-end container (see code), I get a connection refused. Any ideas?

Slimmed version of docker-compose.yml

version: '3.3'

services:
  proxy:
    networks:
      - backend
      - frontend
    ports:
      - 80:80

  api:
    networks:
      - backend

  frontend:
    networks:
      - backend
      - frontend

networks:
  backend:
    driver: overlay
  frontend:
    driver: overlay

Slimmed version of nginx.conf

http {
  upstream api {
    server api:3000;
  }

  upstream frontend {
    server frontend:4000;
  }

  server {
    listen 80;
    server_name api.project.dev;

    location / {
      proxy_pass  http://api;
    }
  }

  server {
    listen 80;
    server_name project.dev;

    location / {
      proxy_pass  http://frontend;
    }
  }
}

You can find more context, all code and steps to reproduce in my repo on github.