Macbook Pro intel errors "too many open files" prevents app from loading website

[nameoflotsnames2]

Issue type: logs spam “too many open files” does not allow app to work via localhost:8000
Macos: 14.3.1
MacBook Pro 1.4 GHz Quad-Core Intel Core i5, 8 GB memory
Docker Desktop App Version: 4.30.0 (149282) Engine: 26.1.1
Settings:
CPU Limit: 8
Memory Limit: 6gb
Swap: 2 gb
Virtual Disk: 64gb
Docker Disk Available: 46gb of 62gb
Ram 5.12gb used
CPU used: 5.39%
Signed in

Client has many Containers networked together.
When building and sending to Docker via

docker compose up -d

The apps launch and are accessible all but 1! And of course its the one I need to work with.
The app I need is a frontend react app that builds and loads node_modules files.

checking the logs I get many lines like this about 16 at count

[ERROR] Cannot read file "node_modules/@mui/icons-material/esm/CameraEnhanceOutlined.js": too many open files

✘ [ERROR] Cannot read file "node_modules/@mui/icons-material/esm/LoginOutlined.js": too many open files

I have attempted various forums of this to increase the size

ulimit -n 26040
sysctl -a | grep maxfiles
kern.maxfiles: 30720
kern.maxfilesperproc: 10240

sudo sysctl -w kern.maxfilesperproc=20240

docker run --privileged --ulimit nofile=15000:15000 appname:latest

The funny thing is if I run the necessary image with

docker run -p 8000:8000 appname:latest

I can access the app at localhost:8000 but I cant see the logs to see if they are spamming. However this is not a solution as I need the other backend apis communicating to each other.

My question is, for macos can and should I somehow raise the files limit?
My attempts did nothing I can tell, but I know not what I am doing really.
Also any commands I can try running to give some insight?

Thank you

More details from reading the other messages

docker version; docker info; uname -a

Client:
 Cloud integration: v1.0.35+desktop.13
 Version:           26.1.1
 API version:       1.45
 Go version:        go1.21.9
 Git commit:        4cf5afa
 Built:             Tue Apr 30 11:44:56 2024
 OS/Arch:           darwin/amd64
 Context:           desktop-linux

Server: Docker Desktop 4.30.0 (149282)
 Engine:
  Version:          26.1.1
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.9
  Git commit:       ac2de55
  Built:            Tue Apr 30 11:48:28 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.31
  GitCommit:        e377cd56a71523140ca6ae87e30244719194a521
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
Client:
 Version:    26.1.1
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.14.0-desktop.1
    Path:     /Users/neat/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.27.0-desktop.2
    Path:     /Users/neat/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.29
    Path:     /Users/neat/.docker/cli-plugins/docker-debug
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/neat/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.23
    Path:     /Users/neat/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.4
    Path:     /Users/neat/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.1.0
    Path:     /Users/neat/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/neat/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.8.0
    Path:     /Users/neat/.docker/cli-plugins/docker-scout

Server:
 Containers: 7
  Running: 7
  Paused: 0
  Stopped: 0
 Images: 6
 Server Version: 26.1.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e377cd56a71523140ca6ae87e30244719194a521
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.6.26-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 5.785GiB
 Name: docker-desktop
 ID: b79fb2c5-5c08-4324-9d25-5cd0a1e38962
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/neat/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile
Darwin cool-MacBook-Pro.local 23.3.0 Darwin Kernel Version 23.3.0: Wed Dec 20 21:28:58 PST 2023; root:xnu-10002.81.5~7/RELEASE_X86_64 x86_64
neat@cool-MacBook-Pro appbox % 

[rimelek (Ákos Takács)]

@nameoflotsnames2

My question is, for macos can and should I somehow raise the files limit?

@nameoflotsnames2

checking the logs I get many lines like this about 16 at count

Where did you check the logs? You mean the container logs? In that case it is not a macos issue. Containers are running in the virtual machine of Docker desktop so the logs have nothing to do with the macOS kernel. There is no such thing as macOS container yet.

@nameoflotsnames2

I can access the app at localhost:8000 but I cant see the logs to see if they are spamming.

Why not? Since you ran the container in the foreground (attached to the standard output and error streams), you should see the logs.

There was a similar topic here too:

What are baseline expectations for the number of open file handles with Docker in WSL2

Where do you get that message exactly? I think you can get the limit by running the following in the WSL distribution: sysctl fs.file-max or cat /proc/sys/fs/file-max but that should be way bigger than 436 Those are related to the WSL integration. If you disable WSL integration, you will probably not see any Docker related process since Docker will run in its own WSL distribution even if you have integration.

This was on Windows, but it happened on macOS as well

too many open files

What happened when you tried this command?

@nameoflotsnames2

docker run --privileged --ulimit nofile=15000:15000 appname:latest

I think that ulimit should have helped, although I never had to use it, but it will not persist. You will need to add that to the compose service. If the solution requires changing kernel parameters on the host, that is not impossible (or at least it wasn’t), but it will not persist either. If you want a full control over Docker and its Linux host, you can run a virtual machine using multipass. I use it like this:

multipass launch --name docker-vm docker
multipass docker info

That creates a VM called “docker-vm” from the VM image called “docker”. Optionally you can change CPU and memory limits. Then the second command is how you use the docker commands to communicat with the docker daemon inside the VM thanks to the multipass aliases. Or you just run a shell in the VM:

multupass shell docker-vm

Here is a guide from the multipass documentation

Running a container with the Docker blueprint in Multipass | Multipass...

You could also mount folders from your host to the Docker VM, but I think when I did it, that made my multipass freeze when my Mac woke up and many times I had to destroy the VM and reinstall it. So if you can make it work with Docker Desktop, you can more easily mount files into containers from the Mac, but if you need full control over the VM, and you can copy everything inside it, you can use Multipass.

[nameoflotsnames2]

@rimelek

Where did you check the logs? You mean the container logs? In that case it is not a macos issue. Containers are running in the virtual machine of Docker desktop so the logs have nothing to do with the macOS kernel. There is no such thing as macOS container yet.

Inside Docker Desktop clicking on the running container and reaching the tab for logs

@rimelek

Why not? Since you ran the container in the foreground (attached to the standard output and error streams), you should see the logs.

I was wrong, I did not know it makes a new container for each time I send that command. I have checked the new builds and the logs stop at launching Vite.

On that note about Vite, when I try to launch a run with -it or interact mode, Vite has its own key input system it runs in the computers console app which seems to block any expected repl input I was expecting to have. I futzed with configs and only find dont clear previous logs option.

@What are baseline expectations for the number of open file handles with Docker in WSL2

Where do you get that message exactly? I think you can get the limit by running the following in the WSL distribution: sysctl fs.file-max or cat /proc/sys/fs/file-max but that should be way bigger than 436 Those are related to the WSL integration.

I found the console input option in Docker Desktop app for each container. So when I run those commands linked in your replies and add some more from chatgpt I get…
In Docker Console

lsof | wc -l
lsof 2>&1 |wc -l
around 70

in Docker console

sysctl fs.file-max
> fs.file-max = 605760

cat /proc/sys/fs/file-max
> 605760

And in MacOS Terminal app

docker run --rm -it --pid host ubuntu bash -c "apt-get update && apt-get install -y lsof && lsof | wc -l"
> 17144
docker run --rm -it --pid host alpine sh -c "apk add --no-cache lsof && lsof | wc -l"
> 17117
docker run --rm -it --pid host bash -c "lsof | wc -l"
> 150

For lsof -Pn

lsof -Pn

1       /usr/local/bin/node     0       /dev/null
1       /usr/local/bin/node     1       pipe:[125991]
1       /usr/local/bin/node     2       pipe:[123975]
1       /usr/local/bin/node     3       anon_inode:[eventpoll]
1       /usr/local/bin/node     4       anon_inode:[io_uring]
1       /usr/local/bin/node     5       anon_inode:[io_uring]
1       /usr/local/bin/node     6       pipe:[119683]
1       /usr/local/bin/node     7       pipe:[119683]
1       /usr/local/bin/node     8       pipe:[119684]
1       /usr/local/bin/node     9       pipe:[119684]
1       /usr/local/bin/node     10      anon_inode:[eventfd]
1       /usr/local/bin/node     11      anon_inode:[eventpoll]
1       /usr/local/bin/node     12      anon_inode:[io_uring]
1       /usr/local/bin/node     13      anon_inode:[io_uring]
1       /usr/local/bin/node     14      pipe:[123451]
1       /usr/local/bin/node     15      pipe:[123451]
1       /usr/local/bin/node     16      anon_inode:[eventfd]
1       /usr/local/bin/node     17      anon_inode:[eventpoll]
1       /usr/local/bin/node     18      anon_inode:[io_uring]
1       /usr/local/bin/node     19      anon_inode:[io_uring]
1       /usr/local/bin/node     20      pipe:[119687]
1       /usr/local/bin/node     21      pipe:[119687]
1       /usr/local/bin/node     22      anon_inode:[eventfd]
1       /usr/local/bin/node     23      /dev/null
1       /usr/local/bin/node     24      /dev/urandom
1       /usr/local/bin/node     25      /root/.npm/_logs/2024-06-09T04_07_05_193Z-debug-0.log
53      /bin/busybox    0       /dev/pts/0
53      /bin/busybox    1       /dev/pts/0
53      /bin/busybox    2       /dev/pts/0
53      /bin/busybox    10      /dev/tty
73      /usr/local/bin/node     0       /dev/null
73      /usr/local/bin/node     1       pipe:[125991]
73      /usr/local/bin/node     2       pipe:[123975]
73      /usr/local/bin/node     3       anon_inode:[eventpoll]
73      /usr/local/bin/node     4       anon_inode:[io_uring]
73      /usr/local/bin/node     5       anon_inode:[io_uring]
73      /usr/local/bin/node     6       pipe:[121499]
73      /usr/local/bin/node     7       pipe:[121499]
73      /usr/local/bin/node     8       pipe:[121500]
73      /usr/local/bin/node     9       pipe:[121500]
73      /usr/local/bin/node     10      anon_inode:[eventfd]
73      /usr/local/bin/node     11      anon_inode:[eventpoll]
73      /usr/local/bin/node     12      anon_inode:[io_uring]
73      /usr/local/bin/node     13      anon_inode:[io_uring]
73      /usr/local/bin/node     14      pipe:[126056]
73      /usr/local/bin/node     15      pipe:[126056]
73      /usr/local/bin/node     16      anon_inode:[eventfd]
73      /usr/local/bin/node     17      anon_inode:[eventpoll]
73      /usr/local/bin/node     18      anon_inode:[io_uring]
73      /usr/local/bin/node     19      anon_inode:[io_uring]
73      /usr/local/bin/node     20      pipe:[121503]
73      /usr/local/bin/node     21      pipe:[121503]
73      /usr/local/bin/node     22      anon_inode:[eventfd]
73      /usr/local/bin/node     23      socket:[124032]
73      /usr/local/bin/node     24      /dev/urandom
73      /usr/local/bin/node     26      socket:[124034]
73      /usr/local/bin/node     27      socket:[124037]
73      /usr/local/bin/node     28      /dev/null
73      /usr/local/bin/node     29      /dev/urandom
93      /app/node_modules/@esbuild/linux-x64/bin/esbuild        0       socket:[124033]
93      /app/node_modules/@esbuild/linux-x64/bin/esbuild        1       socket:[124035]
93      /app/node_modules/@esbuild/linux-x64/bin/esbuild        2       pipe:[123975]
93      /app/node_modules/@esbuild/linux-x64/bin/esbuild        3       anon_inode:[eventpoll]
93      /app/node_modules/@esbuild/linux-x64/bin/esbuild        4       pipe:[125049]
93      /app/node_modules/@esbuild/linux-x64/bin/esbuild        5       pipe:[125049]
93      /app/node_modules/@esbuild/linux-x64/bin/esbuild        24      /dev/urandom
93      /app/node_modules/@esbuild/linux-x64/bin/esbuild        29      /dev/urandom

I see that the above stops at node_modules, which is the same dir root name as the original errors

 ✘ [ERROR] Cannot read file "node_modules/@mui/icons-material/esm/VideogameAssetOffOutlined.js": too many open files

@rimelek

What happened when you tried this command?

@nameoflotsnames2

docker run --privileged --ulimit nofile=15000:15000 appname:latest

No change in the issues behavior.

I will look into the multipass. For now I am pasting a slimed down docker-compose file with clients info swapped out if it helps anything. Im told all things go though traefik which routes the ports so I should be able to reach localhost:8000 with no problem normally without running the direct port line I noted above and here
docker run -p 8000:8000 appname

docker-compose.yml file shortened

services:
  db:
    ports:
      - "5432:5432"
    healthcheck:
      test:
        ["CMD", "pg_isready", "-d", "${POSTGRES_DB}", "-U", "${POSTGRES_USER}"]


  model:
    image: model
    networks:
      - appname-network
    # ports:
    # to open up streamlit
    # - "8501:8501"
    command:
      [
        "/usr/local/bin/python",
        "-m",
        "uvicorn",
        "main:app",
        "--reload",
        "--host",
        "0.0.0.0",
        "--port",
        "8000",
        "--log-level",
        "debug",
      ]

    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.boxes.rule=Host(`boxes.brand.localhost`)"
      - "traefik.http.routers.boxes.entrypoints=web"
      - "traefik.http.services.boxes.loadbalancer.server.port=8000"


  backend:
    image: appname-backend
    container_name: backend
    build:
      context: ./backend
      dockerfile: Dockerfile
      target: backend_local
    networks:
      - appname-network
    ports:
      - "9000:80"
    volumes:
      - type: bind
        source: ./backend/app
        target: /app
    command: [
        "/usr/local/bin/python",
        "-m",
        "uvicorn",
        "app.main:app",
        "--reload",
        "--host",
        "0.0.0.0",
        "--port",
        "80",
        # "--log-level",
        # "debug",
      ]
    labels:
      # Explicitly tell Traefik to expose this container
      - "traefik.enable=true"
      # The domain the service will respond to
      - "traefik.http.routers.backend.rule=Host(`app.brand.localhost`) && (PathPrefix(`/docs`) || PathPrefix(`/api`))"
      # Allow request only from the predefined entry point named "web"
      - "traefik.http.routers.backend.entrypoints=web"
      - "traefik.http.services.backend.loadbalancer.server.port=80"

  frontend:
    build:
      context: ../appname-front
      dockerfile: ./Dockerfile
      target: dev
    env_file:
      - .env.local
    image: "appname-front:latest"
    working_dir: /app
    networks:
      - appname-network
    volumes:
      - type: bind
        source: ../appname-front
        target: /app
    labels:
      # Explicitly tell Traefik to expose this container
      - "traefik.enable=true"
      # The domain the service will respond to
      - "traefik.http.routers.frontend.rule=Host(`app.brand.localhost`)"
      # Allow request only from the predefined entry point named "web"
      - "traefik.http.routers.frontend.entrypoints=web"
      - "traefik.http.services.frontend.loadbalancer.server.port=8000"

  traefik:
    # https://doc.traefik.io/traefik/user-guides/docker-compose/basic-example/
    image: "traefik:v2.10"
    container_name: "traefik"
    command:
      - "--api=true"
      - "--log.level=DEBUG"
      - "--api.insecure=true"
      - "--api.dashboard=true"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.web.address=:80"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.traefik.service=api@internal"
      - "traefik.http.routers.traefik.rule=Host(`traefik.brand.localhost`)"
      - "traefik.http.routers.traefik.entrypoints=web"
      - "traefik.docker.network=appname-network"
    ports:
      - "8079:443"
      - "80:80"
      - "8080:8080"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"

    networks:
      - appname-network
volumes:
  backend-db-data:
  test-backend-db-data:

networks:
  appname-network:
    driver: bridge

Is there a command to walk though the run routine so I can stop it and inspect right when it gives and error?

Thank you for your help!

[bluepuma77]

Potential workaround: You should be able to run the container with docker run and still use the same network as the compose containers. Either you create the Docker network manually on CLI or you create the network in compose, set name: and attachable: true.

[nameoflotsnames2]

I have an answer. In my case it was two fold.
First, its due to the config

    volumes:
      - type: bind
        source: ./backend
        target: /app

Namely the -type: bind, its keeping the entire project folder open, and node apps are SUPER files heavy.
The next is Docker changed the internal max files limit to respect the systems, not an arbitrary number. I cant find the change log about it, its nary a footnote item.

So that means for the mac, you need to change the systems values.

check
ulimit -n
sysctl kern.maxfiles
sysctl kern.maxfilesperproc

These will give you numbers to work with.
Then

sudo nano /etc/sysctl.conf
kern.maxfiles=20480
kern.maxfilesperproc=20480

give permissions root wheel

then

sudo nano /Library/LaunchDaemons/limit.maxfiles.plist

give permissions root wheel I guess

file limit.maxfiles.plist ^

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>limit.maxfiles</string>
    <key>ProgramArguments</key>
    <array>
        <string>sysctl</string>
        <string>-w</string>
        <string>kern.maxfiles=20480</string>
        <string>kern.maxfilesperproc=20480</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
</dict>
</plist>

sudo launchctl load -w /Library/LaunchDaemons/limit.maxfiles.plist

Ask chatgpt and google the key works in here about it for your systems version of steps.
Main thing is both values maxfilesperproc need to match the value in the plist file, cause you then have to restart to run that plist file to make the value change stick

Once you restart computer and before you launch docker, check with sysctl kern.maxfilesperproc
and then launch an app with your heavy files and check again, youll see the new amount. Youll also see that you can STILL go over, which then means move the number up. BUT eventually the core system wont allow a too large of a number.

OR, buy a new $$$ computer…