Docker compose up stuck on "starting" when using NFS volume

Hi,

I have created a NFS server and I plan to use it to store docker container data.

If I mount the NFS share in ubuntu via mount command everything works.

Now, I have created the docker volume successfully and I can see it with inspect:
docker volume inspect immich_immich_data

[
    {
        "CreatedAt": "2023-08-22T21:17:18Z",
        "Driver": "local",
        "Labels": {
            "com.docker.compose.project": "immich",
            "com.docker.compose.version": "2.18.1",
            "com.docker.compose.volume": "immich_data"
        },
        "Mountpoint": "/var/lib/docker/volumes/immich_immich_data/_data",
        "Name": "immich_immich_data",
        "Options": {
            "device": ":/export/immich",
            "o": "nfsvers=4,addr=[192.168.1.235],nolock,soft,rw",
            "type": "nfs"
        },
        "Scope": "local"
    }
]

I have the following docker compose:

version: "3.8"

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: [ "start.sh", "immich" ]
    volumes:
      - type: volume
        source: immich_data
        target: /usr/src/app/upload
        volume:
          nocopy: true
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.yml
    #   service: hwaccel
    command: [ "start.sh", "microservices" ]
    volumes:
      - type: volume
        source: immich_data
        target: /usr/src/app/upload
        volume:
          nocopy: true
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  immich-web:
    container_name: immich_web
    image: ghcr.io/immich-app/immich-web:${IMMICH_VERSION:-release}
    env_file:
      - .env
    restart: always

  typesense:
    container_name: immich_typesense
    image: typesense/typesense:0.24.1@sha256:9bcff2b829f12074426ca044b56160ca9d777a0c488303469143dd9f8259d4dd
    environment:
      - TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
      - TYPESENSE_DATA_DIR=/data
      # remove this to get debug messages
      - GLOG_minloglevel=1
    volumes:
      - tsdata:/data
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:70a7a5b641117670beae0d80658430853896b5ef269ccf00d1827427e3263fa3
    restart: always

  database:
    container_name: immich_postgres
    image: postgres:14-alpine@sha256:28407a9961e76f2d285dc6991e8e48893503cc3836a4755bbc2d40bcc272a441
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

  immich-proxy:
    container_name: immich_proxy
    image: ghcr.io/immich-app/immich-proxy:${IMMICH_VERSION:-release}
    environment:
      # Make sure these values get passed through from the env file
      - IMMICH_SERVER_URL
      - IMMICH_WEB_URL
    ports:
      - 2283:8080
    depends_on:
      - immich-server
      - immich-web
    restart: always

volumes:
  pgdata:
  model-cache:
  tsdata:
  immich_data:
    driver_opts:
      type: "nfs"
      o: nfsvers=4,addr=[192.168.1.235],nolock,soft,rw
      device: ":/immich"

You can see how I create the nfs volume and attach it to to containers.

Now If I compose up -d this file I get the following:

  • Executing task: docker compose -f “immich/docker-compose.yml” up -d --build
[+] Building 0.0s (0/0)                                                                                                                              
[+] Running 5/7
 ✔ Container immich_machine_learning  Started                                                                                                   0.5s 
 ✔ Container immich_typesense         Started                                                                                                   0.5s 
 ✔ Container immich_redis             Started                                                                                                   0.3s 
 ✔ Container immich_web               Started                                                                                                   0.5s 
 ✔ Container immich_postgres          Started                                                                                                   0.3s 
 ⠋ Container immich_microservices     Starting                                                                                                  250.0s 
 ⠋ Container immich_server            Starting                                                                                                  250.0s

And it becomes stuck at this point forever.

I’m pretty sure it’s not a problem with the compose.yml file because if I remove the:

    driver_opts:
      type: "nfs"
      o: nfsvers=4,addr=[192.168.1.235],nolock,soft,rw
      device: ":/immich"

part from docker-compose.yml efectively converting the volume into a simple local volume everything works and spins up.
So that leads me to believe that there’s some issue with docker NFS.

Stop your compose project with docker compose -f immich/docker-compose.yml down, remove the volume with docker volume rm immich_immich_data, then start your compose project again.

If this is working, add your other option arguments one by one and repeat the steps from above to figure out which argument breaks the volume.

The manual docker volume rm step is required, as volumes are immutable: no configuration changes in the compose file will be reflected back to the volume declaration, unless it’s deleted and re-created.

1 Like

This is SOLVED, or at least the container starts now.

This doesn’t work:

volumes:
  pgdata:
  model-cache:
  tsdata:
  immich_data:
    driver_opts:
      type: "nfs"
      o: nfsvers=4,addr=[192.168.1.235],nolock,soft,rw
      device: ":/immich"

This does work:

volumes:
  pgdata:
  model-cache:
  tsdata:
  immich_data:
    driver_opts:
      type: "nfs"
      o: nfsvers=4,addr=192.168.1.235,nolock,soft,rw
      device: ":/immich"

Remember kids, if somewhere you see [ip-address-here] you have to substitue the [ ] too :crazy_face: