Docker container can't startup with a 166G(nfs/efs) volume

I have a folder with 166G files (470k+ files nfs/efs) in the host machine (AWS ec2 ubuntu 18), when I volume to the docker, the docker container is stuck and can’t startup, if we try another 2G folder, it can startup.
Does someone know why? Is there a limitation for the volume?
my docker version is 20.10.7.

I do have bind-mounts on block devices in the TB range with way more files + I have remote machines that use nfs exports from the machine that expose these block devices as named volume backed as nfs. Works without issues.

By volume you mean:

  • bind-mount?
  • named volume backed by bind-mount?
  • named volume backed by nfs/efs?
  • named volume backed by ebs (using a plugin)?

Are you sure it’s not a permission problem? What is the exact error message?

I formatted the compose file content in your lat post by wrapping it in a </> block.
After I saved the change, the post disappeared… at least I can’t see it anymore.
Sorry if it realy become deleted.

Yes, that’s weird, my reply disappeared, so let me post it again, thank you @meyay . I use the EFS mounted to the EC2 with amazon-efs-utils tool, and then volume to the docker, below is my docker-compose file

        image: xxxx
        restart: always
            - ASPNETCORE_ENVIRONMENT=Production
            - "8002:8002"
            - "./BinaryObjects/Static:/app/App_Data/BinaryObjects/Static"
            - "./BinaryObjects/Temp:/app/App_Data/BinaryObjects/Temp"

the /BinaryObjects is the root which EFS mounted to, and /Static folder has the 166G files, and /Temp folder has 10M files, if I remove the /Static in volumes, and the docker container can startup. so I think it is not the permission issue. I did the same thing in our staging env, and it works well. I have tried that created the nfs volumes first,

docker volume create \
    --driver local \
    --opt type=nfs \
    --opt,nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport \
    --opt device=:/Static 

docker volume create \
    --driver local \
    --opt type=nfs \
    --opt,nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport \
    --opt device=:/Temp 

and no luck, the docker container can’t startup neither, and works if I remove the efs-static volume, it is just stuck, and no error message, please refer to below screenshots.

Can you please share how you mount your TB files to docker? Maybe I can take a try. thank you so much.

Update: after I upgraded the OS from ubuntu 18.04 to ubuntu 20.04, this issue has gone. the docker can startup in 6 minutes. but it is still longer than before. in the past, the docker can startup in 1 minute.