Unable to remove filesystem

General
1

I’m trying to remove a stopped docker when I’m getting

% docker rm plex
Error response from daemon: unable to remove filesystem for 1407e51fefad6dec47eeb93aedbd0dece7fe35c0a71cc459194ed916a7f19cf1: unlinkat /var/lib/docker/containers/1407e51fefad6dec47eeb93aedbd0dece7fe35c0a71cc459194ed916a7f19cf1: directory not empty

If I try to remove it manually:
rm: cannot remove ‘/var/lib/docker/containers/1407e51fefad6dec47eeb93aedbd0dece7fe35c0a71cc459194ed916a7f19cf1’: Directory not empty

% sudo ls -l /var/lib/docker/containers/1407e51fefad6dec47eeb93aedbd0dece7fe35c0a71cc459194ed916a7f19cf1
ls: cannot access /var/lib/docker/containers/1407e51fefad6dec47eeb93aedbd0dece7fe35c0a71cc459194ed916a7f19cf1/hostconfig.json: No such file or directory
ls: cannot access /var/lib/docker/containers/1407e51fefad6dec47eeb93aedbd0dece7fe35c0a71cc459194ed916a7f19cf1/hostconfig.json: No such file or directory
ls: cannot access /var/lib/docker/containers/1407e51fefad6dec47eeb93aedbd0dece7fe35c0a71cc459194ed916a7f19cf1/hostconfig.json: No such file or directory
ls: cannot access /var/lib/docker/containers/1407e51fefad6dec47eeb93aedbd0dece7fe35c0a71cc459194ed916a7f19cf1/hostconfig.json: No such file or directory
ls: cannot access /var/lib/docker/containers/1407e51fefad6dec47eeb93aedbd0dece7fe35c0a71cc459194ed916a7f19cf1/hostconfig.json: No such file or directory
ls: cannot access /var/lib/docker/containers/1407e51fefad6dec47eeb93aedbd0dece7fe35c0a71cc459194ed916a7f19cf1/hostconfig.json: No such file or directory
ls: cannot access /var/lib/docker/containers/1407e51fefad6dec47eeb93aedbd0dece7fe35c0a71cc459194ed916a7f19cf1/hostconfig.json: No such file or directory
total 0
-????????? ? ? ? ?            ? hostconfig.json
-????????? ? ? ? ?            ? hostconfig.json
-????????? ? ? ? ?            ? hostconfig.json
-????????? ? ? ? ?            ? hostconfig.json
-????????? ? ? ? ?            ? hostconfig.json
-????????? ? ? ? ?            ? hostconfig.json
-????????? ? ? ? ?            ? hostconfig.json

Any ideas?

2

I would put my money on a corrupt filesystem.

if findmnt --target /var/lib/docker returns that the target is /, a reboot should initiate a filesystem check, if it’s not, the source should be a different drive other than the one for the root filesystem.

In the second case:
– Stop the docker daemon: service docker stop
– unmount the drive: umount /dev/{drive}
– use the check/repair tool for the used filesystem: e.g. fsck.xfs -n /dev/{drive}, fsck.ext4 /dev/{drive} or whatever filesystem you use and however its fsck. needs to be called
– mount the device again: mount -a (assumed it’s registred in /etc/fstab)
– Start the docker deaemon: service docker start
– Delete the container: docker rm plex

3

Thank you, I ended up rebooting the server, that solved the issue