Docker for Azure shows wrong disk space

General
1

When running df on Docker for Azure it shows 28.0G used :

swarm-manager000002:~$ df -h
Filesystem Size Used Available Use% Mounted on
overlay 29.4G 28.0G 0 100% /

but when doing du -sh / it shows just 2.5G

swarm-manager000002:~$ du -sh / 2>/dev/null
2.5G /

Any idea why?

P.S. Also master is kind of down :frowning:

swarm-manager000002:~$ docker node ls
Error response from daemon: rpc error: code = 14 desc = grpc: the connection is unavailable

Additional info:

swarm-manager000002:~$ docker info
Containers: 5
Running: 5
Paused: 0
Stopped: 0
Images: 6
Server Version: 17.03.1-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true

2

We’ve also encountered the same problem a few days ago, we’re still looking for a solution as the docker agent stopped working properly.

3

This is what i figured out - there is a docker container (from Docker for Azure) on every machine listening to syslog events from other containers (syslog logging is what is turned by default). This syslog container mounts Azure File System through SMB. At some moment the size of the whole Azure File System maps to the size of the local SMB folder. It leads to “out of disk space” situation. If you kill that container, you get the disk space back. If it is a problem of SMB, or Azure or Docker for Azure - i don’t know :frowning: