I have installed the Docker package on my Synology NAS.
With several containers I have installed, the moment I start them, Docker Daemon (process dockerd) starts writing to disk every 30 seconds, whether there is activity in the container or not. As a result, my Synology never goes to sleep.
Like clockwork, every 30 seconds I see about 30 lines like these in my “hibernation debug log”:
[1089851.478049] ppid:1(systemd), pid:10393(dockerd), WRITE block 2405518528 on dm-2 (64 sectors)
[1089851.478102] ppid:1(systemd), pid:10393(dockerd), WRITE block 2403421440 on dm-2 (128 sectors)
[1089851.478172] ppid:1(systemd), pid:10393(dockerd), WRITE block 2405518592 on dm-2 (128 sectors)
How can I find out what the process is writing (which files), and then - how to make it stop writing?
I did - but Synology support is absolutely useless. First, they said they did not write docker - and can’t support it - and then they suggested I study the documentation for their Linux kernel. Less than helpful.
I have three containers installed - and one of them works fine (“OwnTone” iTunes server - does not have any disk activity, is just listening for an iTunes remote to wake it up).
The other two both behave in the very same way: As soon as they are started, process ‘docker’ writes something to disk every 30 seconds.
Does anyone have an idea how to figure out what file Docker is writing to when it shows up with “WRITE block 2405518528 on dm-2” in my “synology hibernation log”?
I mean - there must be a way - but everything I tried (BTRFS file system commands…) did not yield any results…
But you could try to turn off container logs and see if it helps.
docker run -it --log-driver none <image>...
or in compose:
services:
myservice:
..
logging:
driver: none
...
Be sure to never edit the container in the Syno-UI again, as the ui removes settings it doesn’t support.
Aren’t they funny. They altered the sources to make it work with libraries (where they of course altered the sources as well) and added additional features to it… and broke the swarm-mode since it was available in docker-ce 17.05. I doubt that any of their modifications went upstream.
If what they claim is true, the binaries of docker’s official binary releases should be a drop-in replacment for the docker binaries from the Synology docker distribution … well, they aren’t
Thanks a lot for your ideas. I tried the docker run command (had to add “-- network host” to the command for it to work) - but it did not help. The log is still full of “WRITE block….” lines
If you have more than one volume, you can use sudo pvs /dev/md2 to find out which volume group id the device md2 has and mount | grep {whatever id your volume group md2 has} to find out where its mounted in the filesystem.
Maybee you can leverarage lsof | grep dockerd (if not available. use sudo synogear install to enable it) to see where it actualy has open file handles - I would expect it to write in the volume detected by the method from above. If it doesn’t then it’s a sign it is something where Syno’s dockerd sends something to another Syno service, like the logging service.
Though, since it appears that data access is done to a (Synology) volume, it could be indeed something written in docker’s root data folder (use sudo docker info --format '{{.DockerRootDir}}' to find out where it is). If you use a ssd drive in your NAS, you could move the docker package there.
You wrote that it happens ever 30 seconds, you might want to check if any of your containers commes with a healthcheck and if it does, try to start the container without the healthcheck (this is not going to work from the Syno docker UI) .
Thank you for all your help on this!
In the end, I managed to wrestle Synology into looking at the problem - and indeed, the health check was the problem, on both my trouble containers: * Our developers took another look, and determined that the container(s) both contain a setting called healthcheck, which requires it to report it’s status back to a config.v2.json file every 5 seconds.*
Ok, it was 30 seconds, not 5 - but after re-creating a container using the ‘docker run’ command with the parameter ‘–no-healthcheck’, all the ‘WRITE’ activity disappeared.
Interestingly, I got a gazillion of ‘READ’ entries in my debug log now - but after a day or so, the NAS properly went to sleep (I guess that might have to do with caching).
