Docker Community Forums

Share and learn in the Docker community.

Docker 19.03 cant see changes in volume

Hi,

I have a container service (watchtower) that automatically updates my containers. To be able to check for updates, I must have a valid token. For that, I have a cronjob that automatically updates the auth token in a config.json file. This has been working fine for about 10 months, the updated token has been fetched by the container that uses it.

After upgrading to from 18.09 to 19.03, the container can no longer detect changes in the config.json file. It only successfully gets updated when starting the container, but when trying to manually invalidate the token with vim or nano, the changes aren’t reflected.

The command used for setting up the volumes in docker-compose is:

volumes:
  - /var/run/docker.sock:/var/run/docker.sock
  - /root/.docker/config.json:/config.json

Are there any changes in 19.03 that broke this behavior?

Probably a security fix to prevent the container user from accessing your root owned directories.
I believe un less you set some permissions flag your container is not allowed to do work with the host directories.
YOu should add the host directories to a GROUP that corresponds to a GID that the container user is apart of.
Iff container user is 1001 and group 1001 then the host directory needs a chown for a group with the same ID.
the command id should display something like this
uid=1000(ubuntu) gid=1000(ubuntu) groups=1000(ubuntu),4(adm),20(dialout),24(cdrom),25(floppy),27(sudo),29(audio),30(dip),44(video),46(plugdev),108(lxd),114(netdev)
On the host:
chown :<group with id 1000> /root/.docker/config.json
This is speculation.
let me know iff that is helpful.
Cheers,
Z

Good idea, though if it works after (re-)starting the container, it is highly unlikely that permissions are the problem.

The behavior would make sense for configs and secrets as a deployed key never get updated, unless the key is changes.

Though for bind-mounts it doesn’t make sense that changed content is not visible inside the vm. It would make sense that it doesn’t get updated when the file is replaced, as the file handle used for the mount changes for the new file on the host, but the container still uses the old handle.At least this is the behavior for directories… I am not 100% sure if the behavior is the same for single files.

I have read all the issues where this has been reported before. Nothing I have tried makes any difference. It doesn’t work. Even though volume is created in the container, it does not show any directories and files from the source path. Neither Issue #25 or Issue #50 solve this problem.

Have disabled the firewall.
Shared c in docker settings. have reset and supplied credentials.
Have tried all variations of command syntax for volume parameter
No help from freenode docker channel.
Java client api using mounts works fine if Linux -> Linux. Same code fails with windows.
When I inspect container it contains the proper source location.
Fails the same whether run under powershell or cmd.
Diagnostic ID from “Diagnose & Feedback” in the menu.
a reproducible case if this is a bug, Dockerfiles FTW
page URL if this is a docs issue or the name of a man page
host distribution and version (Windows version, build number, etc)
A86A3BDC-3635-4682-A3B1-467D8ABE2260/2016-09-13_09-56-40

Steps to reproduce the behavior
… docker run -it -v //c/Users:/users Ubuntu bash
…ls -al /users
Volume is mounted but nothing in the directory.

Thanks for your answers.
@zacharyhdb actually your solution might just be what was the reason.
At least I’m able to reproduce the issue in this way:

Give config.json the same GID as user that runs container.
-Can see that the auth token works-
Use nano to edit the token (like changing a 1 to 1 2)
-Can see that the logs gives error messages about invalid token-

Will try to let this run for a while, and see if it works even when the cronjob updates the token.

1 Like

As written in my previous reply, I can manage to recreate this manually by changing the GID of the file, and when “invalidating” the file with nano the change is reflected into the container using the file.

However, when running a docker command to refresh the token, the link breaks.

I read somewhere that this could be fixed by creating a symbolic link, and when starting the container mount the symbolic link instead of the file. In my case this breaks the link. Also when using vim to change the file, the link is broken…

I would suggest, creating a non root user… to run the container…
Then adjust your bind-mounts in a non root directory.
also make sure the crontab job is non-root user as well.
IFF that does not work… Please post some more details on the process and outputs.
GOod luck.

This has been solved. Thank you for your replies.

The solution;
Watchtower made is possible to read config.json file from a specified folder, instead of /.
This solution enables mounting the folder instead of file, which might have been broken in some way with the 19.03 update.

Watchtower issue:

Watchtower commit with solution: