Docker Community Forums

Share and learn in the Docker community.

Docker for Windows should resync VM time when computer resumes from sleep


(June Rhodes) #1

Expected behavior

The clock in the Hyper-V machine is in sync with the host at all times.

Actual behavior

When the computer goes to sleep and is resumed, the Hyper-V virtual machine clock is delayed by the amount of time the computer was asleep. This causes all sort of subtle issues, e.g. I just spent several hours wondering why my OAuth credentials were working on my host machine, but failing with invalid_grant when running from the container, and it turned out that the clock in the VM was out of sync.

Steps to reproduce the behavior

You can reproduce this bug by doing the following:

  • Create a Dockerfile that just has like:
FROM node:latest
CMD [ "date" ]
  • Start Docker
  • Run the container to verify the clock is in sync
  • Send the computer to sleep
  • Resume the computer a while later
  • Run the container again and you’ll see less time has passed

For me, when I ran this I got output like the following (this is a Git Bash shell on Windows):

June@DESKTOP-CVHO135 ~/Documents/Projects/test
$ docker build -t date:latest .
Sending build context to Docker daemon 2.048 kB
Step 1 : FROM node:latest
 ---> e79fe5711c94
Step 2 : CMD date
 ---> Running in fb8525d9999b
 ---> 34da986209f0
Removing intermediate container fb8525d9999b
Successfully built 34da986209f0
SECURITY WARNING: You are building a Docker image from Windows against a non-Windows Docker host. All files and directories added to build context will have '-rwxr-xr-x' permissions. It is recommended to double check and reset permissions for sensitive files and directories.

June@DESKTOP-CVHO135 ~/Documents/Projects/test
$ docker run date
Sun Jul 10 13:01:40 UTC 2016

June@DESKTOP-CVHO135 ~/Documents/Projects/test
$ docker run date
Sun Jul 10 13:02:02 UTC 2016

Between running these commands, I had my computer asleep for 1 minute and 15 seconds (I timed it on my phone). Indeed you can see it in the screenshot here that the minutes no longer line up:


Syncing clock with host
(Michael Friis) #2

Thanks for reporting - we’re actively working on time-sync improvements!


(Michael Friis) #3

Also, does the VM clock catch up with the Windows one over time, or is it permanently stuck in the past?


(Andy Neff) #4

When I have this problem, it does not appear to catch up, but I only typically give it about 20 minutes which it the amount of time it takes me to figure out what is going on and restart dockers :wink: My scenario is cmake no longer will allow me to compile because it thinks my mounted directory is always modified in the future, and it never compiles :frowning:

Thanks for looking into this!

Client:
 Version:      1.12.0-rc3
 API version:  1.24
 Go version:   go1.6.2
 Git commit:   91e29e8
 Built:        Sat Jul  2 00:09:24 2016
 OS/Arch:      windows/amd64
 Experimental: true

Server:
 Version:      1.12.0-rc3
 API version:  1.24
 Go version:   go1.6.2
 Git commit:   876f3a7
 Built:        Tue Jul  5 02:20:13 2016
 OS/Arch:      linux/amd64
 Experimental: true

Update: It still happens on RC3. I slept for 2 minutes and 7 seconds, and let it run over night, and the time difference did not changes


(Mountainpenguin) #5

also a mac problem too


(Offix) #6

I also observe the problem for Windows and various different containers. In my cases I do not observe that the time catches uo with the system time. Only a restart will sync the time again. I am using the Windows version with Hyper-V.


(Tarand) #7

this is an annoying problem! I also heavily see it on my Windows (10) host.
Any forecast when it will be solved?
Is there a convenient workaround except rebooting the PC?


(Michael Friis) #8

Is your clock permanently stuck or does it catch up?


(Tarand) #9

Permanently! When I wake the PC up from hibernate (or standby), all running containers seem to have forgotten about the time in that mode. That means for example, after a weekend of inactivity, the clock will be running late by two days. And this situation will persist until you restart docker.

Thanks,
Thomas


(Assortify) #10

Confirmed I have experienced the same behavior running Hyper-V based Docker VM. I can see this affecting a lot of people who run on laptops and sleep the computer. I have occasionally been able to restart the container to fix.


(Assortify) #11

In my mind this is a critical bug, I am forced to restart docker several times during the day.


(Claus F. Strasburger) #12

Is there a bug open for this? Still happening on .1.12.3 (build 7135). Makes docker really annoying to use on laptops (every coffee break means restarting docker?)


(Cristualexandru) #13

In my case it’s off by a few days…

$ docker run -t node date
Fri Oct 21 09:51:09 UTC 2016

$ docker run -t centos date
Unable to find image 'centos:latest' locally
latest: Pulling from library/centos
8d30e94188e7: Pulling fs layer
8d30e94188e7: Verifying Checksum
8d30e94188e7: Download complete
8d30e94188e7: Pull complete
Digest: sha256:2ae0d2c881c7123870114fb9cc7afabd1e31f9888dac8286884f6cf59373ed9b
Status: Downloaded newer image for centos:latest
Fri Oct 21 09:52:56 UTC 2016

$ date
Tue, Oct 25, 2016  6:02:56 PM

$ docker run -t centos date
Fri Oct 21 09:53:07 UTC 2016

$ date
Tue, Oct 25, 2016  6:03:04 PM

$ docker run -t node date
Fri Oct 21 09:53:17 UTC 2016

$ date
Tue, Oct 25, 2016  6:03:11 PM

we can set the date of MobyLinuxVM manually using How can I SSH into the Beta's MobyLinuxVM

docker run --net=host --ipc=host --uts=host --pid=host -it --security-opt=seccomp=unconfined --privileged --rm alpine /bin/sh

/ # hostname
moby
/ # id
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),11(floppy),20(dialout),26(tape),27(video)
/ # date -s '2016-10-25 03:33:21'
Tue Oct 25 03:33:21 UTC 2016
/ # date
Tue Oct 25 03:33:23 UTC 2016
/ #

then if we check again from a docker container:

$ docker run -t node date
Tue Oct 25 03:36:44 UTC 2016
$ date -u
Tue, Oct 25, 2016  3:36:49 PM

so I used the following command from cygwin to quickly sync the time of the MobyLinuxVM with the windows host:

$ docker run --net=host --ipc=host --uts=host --pid=host --security-opt=seccomp=unconfined --privileged --rm alpine date -s "`date -u '+%Y-%m-%d %H:%M:%S'`"
Tue Oct 25 15:49:57 UTC 2016

# check the time again
$ docker run -t node date '+%Y-%m-%d %H:%M:%S'; date -u '+%Y-%m-%d %H:%M:%S'
2016-10-25 15:50:36
2016-10-25 15:50:37

(Phppro) #15

Any update on this? I have to restart docker every morning to get the time correct.


(Mborgmann) #16

I’m using @cristualexandru solution to update the time in the container every time I unlock my computer.

Prerequisites (There should be a possibility for most operating systems):

  • Windows 10 (or a possibility to start a command at login)
  • Installed “bash” (Cygwin or similar) environment installed
  1. Create a file “docker-sync-time.bat” with the following content:

    @echo off

    echo Syncing docker container time…
    bash -c “docker run --net=host --ipc=host --uts=host --pid=host --security-opt=seccomp=unconfined --privileged --rm alpine date -s “$(date -u ‘+%%Y-%%m-%%d %%H:%%M:%%S’)””

  2. Start “Task Scheduler”

  3. Add a task (“Task Scheduler Library” -> “Create New Task…”) that starts the batch script every time you login


(Dcrawford) #17

I use ntp with all my other hyper-v VMs. But, if the time offset is more than 1000s (which always happens after a resume) then ntp refuses to sync the time. But, adding “tinker panic 0” to /etc/ntp.conf allows ntp sync with any offset. I am not sure what the security implications are for this, but it seams to work fine for me. I would suggest including ntp in the Moby VM image with “tinker panic 0”


(Rubenhenriques) #18

Hey :slight_smile:

A quick fix for those who need to restart Docker for Windows every single morning after sleep is:

docker run -it --pid host --privileged --rm debian nsenter -i -m -n -t 1 service ntpd start```

Notice ntpd can take up to 1024 seconds in order to update a container out of sync.

Best regards,
Rúben

(Zeilush) #19

I get this error when executing “get-vm | set-vm -automaticstopaction save”

set-vm : "MobyLinuxVM": Fehler beim Ändern der Einstellungen.

Unerwarteter Fehler: Allgemeiner Fehler (0x80041001).
“MobyLinuxVM”: Fehler beim Ändern der Einstellungen (ID des virtuellen Computers 51BAF990-F1CC-4F25-AB94-A3B7E4B5896C).
MobyLinuxVM: Unerwarteter Fehler: Allgemeiner Fehler (0x80041001). (ID des virtuellen Computers:
51BAF990-F1CC-4F25-AB94-A3B7E4B5896C)
In Zeile:1 Zeichen:10

  • get-vm | set-vm -automaticstopaction save
  •      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : InvalidOperation: (:slight_smile: [Set-VM], VirtualizationException
    • FullyQualifiedErrorId : InvalidState,Microsoft.HyperV.PowerShell.Commands.SetVM

(Rubenhenriques) #20

zeilush,

Set-VM needs MobyLinuxVM to be off, so quit your Docker first and try again.

Note: This is not required as of Docker for Windows version 1.13


(Gustavo Schirmer) #21

Can you guys test something like this?

@powershell $datetime = Get-Date; $dt = $datetime.ToUniversalTime().ToString('yyyy-MM-dd HH:mm:ss'); docker run --net=host --ipc=host --uts=host --pid=host --security-opt=seccomp=unconfined --privileged --rm alpine date -s $dt

but looks like you can solve this going to Hyper-V manager and setting time sync:
MobyLinuxVM > Settings > Integration Services > Time synchronization