In place docker service upgrades on systems with running containers


I have a question about docker service upgrades. I have several hosts that are running x number of containers. I do NOT wish to stop the containers or the host (downtime :(). I do however wish to upgrade the docker-ce installation running underneath these containers. Specifically, I want to upgrade from “Docker version 17.03.1-ce, build c6d412e” to “Docker version 17.12.1-ce, build 7390fc6”. I use chef to provision these systems. The upgrade works fine during testing when I update my recipe to the new version and run it on a system running the older version of docker; as long as I don’t have any containers running at the time. If I do have containers running when I attempt the upgrade, the ‘dockerd-default’ file version does NOT get updated and the service cannot restart. Most of the other docker binaries in the /usr/bin folder do get the new version. What is the recommended path forward in this scenario?