Docker Community Forums

Share and learn in the Docker community.

Overlay2 - docker images lost after reboot

Hello,

I’ve been using docker for 4 years inside an Ubuntu 14.04 vagrant VM (windows host) with devicemapper fs without^^ problems.
I recently switched to linux mint 19.3 running docker (19.03.11) on the host with overlay2.
It’s been twice that after a reboot, I lost all my docker images, only the running ones were not deleted.
I confirmed that the images layers were well gone by checking in the docker/data/overlay2.

Linux mint frequently hangs on shutdown so I don’t know if that may be the issue.
Apart from that, I don’t do anything off the chart expect having change the path where docker store its data with a daemon.json file: “data-root”: “/home/myuser/.docker/data”

Let me know if I can provide more info.

Steps to reproduce the issue:

  1. apt-get install -y docker-ce
  2. systemctl stop docker
  3. mkfs.xfs -f -n ftype=1 /dev/sdb1
  4. mount /dev/sdb1 /var/lib/docker/overlay2/
  5. systemctl restart docker
    Describe the results you received:
    using the lsblk the docker mount point (/var/lib/docker/overlay2) is lost, and the previous pulled images are broken.

Describe the results you expected:
anywhere would be the mount point, not only the /var/lib/docker directory, so If docker storage mount it to /var/lib/docker/overlay2 to an individual disk, the other docker dependent component such as image, network, runtimes… would also be installed in system disk (although, this it’s a bad practice), but the mount point shouldn’t be lost as well as the docker image data.

Note
If I changed the mount point to its parent directory, such as mount /dev/sdb1 /var/lib/docker, it WILL NOT lost and be OK after restart docker.service

Output of docker version:

root@QianZhangDGX1-1:~# docker version
Client:
Version: 18.09.2
API version: 1.39
Go version: go1.10.6
Git commit: 6247962
Built: Sun Feb 10 04:13:47 2019
OS/Arch: linux/amd64
Experimental: false

Server: Docker Engine - Community
Engine:
Version: 18.09.2
API version: 1.39 (minimum version 1.12)
Go version: go1.10.6
Git commit: 6247962
Built: Sun Feb 10 03:42:13 2019
OS/Arch: linux/amd64
Experimental: false
Output of docker info:

root@QianZhangDGX1-1:~# docker info
Containers: 46
Running: 29
Paused: 0
Stopped: 17
Images: 17
Server Version: 18.09.2
Storage Driver: overlay2
Backing Filesystem: xfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: nvidia runc
Default Runtime: nvidia
Init Binary: docker-init
containerd version: 9754871865f7fe2f4e74d43e2fc7ccd237edcbce
runc version: 6635b4f0c6af3810594d2770f662f34ddc15b40d-dirty
init version: fec3683
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.15.0-45-generic
Operating System: Ubuntu 18.04.2 LTS
OSType: linux
Architecture: x86_64
CPUs: 80
Total Memory: 503.8GiB
Name: QianZhangDGX1-1
ID: ORK7:SABZ:3UTZ:23CS:RMQZ:M5ER:Q4UY:3KD2:LDIF:L2LD:KWDF:H5WB
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
docker-registry.xxx.com
127.0.0.0/8
Registry Mirrors:
https://docker-registry.xxx.com/
Live Restore Enabled: false
Product License: Community Engine

WARNING: No swap limit support

Hello, has this been fixed in the latest release? I am still having this problem and have to recreate all my docker images each time.