Devmapper space issue

Issue type: Generic space issue
OS Version/build: Red Hat Enterprise Linux Server release 7.3 (Maipo)
App version:

    Client:
     Version:         1.12.6
     API version:     1.24
     Package version: docker-common-1.12.6-11.el7.centos.x86_64
     Go version:      go1.7.4
     Git commit:      96d83a5/1.12.6
     Built:           Tue Mar  7 09:23:34 2017
     OS/Arch:         linux/amd64

Server:
 Version:         1.12.6
 API version:     1.24
 Package version: docker-common-1.12.6-11.el7.centos.x86_64
 Go version:      go1.7.4
 Git commit:      96d83a5/1.12.6
 Built:           Tue Mar  7 09:23:34 2017
 OS/Arch:         linux/amd64

Steps to reproduce
I have no containers running currently and have some docker images pertaining to Kubernetes which will be used by the Kubernetes service.

  sudo docker ps -a
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
    [kubeuser4@kubenode4 Employee]$ sudo docker images
    REPOSITORY                                            TAG                 IMAGE ID            CREATED             SIZE
    docker.io/busybox                                     latest              00f017a8c2a6        5 days ago          1.11 MB
    registry.access.redhat.com/rhel7/pod-infrastructure   latest              34d3450d733b        6 weeks ago         205 MB
    docker.io/java                                        8                   d23bdf5b1b1b        8 weeks ago         643.1 MB
    gcr.io/google_containers/heapster_grafana             v2.6.0-2            b43443930626        12 months ago       230 MB

When I try to create a docker image of my application that needs to be used, I get the below error.

devmapper: Thin Pool has 8783 free data blocks which is less than minimum required 163840 free data blocks. Create more free space in thin pool or use dm.min_free_space option to change behavior

I tried the cleaning up as mentioned in the other forums, but not helped much and getting the same error. When I tried to run with this sudo docker --storage-opt dm.min_free_space=0%, seems like it starts as a daemon, but still it failed with another error “docker-runc not installed on system” and also I dont want to run it as a daemon.

Below are some command outputs

sudo dmsetup status
localvg00-lv_home: 0 20971520 linear
localvg00-lv_home: 20971520 20971520 linear
docker-251:5-134039-pool: 0 209715200 thin-pool 924 848/524288 1629226/1638400 - rw discard_passdown queue_if_no_space
localvg00-lv_tmp: 0 4194304 linear
localvg00-lv_swap: 0 8388608 linear
localvg00-lv_root: 0 2097152 linear
localvg00-lv_root: 2097152 20971520 linear
localvg00-lv_usr: 0 16777216 linear
localvg00-lv_var: 0 8388608 linear
localvg00-lv_var: 8388608 62914560 linear

sudo docker info
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 4
Server Version: 1.12.6
Storage Driver: devicemapper
 Pool Name: docker-251:5-134039-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 10.74 GB
 Backing Filesystem: xfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 106.8 GB
 Data Space Total: 107.4 GB
 Data Space Available: 601.2 MB
 Metadata Space Used: 3.473 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.144 GB
 Thin Pool Minimum Free Space: 10.74 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 WARNING: Usage of loopback devices is strongly discouraged for production use. Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.135-RHEL7 (2016-11-16)
Logging Driver: journald
Cgroup Driver: systemd
Plugins:
 Volume: local
 Network: overlay null bridge host
Swarm: inactive
Runtimes: runc docker-runc
Default Runtime: docker-runc
Security Options: seccomp
Kernel Version: 4.1.12-61.1.28.el7uek.x86_64
Operating System: Oracle Linux Server 7.3
OSType: linux
Architecture: x86_64
Number of Docker Hooks: 2
CPUs: 2
Total Memory: 7.545 GiB
Name: kubenode4

I had also tried increasing all the physical volume size and logical volume size on my linux machine, but still it doesnt work. Kindly help.

I seriously think that there should be more documentation provided like this with the latest ongoing versions in Docker

https://jpetazzo.github.io/2014/01/29/docker-device-mapper-resize/

I find myself in a similar spot like this guy

but it seems like there is limited help and documentation available on this for beginners w.r.t Linux Filesystem.

Folks are not even viewing in stack overflow if they see anything related to Docker devicemapper storage issue.

Would make life easier if there is someone out who could help and also kindly improve the documentation on this subject.

Thanks,

thanks for noticing. to be honest i have given up trying to get docker working, but if anyone can help me out, i’ll have another go.

After several days of learning basics on Linux storage, I was able to get over the issue by creating a new disk and following the instructions in this site.

https://platform9.com/support/preparing-centos-7-system-running-containers/

Tried to get it working on the same disk as the one which I was using, but wasnt able to. Think it is not possible also without setting on a new disk. Hope this helps in future.