I/O not stable in Docker containers (CentOS 6.7, docker 1.7.1)

The I/O in the Docker containers is not very stable. When we download a zip file from a container the download time is very jumpy. When we look at the IO usage on the Docker engine (running on CentOS 6.7) with ‘iotop’ we see that the reading is changing a lot between 500 Kb and 10Mb. The containers and the host are not very busy, CPU usage is around 10% and memory usage about 25% and those are stable when the download is busy.
When we download a file from the Docker host it is stable, it seemed like that the Docker engine is the cause of the unstableness. Does anyone has a clue for this?
Next to this the containers respond sometimes very fast as expected, but sometimes also have delays of about 10 to 30 seconds to show a simple webpage on Nginx or Apache. Probably also the same issue.

Running Docker engine 1.7.1 on CentOS 6.7 which is running in a VM.

mount ; docker info
/dev/mapper/vg_centos6-lv_root on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
/dev/sda1 on /boot type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
Containers: 10
Images: 265
Storage Driver: devicemapper
Pool Name: docker-253:0-1835199-pool
Pool Blocksize: 65.54 kB
Backing Filesystem: extfs
Data file: /dev/loop0
Metadata file: /dev/loop1
Data Space Used: 13.55 GB
Data Space Total: 107.4 GB
Data Space Available: 93.82 GB
Metadata Space Used: 19.08 MB
Metadata Space Total: 2.147 GB
Metadata Space Available: 2.128 GB
Udev Sync Supported: true
Deferred Removal Enabled: false
Data loop file: /var/lib/docker/devicemapper/devicemapper/data
Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
Library Version: 1.02.95-RHEL6 (2015-07-29)
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 2.6.32-573.3.1.el6.x86_64
Operating System: <unknown>
CPUs: 4
Total Memory: 11.63 GiB

Does this happen on the container’s layered filesystem? Does it persist if you download to a volume instead?

Can you share all the information you have on your host’s filesystem, and the storage driver you are using? mount ; docker info should return that.

/Jeff

I added the output of ‘mount; docker info’ to this post. The files which are download are on a volume in the host filesystem. These volumes will be ‘mounted’ by the docker containers with the -v option.

Has anyone an idea why the I/O can be so unstable?