Why does docker prompt "Permission denied" when backing up the data volume?

This topic is from SO, but can’t resolve it, so I repost it here, thanks!

I am following the docker document to test the backup process of data volumes.

The following 2 steps are all OK:

docker create -v /dbdata --name dbdata training/postgres /bin/true
docker run -d --volumes-from dbdata --name db1 training/postgres

But the output of backing up data is:

[root@localhost data]# docker run --volumes-from dbdata -v $(pwd):/backup ubuntu tar cvf /backup/backup.tar /dbdata
tar: /backup/backup.tar: Cannot open: Permission denied
tar: Error is not recoverable: exiting now
[root@localhost data]# pwd
/root/data
[root@localhost data]# ls -alt
total 4
drwxrwxrwx.  2 root root    6 May  7 21:33 .
drwxrwx-w-. 15 root root 4096 May  7 21:33 ..

I am working as a root user, so why does it prompt “Permission denied”?

After executing debug command:

docker run --name ins --volumes-from dbdata -v $(pwd):/backup ubuntu sleep 99999 &
docker inspect ins

The output is:

	[{
	"AppArmorProfile": "",
	"Args": [
		"99999"
	],
	"Config": {
		"AttachStderr": true,
		"AttachStdin": false,
		"AttachStdout": true,
		"Cmd": [
			"sleep",
			"99999"
		],
		"CpuShares": 0,
		"Cpuset": "",
		"Domainname": "",
		"Entrypoint": null,
		"Env": null,
		"ExposedPorts": null,
		"Hostname": "83e3e1715648",
		"Image": "ubuntu",
		"MacAddress": "",
		"Memory": 0,
		"MemorySwap": 0,
		"NetworkDisabled": false,
		"OnBuild": null,
		"OpenStdin": false,
		"PortSpecs": null,
		"StdinOnce": false,
		"Tty": false,
		"User": "",
		"Volumes": null,
		"WorkingDir": ""
	},
	"Created": "2015-05-08T01:36:35.564512894Z",
	"Driver": "devicemapper",
	"ExecDriver": "native-0.2",
	"ExecIDs": null,
	"HostConfig": {
		"Binds": [
			"/root/data:/backup"
		],
		"CapAdd": null,
		"CapDrop": null,
		"ContainerIDFile": "",
		"Devices": [],
		"Dns": null,
		"DnsSearch": null,
		"ExtraHosts": null,
		"IpcMode": "",
		"Links": null,
		"LxcConf": [],
		"NetworkMode": "bridge",
		"PidMode": "",
		"PortBindings": {},
		"Privileged": false,
		"PublishAllPorts": false,
		"ReadonlyRootfs": false,
		"RestartPolicy": {
			"MaximumRetryCount": 0,
			"Name": ""
		},
		"SecurityOpt": null,
		"VolumesFrom": [
			"dbdata"
		]
	},
	"HostnamePath": "/var/lib/docker/containers/83e3e171564841460b206a8699c1890e2b910bcd2232fdc7202cbff9210b5362/hostname",
	"HostsPath": "/var/lib/docker/containers/83e3e171564841460b206a8699c1890e2b910bcd2232fdc7202cbff9210b5362/hosts",
	"Id": "83e3e171564841460b206a8699c1890e2b910bcd2232fdc7202cbff9210b5362",
	"Image": "07f8e8c5e66084bef8f848877857537ffe1c47edd01a93af27e7161672ad0e95",
	"MountLabel": "system_u:object_r:svirt_sandbox_file_t:s0:c414,c650",
	"Name": "/ins",
	"NetworkSettings": {
		"Bridge": "docker0",
		"Gateway": "172.17.42.1",
		"GlobalIPv6Address": "",
		"GlobalIPv6PrefixLen": 0,
		"IPAddress": "172.17.0.6",
		"IPPrefixLen": 16,
		"IPv6Gateway": "",
		"LinkLocalIPv6Address": "fe80::42:acff:fe11:6",
		"LinkLocalIPv6PrefixLen": 64,
		"MacAddress": "02:42:ac:11:00:06",
		"PortMapping": null,
		"Ports": {}
	},
	"Path": "sleep",
	"ProcessLabel": "system_u:system_r:svirt_lxc_net_t:s0:c414,c650",
	"ResolvConfPath": "/var/lib/docker/containers/83e3e171564841460b206a8699c1890e2b910bcd2232fdc7202cbff9210b5362/resolv.conf",
	"RestartCount": 0,
	"State": {
		"Error": "",
		"ExitCode": 0,
		"FinishedAt": "0001-01-01T00:00:00Z",
		"OOMKilled": false,
		"Paused": false,
		"Pid": 3614,
		"Restarting": false,
		"Running": true,
		"StartedAt": "2015-05-08T01:36:36.231389015Z"
	},
	"Volumes": {
		"/backup": "/root/data",
		"/dbdata": "/var/lib/docker/vfs/dir/df0378f15f61c8f2e220421968fe181cdcf1a03613218c716c81477dda4bdf76"
	},
	"VolumesRW": {
		"/backup": true,
		"/dbdata": true
	}
}
]

The root cause is about selinux, set it as permissive mode, it works!

I had a similar issue. Not an selinux expert but trying to find a resolution without completely disabling it.
It turned out that the readonly flag on the mounted volume in my docker run was setting an selinux category (MCS) on the folder therefore giving me a permission denied.
Still investigating why the readonly flag is causing that problem.

To know if you have the same issue do ls -Z on the directory. Mine showed the following
drwx—r–. vmail vmail system_u:object_r:svirt_sandbox_file_t:s0:c334,c451 mailboxes
I ran find . -exec chcat – -c334 {} ; on each category and remove the readonly flag for now.

@mennatm: Thanks for your reply!

The following is my docker container environment:

root@69d0c0c39fff:/# ls -altZ
total 72
drwxrwxrwx.   2 root root unconfined_u:object_r:admin_home_t:s0                28 May 14 09:00 backup
drwxr-xr-x.  10 root root system_u:object_r:svirt_sandbox_file_t:s0:c84,c973 4096 May 14 08:33 usr

The tar doesn’t work OK in backup directory but OK in usr:

root@69d0c0c39fff:/# tar cvf /backup/gitlab_data.tar /var/opt/gitlab /var/log/gitlab /etc/gitlab
tar: /backup/gitlab_data.tar: Cannot open: Permission denied
tar: Error is not recoverable: exiting now
root@69d0c0c39fff:/# tar cvf /usr/gitlab_data.tar /var/opt/gitlab /var/log/gitlab /etc/gitlab
tar: Removing leading `/' from member names
/var/opt/gitlab/
/var/opt/gitlab/.gitconfig
....

I am a little confused about the sensitivity and categories, if possible, could you comment on this? Thanks in advance!

Selinux has an MLS and MCS option which sets up files/folders or users with certain sensitivity and category level. This is the s0:c84,c973 part
See http://selinuxproject.org/page/NB_MLS

What I do is run the following command on all mounted docker volumes
chcon --recursive --type=svirt_sandbox_file_t --range=s0
which resets the right folder with the right permissions