Apparmor denies everything

Docker Engine General
1

Hi,

I just installed Docker or on a Debian 11 server. Then I tired to run

docker run -p 3000:80 nginx:latest

To test if the installation is all good and I can run containers.

However the output I got was:

/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2022/12/12 11:25:09 [emerg] 1#1: socket() 0.0.0.0:80 failed (13: Permission denied)
nginx: [emerg] socket() 0.0.0.0:80 failed (13: Permission denied)

This is the relevant content of syslog:

Dec 12 12:25:05 betalayout-8 systemd[1]: var-lib-docker-overlay2-f04128770a0f76717a889f42cb01517377abb2d8eb332ba3dda6fd0fba69a6e3\x2dinit-merged.mount: Succeeded.
Dec 12 12:25:06 betalayout-8 systemd[1]: var-lib-docker-overlay2-f04128770a0f76717a889f42cb01517377abb2d8eb332ba3dda6fd0fba69a6e3-merged.mount: Succeeded.
Dec 12 12:25:06 betalayout-8 systemd[2198]: var-lib-docker-overlay2-f04128770a0f76717a889f42cb01517377abb2d8eb332ba3dda6fd0fba69a6e3-merged.mount: Succeeded.
Dec 12 12:25:07 betalayout-8 kernel: [ 1624.815397] docker0: port 1(vethfc1ff6e) entered blocking state
Dec 12 12:25:07 betalayout-8 kernel: [ 1624.815407] docker0: port 1(vethfc1ff6e) entered disabled state
Dec 12 12:25:07 betalayout-8 kernel: [ 1624.815496] device vethfc1ff6e entered promiscuous mode
Dec 12 12:25:07 betalayout-8 systemd-udevd[4612]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Dec 12 12:25:07 betalayout-8 systemd-udevd[4612]: Using default interface naming scheme 'v247'.
Dec 12 12:25:07 betalayout-8 systemd-udevd[4613]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Dec 12 12:25:07 betalayout-8 systemd-udevd[4613]: Using default interface naming scheme 'v247'.
Dec 12 12:25:08 betalayout-8 containerd[2480]: time="2022-12-12T12:25:08.564032100+01:00" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
Dec 12 12:25:08 betalayout-8 containerd[2480]: time="2022-12-12T12:25:08.564231564+01:00" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
Dec 12 12:25:08 betalayout-8 containerd[2480]: time="2022-12-12T12:25:08.564271776+01:00" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Dec 12 12:25:08 betalayout-8 containerd[2480]: time="2022-12-12T12:25:08.564751200+01:00" level=info msg="starting signal loop" namespace=moby path=/run/containerd/io.containerd.runtime.v2.task/moby/e276b9bca6b2124a791fdb09216357a0208c56462969aae48b84c2603920d16c pid=4656 runtime=io.containerd.runc.v2
Dec 12 12:25:08 betalayout-8 systemd[2198]: run-docker-runtime\x2drunc-moby-e276b9bca6b2124a791fdb09216357a0208c56462969aae48b84c2603920d16c-runc.IrwhNB.mount: Succeeded.
Dec 12 12:25:08 betalayout-8 systemd[1]: run-docker-runtime\x2drunc-moby-e276b9bca6b2124a791fdb09216357a0208c56462969aae48b84c2603920d16c-runc.IrwhNB.mount: Succeeded.
Dec 12 12:25:08 betalayout-8 systemd[1]: Started libcontainer container e276b9bca6b2124a791fdb09216357a0208c56462969aae48b84c2603920d16c.
Dec 12 12:25:08 betalayout-8 kernel: [ 1625.861653] eth0: renamed from vethfd1bc3a
Dec 12 12:25:08 betalayout-8 kernel: [ 1625.877753] IPv6: ADDRCONF(NETDEV_CHANGE): vethfc1ff6e: link becomes ready
Dec 12 12:25:08 betalayout-8 kernel: [ 1625.877812] docker0: port 1(vethfc1ff6e) entered blocking state
Dec 12 12:25:08 betalayout-8 kernel: [ 1625.877819] docker0: port 1(vethfc1ff6e) entered forwarding state
Dec 12 12:25:09 betalayout-8 kernel: [ 1626.293983] audit: type=1400 audit(1670844309.296:42): apparmor="DENIED" operation="create" profile="docker-default" pid=4677 comm="nginx" family="unix" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create" addr=none
Dec 12 12:25:09 betalayout-8 kernel: [ 1626.293999] audit: type=1400 audit(1670844309.296:43): apparmor="DENIED" operation="create" profile="docker-default" pid=4677 comm="nginx" family="unix" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create" addr=none
Dec 12 12:25:09 betalayout-8 kernel: [ 1626.294433] audit: type=1400 audit(1670844309.296:44): apparmor="DENIED" operation="create" profile="docker-default" pid=4677 comm="nginx" family="unix" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create" addr=none
Dec 12 12:25:09 betalayout-8 kernel: [ 1626.294445] audit: type=1400 audit(1670844309.296:45): apparmor="DENIED" operation="create" profile="docker-default" pid=4677 comm="nginx" family="unix" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create" addr=none
Dec 12 12:25:09 betalayout-8 kernel: [ 1626.297309] audit: type=1400 audit(1670844309.300:46): apparmor="DENIED" operation="create" profile="docker-default" pid=4677 comm="nginx" family="inet" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create"
Dec 12 12:25:09 betalayout-8 systemd[1]: docker-e276b9bca6b2124a791fdb09216357a0208c56462969aae48b84c2603920d16c.scope: Succeeded.
Dec 12 12:25:09 betalayout-8 dockerd[2578]: time="2022-12-12T12:25:09.338018228+01:00" level=info msg="ignoring event" container=e276b9bca6b2124a791fdb09216357a0208c56462969aae48b84c2603920d16c module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Dec 12 12:25:09 betalayout-8 containerd[2480]: time="2022-12-12T12:25:09.339017432+01:00" level=info msg="shim disconnected" id=e276b9bca6b2124a791fdb09216357a0208c56462969aae48b84c2603920d16c
Dec 12 12:25:09 betalayout-8 containerd[2480]: time="2022-12-12T12:25:09.339142256+01:00" level=warning msg="cleaning up after shim disconnected" id=e276b9bca6b2124a791fdb09216357a0208c56462969aae48b84c2603920d16c namespace=moby
Dec 12 12:25:09 betalayout-8 containerd[2480]: time="2022-12-12T12:25:09.339186404+01:00" level=info msg="cleaning up dead shim"
Dec 12 12:25:09 betalayout-8 containerd[2480]: time="2022-12-12T12:25:09.357958856+01:00" level=warning msg="cleanup warnings time=\"2022-12-12T12:25:09+01:00\" level=info msg=\"starting signal loop\" namespace=moby pid=4736 runtime=io.containerd.runc.v2\n"
Dec 12 12:25:09 betalayout-8 kernel: [ 1626.519759] docker0: port 1(vethfc1ff6e) entered disabled state
Dec 12 12:25:09 betalayout-8 kernel: [ 1626.519873] vethfd1bc3a: renamed from eth0
Dec 12 12:25:09 betalayout-8 systemd-udevd[4619]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Dec 12 12:25:09 betalayout-8 systemd-udevd[4619]: Using default interface naming scheme 'v247'.
Dec 12 12:25:09 betalayout-8 systemd-udevd[4619]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Dec 12 12:25:09 betalayout-8 kernel: [ 1626.671222] docker0: port 1(vethfc1ff6e) entered disabled state
Dec 12 12:25:09 betalayout-8 kernel: [ 1626.672696] device vethfc1ff6e left promiscuous mode
Dec 12 12:25:09 betalayout-8 kernel: [ 1626.672711] docker0: port 1(vethfc1ff6e) entered disabled state
Dec 12 12:25:09 betalayout-8 systemd[1]: run-docker-netns-2d55c268c616.mount: Succeeded.
Dec 12 12:25:09 betalayout-8 systemd[2198]: run-docker-netns-2d55c268c616.mount: Succeeded.
Dec 12 12:25:09 betalayout-8 systemd[2198]: var-lib-docker-overlay2-f04128770a0f76717a889f42cb01517377abb2d8eb332ba3dda6fd0fba69a6e3-merged.mount: Succeeded.
Dec 12 12:25:09 betalayout-8 systemd[1]: var-lib-docker-overlay2-f04128770a0f76717a889f42cb01517377abb2d8eb332ba3dda6fd0fba69a6e3-merged.mount: Succeeded.

As it seems apparmor blocks the nginx process. I tried with other containers and I get a similar issue.

This server had originally Kubernetes installed which encountered the similar DENIED with most of it’s containers. Then I removed Kubernetes, Docker and containerd completely and installed Docker from fresh. However, this did not solve my issue.

Any ideas?

2

Have you checked this guide? It also mentions Nginx

3

Szia Ákos!

Yes, I have. I understand that I can create a custom profile which will allow what I need for a container. However, the issue I have is not specific for nginx. It affects all containers and it not used to be this strict. For example installation of Kubernetes would be:

  • Install docker
  • Install kubeadm, kubectl and kubelet
  • Kubeadm init
  • Wait a few minutes and enjoy the newly setup Kubernetes instance.

I have done this many times on various machines without any issues.

However, on this particular machine Apparmor denies any container that tries to access anything outside of the container. In nginx’s is case to open the required port. In Kubernetes case it is with the kuber-apiserver.

Unfortunately, There is not much resources that I could find on this topic.

4

Yes, I thought that, but your error log came from AppArmor and not from Docker. I know I need to learn more about AppArmor, but I can’t tell you what went wrong and I hoped you could find some rules that make AppArmor so strict. I still hope you will find something, because I don’t have a Debian 11 to test it and I don’t have time for a longer investigation :frowning: