Configuring Encryption for Swarm Overlay network in Compose

Docker Engine Compose
1

Hi, After much reading it seems that “secure out-of-the-box” does not include the data I want to send between data centers. I think it is great that the inter-workings of orchestrating the swarm clusters are automatically secure, and that the support for securing my data exchanges is already built in; this makes Docker Swarm the winner for deployment. I can see the justification for defaulting to plain text, as many data centers may already have handled building secure links and not want unneeded overhead of additional encryption.

However, how do I enable it in a compose file? I see the “–opt encrypted” (sometimes referred to --opt secure) in the command line documentation and web search results.

Except, how that maps to the compose file is not obvious. I’ve tried:
"driver_opts: encrypted"
and
"driver_opts:

  • opt: encrypted"
    and a few more permutations. So far, I have not been able to determine if the encryption mode is set. I’ve used “docker network inspect” and done some tcpdumps to try to definitively determine if my overlay network is encrypted.

So, I would like to have my overlay network named “backend” encrypted, and, because is seems that syntax errors could silently result in no encryption being enabled, how can I definitively verify that encryption is taking place.

Thanks…

1 Like
2

Hi,

I’m going to answer my own question since, after much additional reading and experimentation, I have been able to solve this compose file configuration issue.

Here is the successful configuration:

networks:
  my-app-backend:
    driver: overlay
    driver_opts:
      encrypted: ""

Using ‘encrypted: “true”’ also works. I stumbled upon this excellent article:
https://github.com/docker/labs/blob/master/networking/concepts/11-security.md the author used “–opt encrypted=true” in his command line example rather than “–opt encrypted”. It works in the compose file, too.

I deployed a swarm that had the visualizer container (dockersamples/visualizer:stable) on the first node and a container, on a second node, that had nslookup and wget included (curl would be fine, too). Both of these containers were hooked to the “my-app-backend” overlay network. I named the visualizer “viz” and did “wget viz:8080” from inside the container on the second node.

I reduced the tcpdump command to the minimum viable command options and had it waiting for traffic, running on the first container. “sudo tcpdump -p esp”.

And presto… encrypted traffic flowed.

sudo tcpdump -p esp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
11:13:27.454802 IP 46.101.xx.xxx > demo-docker-swarm-node01: ESP(spi=0x8e56b83c,seq=0x4), length 180
11:13:27.454897 IP demo-docker-swarm-node01 > 46.101.xx.xxx: ESP(spi=0x02c6bf84,seq=0x3), length 108
11:13:27.498887 IP demo-docker-swarm-node01 > 46.101.xx.xxx: ESP(spi=0x02c6bf84,seq=0x4), length 1480
11:13:27.498935 IP demo-docker-swarm-node01 > 46.101.xx.xxx: ESP(spi=0x02c6bf84,seq=0x5), length 856

Ok, now just a little suggestion (and peeve).

As an example, I modified the configuration key to be ‘"xencrypted: “”’. No error was reported on deployment, but the network traffic was not encrypted, either.

The beauty of all this is you have baked in encryption to the point where it is just part of the fabric. The bad point is it should not be easy for simple configuration errors to go unchecked and result in the network silently reverting to clear text.

I think the encryption switch should be promoted to a first class switch like ‘network_encrypted: “true/false”’. This way the syntax can be accurately checked. So the simple things don’t undo all the hard work you put into implementing security. Possibility, “Secrets and Lies” by Bruce Schneier may be apropos.

4 Likes
3

Thank you. How ridiculous that we’ve had to just try a bunch of permutations until one works. It’d be really nice if they actually documented compose file examples instead of just CLI examples.

4

Do you have to add the ‘encrypted’ option even if you created the network externally?
I have this network declaration in my compose file but still haven’t managed to check traffic get encrypted

networks:
   default:
       external:
          name: swarm-stack
5

I’m going to answer my same question in case it can be helpful to other people.
The network declaration I posted is correct.

6

Just an update on this: using compose version: '3.2', with encrypted: true raises this error if you attempt to deploy a stack:

networks.otp-net.driver_opts.encrypted must be a string or number
1 Like
7

An unencrypted network.

$ docker network inspect unencrypted
...
        "Options": {
            "com.docker.network.driver.overlay.vxlanid_list": "4098"
        },
...

An encrypted network.

$ docker network inspect encrypted
...
        "Options": {
            "com.docker.network.driver.overlay.vxlanid_list": "4098",
            "encrypted": "true"
        },
        "Peers": [
            {
                "Name": "worker-01-0d6b662f1f82",
                "IP": "192.168.1.102"
            }
        ]