Isolated hybrid networks

Hi all!
I’m running docker 17.03.0-ce in a swarm.
I have two services, lets call them Service-A and Service-B and they are deployed with docker-compose.
Both Service-A and B needs to connect to different backend networks where different backend services provides resources (for example Service-A needs a MySQL database and Service-B needs a NFS share).
The network is defined in the infrastructure (different ip-ranges and vlan) and is not routed, the only way to access the resources from the infrastructure point of view is to be on the same L2 network.
In docker, I don’t want either service to be able to reach a resource that exist on a network the service hasn’t been assigned to.
None of the Service-A or Service-B is able to connect to the service when I used the steps below:

First, define some variables to have a consistent naming convention.

NETNS=be01-ns
ExtIF=bond1.521
VETH=be01-eth
BR=be01-br
BRIP=10.5.21.11/24
DockerNet=be01-net

Setup the namespace and add veth’s

ip netns add $NETNS
ip link set $ExtIF netns $NETNS
ip link add $VETH-e0 type veth peer name $VETH-e1
ip link set $VETH-e1 netns $NETNS

Setup a bridge and interfaces in new namespace

ip netns exec $NETNS ip link add name $BR type bridge
ip netns exec $NETNS ip link set dev $VETH-e1 up
ip netns exec $NETNS ip link set dev $BR up
ip netns exec $NETNS ip link set dev $ExtIF up
ip netns exec $NETNS ip link set dev $ExtIF master $BR
ip netns exec $NETNS ip link set dev $VETH-e1 master $BR
ip netns exec $NETNS ip addr flush $ExtIF
ip netns exec $NETNS ip addr add $BRIP dev $BR

Test connectivity to a resource in the backend network

ip netns exec $NETNS ping 10.10.10.10

Setup a docker overlay network:

docker network create -d overlay -o parent=$VETH-e0 --subnet=10.10.10.0/24 --ip-range=10.10.10.128/27 $DockerNet

Deploy Service-A and just a simple oneliner to test connectivity from the docker instance to backend resource.

docker stack deploy -c docker-compose.yml helloworld
docker exec -it $(docker ps | awk '$(NF) ~ /helloworld-be/ {print $(1)}') ping 10.10.10.10

The docker-compose.yml looks like:

---
version: '3'
services:
  helloworld-be:
    image: dockercloud/hello-world
    networks:
      - traefik-net	
      - be01-net
networks:
  traefik-net:
    external:
      name: traefik-net
  be01-net:
    external:
      name: be01-net
```
I don't see any packets on any of the interfaces on the docker host.
Please let me know if I need to clarify the purpose or setup more, I really need help with this since it's a showstopper in our setup.

Hej Robert,

Some thinking:

I don’t think the overlay driver support the parent option. Looking in the source code ie. MACVLAN driver supports this option:

Currently overlay driver has these options: https://github.com/docker/libnetwork/blob/bb4fe0b6f806412fc671348bfdc42a693c00a0ae/drivers/overlay/overlay.go#L21

Reading about MACVLAN in https://docs.docker.com/engine/userguide/networking/get-started-macvlan/ and your question … makes me suspect you have a MACVLAN scenario… and that overlay networking might not suite you in this situation.

Do you want to directly connect your container directly to the network and so it will have a IP from the underlying network ?

If the overlay network would work… it would connect through a the tunnel over the underlying network (Data plane (VXLAN): udp 4789, Control plane: tcp/udp 7946).

Sincerely
Mårten Cassel
@Conoa.se

Hi Mårten!
Sorry for the delay, been busy with other things.
I created the macvlan like this:
docker network create -d macvlan --subnet=10.5.21.0/24 --ip-range=10.5.21.128/28 -o bond1.521 be-mv01
And when I try to deploy the service (docker stack deploy -c Docker/traefik/docker-compose.yml traefik) i get the error message:
network "be-mv01" is declared as external, but it is not in the right scope: "local" instead of "swarm"