Containers unable to access network resources with Macvlan driver and vice versa

[smartaim]

I am running Docker on an Ubuntu Server 18.4 which runs on a virtual machine on Oracle VM VirtualBox in Windows. I am trying to use the Macvlan driver so that each container will get its own IP address. This is all running fine, containers can access eachother but not the host as expected. So I also added a sub interface to make sure that containers can also access the host and also this part works nicely.

However, none of the containers can access the gateway or any other resource outside of my Windows machine. From my Windows machine I can ping the containers, but machines in the same network cannot. I have tried many things but I am really not a network expert. I have spent days on many websites (including this one) to figure out what the problem is. I’ve investigated enough to know that it should be possible, but I am really stuck on how to.

Here is what I have done and configured:

Create Macvlan network:

docker network create --driver macvlan --subnet=192.168.179.0/24 --gateway=192.168.179.1 --ip-range=192.168.179.180/30 --opt parent=enp0s3 macvlan

Result of docker network inspect macvlan:

[
    {
        "Name": "macvlan",
        "Id": "3d15f079dee64234fca1c4c287f2f90e52a9462fc41e33a06be72b799156a2a7",
        "Created": "2018-09-28T14:11:30.165086974+02:00",
        "Scope": "local",
        "Driver": "macvlan",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "192.168.179.0/24",
                    "IPRange": "192.168.179.180/30",
                    "Gateway": "192.168.179.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "a1618dc90a2d31389e870ba3569874f2294ba251fe70e98a3f8f8bcf77e1fbf3": {
                "Name": "SIMS-603",
                "EndpointID": "f7453ebf1985efb0840a833f86446e2d29d00ea0239c22b840bd7c9c4618d3f6",
                "MacAddress": "02:42:c0:a8:b3:b6",
                "IPv4Address": "192.168.179.182/24",
                "IPv6Address": ""
            },
            "b25c20e663c3bf5865851474f041fe9e405285e3ab486c447cbeaf6c10d7ca8c": {
                "Name": "SIMS-462",
                "EndpointID": "c69f15191333daef4c6ddbeaa852ef034884d918d1fd4e3118f6e31bd19fec66",
                "MacAddress": "02:42:c0:a8:b3:b5",
                "IPv4Address": "192.168.179.181/24",
                "IPv6Address": ""
            },
            "dfa650e85ab3e44363be0e6c369b1bf4e5f32b9598d5c73422f0d7223f4c67d8": {
                "Name": "SIMS-363",
                "EndpointID": "3ac556459d8893d0fea770480cb8e589eb2623c881297027dad2fe8db352de5f",
                "MacAddress": "02:42:c0:a8:b3:b4",
                "IPv4Address": "192.168.179.180/24",
                "IPv6Address": ""
            }
        },
        "Options": {
            "parent": "enp0s3"
        },
        "Labels": {}
    }
]

Syntax to create sub interface:

ip link add macvlan-shim link enp0s3 type macvlan mode bridge
ip addr add 192.168.179.190/24 dev macvlan-shim
ip link set macvlan-shim up
ip route add 192.168.179.180/30 dev macvlan-shim

Result of ifconfig:

docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        inet6 fe80::42:8aff:fe58:26fe  prefixlen 64  scopeid 0x20<link>
        ether 02:42:8a:58:26:fe  txqueuelen 0  (Ethernet)
        RX packets 4113  bytes 1483941 (1.4 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 5009  bytes 2962359 (2.9 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

enp0s3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.179.165  netmask 255.255.255.0  broadcast 192.168.179.255
        inet6 fe80::a00:27ff:fee3:8264  prefixlen 64  scopeid 0x20<link>
        ether 08:00:27:e3:82:64  txqueuelen 1000  (Ethernet)
        RX packets 157563  bytes 58704409 (58.7 MB)
        RX errors 36  dropped 0  overruns 0  frame 0
        TX packets 6119  bytes 1975077 (1.9 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 19  base 0xd020

enp0s8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.2.10  netmask 255.255.255.0  broadcast 10.0.2.255
        inet6 fe80::a00:27ff:fe81:fc78  prefixlen 64  scopeid 0x20<link>
        ether 08:00:27:81:fc:78  txqueuelen 1000  (Ethernet)
        RX packets 53273  bytes 76803428 (76.8 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 28332  bytes 1670574 (1.6 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 16  base 0xd240

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 138  bytes 11088 (11.0 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 138  bytes 11088 (11.0 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

macvlan-shim: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.179.190  netmask 255.255.255.0  broadcast 0.0.0.0
        inet6 fe80::3c4a:65ff:fec5:b7a8  prefixlen 64  scopeid 0x20<link>
        ether 3e:4a:65:c5:b7:a8  txqueuelen 1000  (Ethernet)
        RX packets 1234  bytes 88246 (88.2 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 12  bytes 936 (936.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

veth5561744: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::c49d:a2ff:fe7b:4ccf  prefixlen 64  scopeid 0x20<link>
        ether c6:9d:a2:7b:4c:cf  txqueuelen 0  (Ethernet)
        RX packets 4113  bytes 1541523 (1.5 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 5024  bytes 2963465 (2.9 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[justinbod (Justin Bodnar)]

It’s kind of hard to imagine your network without a diagram. Have you tried messing with VirtualBox networking mode? My guess is that’s the issue, since that’s where the buck stops.

[smartaim]

Thanks for your message. Let me give some more information about that.

I have my VM for adapter 1 set to “Bridged Adapter” with adapter type “PCnet-FAST III (Am79C973)” and promiscuous mode set to “Allow All”, as adviced when using the Macvlan driver. Adapter 2 is using “NAT Network” and also has the same setting for promiscuous mode (just to be sure).

Any Docker container running in default bridge mode can be accessed from anywhere in the network on the ports I have exposed and it can also access the internet. So this really seems to be a problem in conjunction with the Macvlan driver.

[mark1n]

Have you solved the problem ? I’m trying to solve the same problem, tried out many configurations but with no luck.

[mark1n]

If you are still interested in it, here may be the key for solving your problem:

Macvlan network and host to container connectity

Older versions of the docker documentation for mavlan networks included following note: When using macvlan, you cannot ping or communicate with the default namespace IP address. For example, if you create a container and try to ping the Docker host’s eth0, it will not work. That traffic is explicitly filtered by the kernel modules themselves to offer additional provider isolation and security. A macvlan subinterface can be added to the Docker host, to allow traffic between the Docker host and…

In my setup the docker host runs in a VM and I had to allow “promiscuous mode” on the vSwitch, otherwise the docker host net interface is not allowed to have more than one MAC address.

[aravinthchellathurai]

This helped. Thanks.

[lazygeekguy]

Thanks, this helped me !