I am struggeling with my setup regarding network connectivity:
Client Computer:
IP Address: 192.168.0.44/24
Gateway: 192.168.0.1
Docker Host (Raspberry)
IP Address: 192.168.10.21 on interface eth0
Gateway: 192.168.0.1
Container Running on Docker Host
Container IP: 192.168.20.50/24
Gateway: 192.168.20.1
I am using a macvlan interface for the docker container.
I created a macvlan sub interface “office-lan” on the Docker Host with the ip addresses assigned: connection add type macvlan ifname office-lan dev eth0 mode bridge
192.168.0.20
192.168.20.1
192.168.20.10
On my router 192.168.0.1 I do have a static route:
192.168.20.0/24 to 192.168.0.20
From my client pc I can successfully ping all the three ip addresses of the office-lan interface
From the docker host I can ping the container ip 192.168.20.50
I cannot ping the docker container from my client
I want to be able to ping the docker container from my client.
I guess I am missing a static route on the docker host, but I do not get which one.
My default gateway for the macvlan sub interface “office-lan” on the docker host is 192.168.0.1.
Promiscous mode for the interfaces eth0 & office-lan are enabled.
I can tracert from my client pc to 192.168.20.1 so I am able to achieve the macvlan sub interface. Somehow I am missing the last step for making the docker container ip reachable.
The router must be within the same subnet, and must have routes to the target subnets.
So the router on 192.168.0.1 must have a route for 192.168.20.0/24. The Macvlan network already uses a router on192.168.20,1 that is within the same subnet 192.168.20.0/24 - this router must be external to your docker host, and it must have a route back to 192.168.0.0/24.
This can only work, if you added an additional macvlan child interface to the docker host as well (often called macvlan-shim), as the macvlan parent interface is permitted (=it’s a kernel security restriction) to communicate with any macvlan child interfaces directly and vice versa. Though, macvlan child interfaces can communicate with other child interfaces, e.g. the macvlan-shim and a macvlan container, or macvlan containers with each other.
Other clients should be able to reach macvlan child interfaces, as long as the route to the macvlan network’s subnet is configured, and the macvlan network’s router has the route back to the clients’ subnet.
Personally, I am not a fan of macvlan networks, and there are only a few use cases why I would use them:
process inside the container network traffic must come from a specific source ip
process inside the container acts on broadcast or multicast messages, or sends them to the network
One last step is missing on my side and I do not get what I am missing.
My macvlan-shim is working like a charm - I can reach the container ip (192.168.20.50) from my docker host. I can ping my client pc from the docker host as well. And I can ping the subnet router ip 192.168.20.1 from my client pc.
I cannot ping the docker container ip from my client pc 192.168.0.44.
System macvlan interface properties
3: office-lan@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 2a:12:26:2b:d3:d1 brd ff:ff:ff:ff:ff:ff
inet 192.168.20.1/24 brd 192.168.20.255 scope global noprefixroute office-lan
valid_lft forever preferred_lft forever
inet 192.168.0.20/24 brd 192.168.0.255 scope global noprefixroute office-lan
valid_lft forever preferred_lft forever
Docker macvlan network details:
IPv4 Address: 192.168.20.0/24 (Docker Container IP is 192.168.20.50)
Gateway: 192.168.20.1
Routes on Docker Host for the macvlan interface:
default via 192.168.0.1 dev office-lan proto dhcp src 192.168.0.56 metric 410
192.168.0.0/24 via 192.168.0.1 dev office-lan
192.168.0.0/24 dev office-lan proto kernel scope link src 192.168.0.20 metric 410
192.168.0.0/24 dev office-lan proto kernel scope link src 192.168.0.56 metric 410
192.168.20.0/24 via 192.168.20.1 dev office-lan
192.168.20.0/24 dev office-lan proto kernel scope link src 192.168.20.1 metric 410
192.168.20.0/24 via 192.168.0.1 dev office-lan proto static metric 410
192.168.20.20 via 192.168.20.1 dev office-lan proto static metric 410
192.168.20.50 via 192.168.20.1 dev office-lan proto static metric 410
Ping from Docker Host to Client PC 192.168.0.44 is fine
ping -I office-lan 192.168.0.44
PING 192.168.0.44 (192.168.0.44) from 192.168.0.20 office-lan: 56(84) bytes of data.
From 192.168.0.1 icmp_seq=1 Redirect Host(New nexthop: 192.168.0.44)
64 bytes from 192.168.0.44: icmp_seq=1 ttl=128 time=2.31 ms
64 bytes from 192.168.0.44: icmp_seq=2 ttl=128 time=2.39 ms
Ping from Client PC to docker container:
,
On the Client Network I got the timeout…
PS C:\Users\KaiKo> ping 192.168.20.50
Ping wird ausgeführt für 192.168.20.50 mit 32 Bytes Daten:
Zeitüberschreitung der Anforderung.
Do you have an any about a route missing? Promiscuous mode is enabled for interface office-lan.
Thank you for sharing. Since your macvlan network does not specify a macvlan_mode, it uses the default bridge mode.
The gateway for your macvlan network is a macvlan child interface on the host. It also has a macvlan child interface in the 192.168.0.0/24 network.
Your routes do not reflect a router for 192.168.10.0/24 to 192.168.0.1. But there is the ip 192.168.0.56 which you didn’t mention before. Are you sure your eth0 desn’t use this ip instead of 192.168.10.21? Wouldn’t it create a network loop if two interfaces use an ip from the same subnet?
Can you test on the docker host, what results you get back for these commands?
ip route get 192.168.0.44
ip route get 192.168.20.50
I believe your choice in using the macvlan shim ip as gateway is part of the problem. Since all my setups used a gateway ip that was a different device than the docker host, things worked right away. I am still not sure if using a macvlan child interface on the docker host as gateway for the macvlan network is something that can work - never did it that way.
There is no routing involved here. Its device to device traffic in the same subnet.
Routing is involved, as you ping cross subnets. Have you tried using traceroute, tracert or however the command is called on your os instead of ping, it should provide more insights.
For ip route get 192.168.0.44 I got this: 192.168.0.44 via 192.168.0.1 dev eth0 src 192.168.10.21 uid 0
For ip route get 192.168.20.50 192.168.20.50 dev office-lan src 192.168.0.20 uid 0
Guess that looks fine.
What I see is I cannot reach my Router from within the macvlan subnet. I tried to reach from the eth0 interface and the docker host ip 192.168.10.21
ping -I 192.168.10.21 192.168.0.1
PING 192.168.0.1 (192.168.0.1) from 192.168.10.21 : 56(84) bytes of data.
64 bytes from 192.168.0.1: icmp_seq=1 ttl=63 time=1.58 ms
From 192.168.20.1 from the shim interface it’s not possible
ping -I 192.168.20.1 192.168.0.1
PING 192.168.0.1 (192.168.0.1) from 192.168.20.1 : 56(84) bytes of data.
... Timeout
Is it possible to route/map the shim gateway 192.168.20.1 to the default gateway 192.168.0.1 on eth0?
My target setup is to have one subnet for core components 192.168.10.0/24 and one for office related components hosted in docker containers 192.168.20.0/24. I want to host some office related software in docker containers and therefore I need to reach the containers directly from the client network 192.168.0.0/24.
That is why my docker host has some additional ip adresses (ip address 192.168.10.21/24 for the docker host device itsenf & 192.168.10.30/24 for the Portainer instance.
Interesting is, that I am able to ping my docker host physical ip 192.168.10.21 from within the docker container with ip 192.168.20.50.
So what works is:
Docker Host → Router (192.168.0.1)
Docker Container → Docker Host (192.168.10.21)
Docker Container → Other Docker Container (192.168.10.30)
Not working
Docker Container → Docker Host → Router
My bad, I wanted to write deny, but wrote permitted. Macvlan child and parent interfaces are not allowed to communicate with each other (and the other way around)
It is indeed interesting. This shouldn’t work. From what I know you would need to access it through the macvlan child interface 192.168.20.1. Have you checked which route it took to ping it?
But then again: the routes you shared show nothing about 192.168.10.0/24.
The answer must be hidden somewhere in settings you didn’t share. I can’t help you with your setup - I do know it works like a charm if the gateway is on an external router.