It could be. Wasn’t it intentional? I don’t know how could this happen accidentally. Can you tell more about your environment? Is it Linux, Windows or Mac?
I experienced some issue with docker networks in the past. When I run many containers without restarting the daemon, after some time Docker run out of IP addresses in the default range (172.16.0.0/20) because it did not reuse the already deleted networks subnets. In that case Docker continues to use IP addresses from 192.168.0.0/16 . It shouldn’t be a problem since Docker can see the already present networks so it can choose an available IP range. But What if the network created by Docker was there before the network on the host appeared?
Once I installed Docker in WSL2 on Windows 10 and I noticed the WSL 2 machine got the IP address from the same range as Docker. For example WSL 2 got 172.17.0.0/24 so when Docker start to created new networks it started from 172.18.0.0/24. Then I rebooted my host and the WSL 2, which knew nothing about the network inside itself, got the IP range 172.18.0.0/24. Then I didn’t have internet in WSL until reboot. So the previously created Docker network could ruin the host’s network. I had to create a large subnet on my Windows 10 host to cover all the IP addresses in 172.16.0.0/20 so WSL 2 could not use any subnet from this range. This way I am good until I have so many containers Docker starts to create networks from 192.168.0.0/16.
The above story is one that can be related to your issue. Also when I last tried macvlan, it didn’t work, but the issue was probably me, not Docker.
Does the following help you help me?
As for the run command, can you point where I can find this? As my docker and container kind of run automatically on reboot and I am not sure where this Run command might be running.
So your problem is that the macvlan host is not able to ping macvlan child interfaces attached to your containers?
Well that’s to be expected, as the kernel has security messures in place that prevent macvlan host to macvlan client interface communication. Docker has no stakes regading the restriction.
Older versions of the docker documentation (arround 17.03) used to point out this circumstance, but it was removed from the docs since… which, in fact could have remained as N.B. note, as it is relevant for the user to understand the cause, but at the same time is not related to Docker at all…
The workaround is to add another macvlan child interface to your host and use it to interact with the other macvlan client interfaces. The macvlan child interfaces should be reachable by other devices in you lan as if they are standalone machines. The forum search should find a couple of hits about the workaround.
The docker network connect commands you shared, make clear why your containers are attached to the macvlan0 network and bridge network. If you would’ve started the container using --network= macvlan0 --ip= .... it would only have an interface in the macvlan0 network.
n.b.: a service on a macvlan client ip must be accessed by its container port, not by “its published port”. There is no port publishing on macvlan interfaces. The macvlan host CAN NOT access the macvlan client ip, without the workound I mentioned earlier.
I used the search term “macvlan” in the forum search and had this thread as first hit:
The section “Syntax to create sub interface:” describes how to add a macvlan client interface to your host. It shows an example of the commands needed to workaround the macvlan limitation I was refering to earlier.
Is 192.168.3.0/24 your local lan?!? If so, this is a call for trouble, as you did not define the --ip-range parameter when creating the macvlan, which should be a range within your subnet that is outside a dhcp servers reach. Your whole setup just works reliable so far, because you declare fix ip addresses to your containers.
Yes. 192.168.3.0/24 is my local LAN
The DHCP in my pfsense is set to range from 100-150
You mean that when I create macvlan0 network on docker I need to set a range dedicated to my containers?
My containers are 192.168.3.5, 192.168.3.6 and 192.168.3.7
If so what would --ip-range contain!?
Hey, thanks again for your help. really appreciated!
The missing --ip-range ristriction is merly an observation. It is not the reason for your macvlan host<->container problem. If you should ever decide to have dhcp on your server, your network dhcp AND the docker embedded macvlan dhcp server will fight over who is going to provide the ip-range to the interface.
Thus said, the fix for your original problem should be:
ip link add macvlan-shim link en01 type macvlan mode bridge
ip addr add 192.168.3.x/24 dev macvlan-shim
ip link set macvlan-shim up
Normaly people then add a route for the macvlan ip-range. Since you don’t have the ip-range set, I doubt it makes any senes in your case.
ip route add 192.168.3.x/y dev macvlan-shim
x = first ip of the cidr range
y = bits of the cidr range
You can use a subnet calulator to identify a free(!) ip range in your subnet for the number of ip addresses you need (outside your dhcp range). You might start with a 27 bit mask (allows 32 ips). A 26 bit mask allows twice the amount ip’s, a 28 bit subnet allows halfe the amount of ip’s.
Plese do NOT use this setting for your /24 bit mask!
Though, there is a cavit with the approach: the settings are ephemeral and need to be re-applied after every reboot. You probably want to add it to a script that is called after a boot. Since I don’t use macvlan myself, I can’t help you with that… I don’t belive in the necessity to use macvlan at all. Never required it during the last 7 years…
Because in Windows docker is run in a vm (regardless wether with wsl1 or wsl2) which has a different ip than the host. The docker bridge network is invisible to the windows host. If you connect to the wsl1 vm or wsl2 distribution (which runs in a vm shared by all wsl2 distributions) you will be able to ping the container from there, the same way like your colleague is able to ping the container on his ubuntu host.