Share and learn in the Docker community.
I am trying to setup an overlay network.
My network gateway is 192.168.10.1.
I have some other servers on this network and I am worried that if I provide a cidr of 192.168.10.1/24, then IP’s that are already in use will be assigned.
Is it possible to provide a cidr of 192.168.10.1/24 but then give a list of IP’s that are not to be used?
Thanks
You simply exclude already occupied ip adresses using --aux-address when creating the ingress network (after deleting the existing one → there can only be one!)
docker network create --ingress --aux-address="host1=192.168.10.x" --aux-address="host2==192.168.10.y" --aux-address="host3=192.168.10.z" ....
Note: I am afraid this is not the right approach to “bridge” docker containers into a local lan. If this is what you end goal ist, then you might want to use the forum search with the term MACVLAN.
@meyay Wonderful, thanks for your help and quick response.
@meyay Hi again, not sure if I am doing something wrong. I have issued the following command to create a new ingress network
docker network create --driver overlay --ingress --subnet=192.168.10.1/24 --gateway=192.168.10.1 --opt com.docker.network.driver.mtu=1200 --aux-address="switch=192.168.10.2" --aux-address="switch2=192.168.10.3" --aux-address="cloud1=192.168.10.10" --aux-address="cloud2=192.168.10.11" --aux-address="cloud3=192.168.10.12" --aux-address="cloud4=192.168.10.13" --aux-address="host25=192.168.10.14" --aux-address="host26=192.168.10.15" --aux-address="host27=192.168.10.16" --aux-address="host28=192.168.10.17" --aux-address="host29=192.168.10.18" --aux-address="host30=192.168.10.19" --aux-address="host31=192.168.10.20" --aux-address="host32=192.168.10.21" --aux-address="host33=192.168.10.22" --aux-address="host34=192.168.10.23" --aux-address="host35=192.168.10.24" --aux-address="mail=192.168.10.25" --aux-address="gitlab=192.168.10.26" --aux-address="host1=192.168.10.27" --aux-address="host2=192.168.10.28" --aux-address="host3=192.168.10.29" --aux-address="host4=192.168.10.30" --aux-address="host5=192.168.10.31" --aux-address="host6=192.168.10.32" --aux-address="host7=192.168.10.33" --aux-address="host8=192.168.10.34" --aux-address="host9=192.168.10.35" --aux-address="host10=192.168.10.36" --aux-address="host11=192.168.10.37" --aux-address="host12=192.168.10.38" --aux-address="host13=192.168.10.39" --aux-address="host14=192.168.10.40" --aux-address="host15=192.168.10.41" --aux-address="host16=192.168.10.42" --aux-address="host17=192.168.10.43" --aux-address="host18=192.168.10.44" --aux-address="host19=192.168.10.45" --aux-address="host20=192.168.10.46" --aux-address="host21=192.168.10.47" --aux-address="host22=192.168.10.48" --aux-address="host23=192.168.10.49" --aux-address="host24=192.168.10.50" ingress
docker network ls
NETWORK ID NAME DRIVER SCOPE
706c92c0d2c4 bridge bridge local
dd1ccaef4306 docker_gwbridge bridge local
75e56fab728e host host local
jcaj51p5k695 ingress overlay swarm
cb004525f3ad none null local
ymgzvg78qx6m portainer_agent_network overlay swarm
and I have created a simple nginx service with port 80 exposed, which has been issued address 192.168.10.7
When I try to access 192.168.10.7 I don’t get a response and I am also unable to ping the address.
My setup is ProxMox running on a home lab server, which has a host address of 192.168.10.12. A Ubuntu 18.04 VM running on IP 192.168.10.50, which is where my single node docker swarm is located.
The Nginx container is running, I can attach to it and curl nginx on http://localhost
The clue I can find to something being wrong, is that in the nginx container network description it gives the following information
Network IP Address Gateway MAC Address Actions
ingress 192.168.10.7 - 02:42:c0:a8:0a:07
As you can see according to the above, a gateway has not been provided to the container by ingress.
Any ideas what might be wrong and how I might even start debugging the issue.
Just to be sure: your use-case is that you want the ingress network to share the subnet and ip-range of one of your local lans, but you want exclude ips that are already in use.
If this is the case, then the “Note:” of my first response applies. If I am mistaken about your use case, it would be helpful to understand the objective.
I am using the docker swarm to host apps while I am developing them.
I have 4 ProxMox servers on IP address 192.168.10.10-13 and a few other VM’s such as databases and mail servers running. The other excluded IP’s are just IP’s for future use. I was originally considering setting up another VLAN 192.168.11.1/24 for my ingress, but I am not an expert with networking and was struggling to get that to work, so I have gone with the simple approach of putting them all on the same subnet.
It’s only for internal home development, so I am happy with any risks.
If it helps I am also running pfSense on a box with 4 network ports. My 4 ProxMox servers are connected via managed Netgear switch to one of the ports on the pfSense. My lan and wifi are on different subnets.
Thank you for confirming my suspicion. The ingress network is supposed to be private to docker. You could try if it is possible to create a macvlan network as ingress - I would be surprised if it works, but I actualy never tried it that way.
actually, I just noticed that the create network command has not worked as expected. Even with the --axu-addresses I have specified, ingress is still assigning those IP addresses to containers. I think I will take down the network and try again taking into account your note about MACVLAN
The network should have excluded those ip’s. Though I must admit I only used it with macvlan, as it’s the only network type where it makes sense to exclude ips.
I am not sure how valuable macvlan for the ingress network actualy is, you would get random ip addresses without name resolution. If you create a normal network and attach the services to such a a network, the service tasks (=scheduled instances of the container) would get random ip’s as well. The option to assign fixed ipv4 addresses does only exist for docker docker-compose and never was implemented for docker swarm.
I am actualy curious if it’s possible to have a macvlan ingress.
Personaly I don’t use macvlan anymore. I made my research years ago and couldn’t accept how it worked with swarm services (=not possible to assign fixed ipv4 addresses) and then solved it the “docker way”.
I have three PVE machines in my homelab and use opnsense to route between my private and homelab networks.
I do use a globaly deployed reverse proxy (Traefik 1.7 with LE stored in consul) with subdomain based reverse proxy rules to forward traffic to the target services. I manage the subdomains in unbound running on the opnsense vm’s. Additionaly I have a failover-ip on the cluster nodes to make sure everything that does not enter the cluster thru the reverse proxy still can be accessed by a static ip.
I resolve the A-record of the *. entry to my WAN-Routers ipv4 and resolve the AAAA record of the *. entry to the primary opnsense vm, which runs ha-proxy to forward the ipv6 traffic to the reverse proxy in the cluster. This way, I can access most of my containers from the internet without having to register subdomains for each and every service. I just set the subdomain in the reverse proxy and a new service is reachable right away.
@meyay Thank you for your help, everything is working now using macvlan.
For anyone else looking to know how to setup macvlan, here is a short and really easy to follow youtube video.
@meyay My plan is to use docker-compose files pulled from a gitlab server to setup the containers. The post below seems to say you can define a fixed IP address in the docker-compose file, which I will try and see what millage I get.
Yep, fixed ips with docker-compose deployments work well.
Update: I forget to add some words in my mail response 