How do I attach a macvlan network and assign a static Ip address in compose file?

Open Source Projects Compose
1

My host IP is 192.168.88.3. I have created a macvlan network named “macvlan_network”. My docker version is:

Client: Docker Engine - Community
Version: 20.10.5
API version: 1.41
Go version: go1.13.15
Git commit: 55c4c88
Built: Tue Mar 2 20:18:46 2021
OS/Arch: linux/arm
Context: default
Experimental: true

Macvlan works as expected and I was successful in assigning specific IP to each container. Though I was able to deploy containers using the run command but having some issues using docker-compose.yml. For example, I like to run my pihole container in 192.168.88.5 on macvlan_network. So, for that instance run command works perfectly fine.

`docker run --name=pihole  --net=macvlan_network --ip=192.168.88.5 -e ServerIP=192.168.88.5 -e WEBPASSWORD=pass -e TZ=timezone -e TEMPERATUREUNIT=c -v /etc-pihole/:/etc/pihole/ -v /etc-dnsmasq.d/:/etc/dnsmasq.d/ --cap-add=NET_ADMIN --restart=unless-stopped pihole/pihole`

But, I am having errors when I use docker compose.

version: "3"

# More info at https://github.com/pi-hole/docker-pi-hole/ and https://docs.pi-hole.net/
services:
  pihole:
    container_name: pihole
    image: pihole/pihole:latest
    ports:
      - "53:53/tcp"
      - "53:53/udp"
      - "67:67/udp"
      - "80:80/tcp"
      - "443:443/tcp"
    environment:
      ServerIP: '192.168.88.5'
      TZ: 'timezone'
      WEBPASSWORD: 'pass'
      PIHOLE_DNS_: '127.0.0.1'
      TEMPERATUREUNIT: 'C'
    # Volumes store your data between container upgrades
    volumes:
      - './etc-pihole/:/etc/pihole/'
      - './etc-dnsmasq.d/:/etc/dnsmasq.d/'
    # Recommended but not required (DHCP needs NET_ADMIN)
    #   https://github.com/pi-hole/docker-pi-hole#note-on-capabilities
    cap_add:
      - NET_ADMIN
    networks:
      macvlan_network:
          driver: macvlan
          driver_opts:
              parent: eth0
          ipam:
            config:
                - 
                  subnet: 192.168.88.0/24
                  gateway: 192.168.88.1
       
    restart: unless-stopped

How, can I add the pihole container on my macvlan_network with the specific IP. Any response will be appreciated.

2

I believe would be something like this:

services:
  portainer:
    image: portainer/portainer
    container_name: portainer
    restart: always
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /path/to/data:/data 
    ports:
      - 8000:8000
      - 9000:9000
    networks:
      vlan:
        ipv4_address: 192.168.0.60

networks:
  vlan:
    driver: macvlan
    driver_opts:
      parent: eth0
    ipam:
      config:
        - subnet: "192.168.0.0/24"
          ip_range: "192.168.0.64/26"
          gateway: "192.168.0.1"
3 
version: "3"

services:
  pihole:
    container_name: pihole
    image: pihole/pihole:latest
    networks:
      macvlan_network:
        ipv4_address: xx.xx.xxx
    restart: unless-stopped
    environment:
      ServerIP: 
      TZ: 
      WEBPASSWORD: 
      TEMPERATUREUNIT: 'C'
      IPv6: 'false'
    volumes:
      - etc-pihole:/etc/pihole
      - etc-dnsmasq.d:/etc/dnsmasq.d
    cap_add:
      - NET_ADMIN

volumes:
  etc-pihole:
    external: true
  etc-dnsmasq.d:
    external: true
networks:
  macvlan_network:
    external:
      name: macvlan_network

This worked for me.

4

You’ve skipped a step there as you’re referencing an already existing macvlan network but haven’t explained how you created that.

I am using the exact same configuration that you have specified, and my macvlan network is already created, but I find that the settings are being ignored and a random subnet is being created each time. I’m not sure what to do.

I read that static IPs (using ipv4_network) were not supported in v3 but you appear to have it working.

I actually have two issues; one is that I cannot deploy my container on my macvlan subnet, but I also have an internal network set up in docker for communication between the pihole container and a cloudflared container, which works fine, but I also cannot assign static IPs to the containers, they get random ones each time.

5

As long as the compose file is deployed using docker-compose, it should work regardless of the compose schema version.

Though, last time I tried it was not working for swarm stack deployments - the odd thing is the compose v3 documentation does not mention any limitation. I am not sure if it’s just a docu bug, or the docu is complete and the feature is available for swarm stack deployments now as well. According this github epic, it was never implemented for swarm services.

6

Thanks. I also found and read that thread.

I am indeed trying to deploy a stack. Perhaps it’s possible to deploy a container on a node with a static IP but not a stack with replication set. Unfortunately that’s the entire reason I built the swarm :wink:

For the time being I have deployed pihole the old fashioned way by exposing ports on the host. All DNS traffic appears to come from the host, but I can live with that for the time being. The frustrating thing is; when ever the cloudflared VM is deployed it gets a new IP and I have to update pihole to restore DNS in my environment.

I tried using smaller subnets, such as a /30 so that there were only 2 IPs available, however it appears that docker won’t deploy the containers to anything smaller than a /29.

Every time I think I’ve found a suitable workaround, there’s something else.

7

I assume you are refering to challanges with portfowarding, because the macvlan ip changes?
You can tackle the issue by not using macvlan, but introduce keepalivd on your nodes that share a failover-ip. The failover-ip will be assigned to one of the healthy nodes - if it dies, the next keepalivd node will claim the ip…This way you can have a fixed portforwarding target ip, but of course still depend on docker’s ingress routing mesh to forward the traffic to the target container.

Off-topic: pihole has the nasty habit to store its statistics database in the config folder. If multiple replicas share the same config folder, they also share the sqllite statistics database, which at one point will be corrupted as all replicas mess with it at the same time.

1 Like
8

Thanks.
I have considered this option. Ideally I don’t want to expose the ports on the host due to the logs issue mentioned and, for other services, I may wish to use the same port on different containers, which is why I was hoping to get macvlan working.

Thanks for the heads up about the statistics. I only run 1 replica at a time so I’m hoping this won’t be an issue but I will bear it in mind.

9

I’ve been chasing this issue for months now. Various solution are claimed to work.
Background: Docker on Synology with DSM7.1.

bash-4.4# docker version
Client:
 Version:           20.10.3
 API version:       1.41
 Go version:        go1.17.1
 Git commit:        55f0773
 Built:             Wed Feb  9 04:04:10 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.3
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.17.1
  Git commit:       b487c8f
  Built:            Wed Feb  9 04:04:31 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.4.3
  GitCommit:        3fa00912415f3e9c6f82dd72119179d599efd13b
 runc:
  Version:          v1.0.0-rc93
  GitCommit:        31cc25f16f5eba4d0f53e35374532873744f4b31
 docker-init:
  Version:          0.19.0
  GitCommit:        ed96d00
bash-4.4# docker-compose version
docker-compose version 1.28.5, build 24fb474e
docker-py version: 4.4.4
CPython version: 3.7.10
OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019

Running portainer, version 2.11.
I have a macvlan with the top a.b.c.128/25 addresses available. I have two containers I run: one with sonos-samba, and one (migrating to) gitea. If I reboot the NAS, the IP address assignment depends on the startup order of the containers, resulting in the wrong IP address being assigned. sonos-samba requires a fixed IP on my LAN because the sonos APP needs it to find the music library. gitea needs a fixed IP address because all my git remote paths have the embedded IP address. Also, the address is embedded in the config file.

Much googling results in many folks having similar requirements. Many “claim” to have a solution. One was that supposedly in Portainer you can “redeploy/edit” the contained and go down to the network area and specify an address. Slam-dunk, that issue was closed as “solved” so I tried that. The IP address assignment did not stick anywhere. Did not take effect. Subsequent “redeploy/edit” shows it does not save the edited IP address value.

Many supposed solutions (as in this thread) suggest that docker-compose can be used, so I have been trying that lately. So far, no suck luck. I have explicitly moved the gitea address from .129 (which is after the .128 the sonos-samba is on) to .131 to test this. Despite my docker-compose attempts, it ignores the IP address setting.

version: "3.4"

services:
  server:
    image: gitea/gitea:latest
    container_name: gitea
    environment:
      - USER_UID=1000
      - USER_GID=1000
      - TZ=AEST
    restart: always
    networks:
      macvlan3:
         ipv4_address: 192.168.3.131
    volumes:
      - '/volume1/docker/gitea/data:/data'
      - '/etc/localtime:/etc/localtime:ro'
    ports:
      - "3000:3000"
      - "22:22"
    cap_add:
      - NET_ADMIN
    extra_hosts:
      - "gitea.zepherin.com:192.168.3.131"

networks:
  macvlan3:
    external:
      name: macvlan3

This corresponds to the “this works for me” comment above.
This does not work for me. I just started my gitea container. The sonos-samba one was already running.

bash-4.4# docker network inspect sglt
[
    {
        "Name": "macvlan3",
        "Id": "sgltu774wevay1kv15m4p8a7f",
        "Created": "2022-04-25T16:25:42.409407246+10:00",
        "Scope": "swarm",
        "Driver": "macvlan",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "192.168.3.0/24",
                    "IPRange": "192.168.3.128/25",
                    "Gateway": "192.168.3.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": true,
        "Ingress": false,
        "ConfigFrom": {
            "Network": "macvlan3.config"
        },
        "ConfigOnly": false,
        "Containers": {
            "b0a59d372aa4c981401c1c9d4bd0603b5774e7100603e28ac7c473b8adeb5040": {
                "Name": "gitea",
                "EndpointID": "7f3faad12292904be4cebbd0bea904fba7121a2e3f0d1900082206865072135b",
                "MacAddress": "02:42:c0:a8:03:81",
                "IPv4Address": "192.168.3.129/24",
                "IPv6Address": ""
            },
            "da8d8b8f8e8664c80aaea11bb4de47c7608319e92f42cbd2daf17ec9657899f7": {
                "Name": "00-sonos-samba",
                "EndpointID": "6cc67f500706d53869d0fa278f4613d41b6996df330fcc7b969b65b797eec847",
                "MacAddress": "02:42:c0:a8:03:80",
                "IPv4Address": "192.168.3.128/24",
                "IPv6Address": ""
            }
        },
        "Options": {
            "parent": "eth0"
        },
        "Labels": {},
        "Peers": [
            {
                "Name": "cd5419e93e80",
                "IP": "192.168.3.33"
            }
        ]
    }
]

Here are the network settings from docker inspect gitea as currently running:

        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "409dbfab01f4c385d0e4744e104c67fa95013349f4c390b23ca2241d2be029e4",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/409dbfab01f4",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "macvlan3": {
                    "IPAMConfig": {},
                    "Links": null,
                    "Aliases": [
                        "b0a59d372aa4",
                        "server"
                    ],
                    "NetworkID": "sgltu774wevay1kv15m4p8a7f",
                    "EndpointID": "7f3faad12292904be4cebbd0bea904fba7121a2e3f0d1900082206865072135b",
                    "Gateway": "192.168.3.1",
                    "IPAddress": "192.168.3.129",
                    "IPPrefixLen": 24,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:c0:a8:03:81",
                    "DriverOpts": null
                }
            }
        }

I have put an explicit ifconfig in the gitea container which seems to do the trick for now, but I would prefer to find a solution which actually works for the next service i might want to deploy on the macvlan.

10

May I suggest to create an ubuntu vm in Synology VMM, and see for yourself that “what the other claim” actualy works in vanila docker (=unmodified from the Docker repos). Also make sure the ip-range you specified is outside your networks dhcp range.

I would have asked if you accidently enabled the swarm-mode and experience this with swarm services, but then I remembered that Synology keeps the swarm mode deliberately broken on their Docker package since 17.05. What you experience is normal for a swarm service deployment, but is not for plain containers.

11

Had no idea there was a synology VMM. I will give that a try.
As far as the macvlan, it is correctly defined – no overlap in the DHCP ranges.

12

I keep looking at that “swarm mode” and some resultant warnings and wonder that myself. I seem to have added something via portainer. Time to see if I can undo/turn-off.

13

Docker itself supports both type of deployments if the swarm mode is active on the node. The way you deploy (docker-compose vs. docker stack deploy) defines the type of deployment. From what I remember Portainer either deploys swarm stacks or compose stacks, depening on wether the swarm mode is enabled on the node or not.

You can check it from the command line: sudo docker info --format '{{.Swarm.LocalNodeState}}'.

14

Ok. I have to admit that running docker on the ubuntu system with my config and with the macvlan defined did work: I set the IP to 192.168.3.135 for this test, and it definitely allocated precisely that. Did not try to deploy other containers in that env on the same vlan, though — it was well past bed time.

So, how come it fails on the Synology+Docker environment? Running gitea under syn+vmm+ubuntu+docker is not feasible. I’d really like to know where to look to attempt to fix this.

15

You might want to join the unofficial synoforum.com and ask there. I would have recommend the official Synology forum if it was any good… From what I remember plenty forum users successfully use the setup you try to use.

16

I ran into the same issue while trying to get a pihole up and running on my Synology DS923+.
I hava a macvlan, named “macvlan” already in place with this specs:

IPV4 Subnet - 192.168.178.0/24
IPV4 Gateway - 192.168.178.1
IPV4 IP Range - 192.168.178.225/27

… so it has an addressable range of:

192.168.178.225 - 192.168.178.254

I wanted pihole to assign 192.168.178.225 but my pihole container always got a wrong ip address assigned by docker.

The only thing that worked for me was to have both, the ip adress defined in the “services” section and my existing macvlan listed below under the “networks” section in my compose .yml file.

services:
  pihole:
    container_name: pihole-macvlan
    image: pihole/pihole:latest
    ports:
      - "53:53/tcp"
...
    networks:
      macvlan:
        ipv4_address: 192.168.178.225
...
networks:
  macvlan:
    name: 'macvlan'
    external: 'true'
17

I’m confused, the Top level definition all works except the ipv4_address for me thus far. However I’m confused why I keep seeing the
ports:
given it’s a native connection to the host interface there’s no reason to pass ports, and from my experience causes errors in the containers, on linux do journalctl -u docker.service to have a peak. If I’ve missed something though I’d love to know.

I’m likewise unable to find any working configuration, I’m using this in ubuntu, not a swarm but still just simply trying to apply a static ip to a custom stack. I’ve tried all of the above and would love a reference to the code and source if anyone ever finds this.