How to allow other network devices access to containers

General
,
1

Hi all,

I am fairly new to Docker and have been trying to setup a small network so that each container can have seperate IP addresses so that other “user” devices on the network can access the containers and it would allow multiple containers to use the same port. Below is an example of the network setup as it currently is.

network
network494×508 15.5 KB

I currently can access containers I don’t try and provide IP addresses to using the host machine’s IP and the specific port needed. What I can’t seem to get working is a way so that API container for example, is accessible on 192.168.1.151 and another container on 192.168.1.152. Below is the docker compose

volumes:
  prometheus-data: {}
  grafana-data: {}
  alertmanager-data: {}
  data: {}
services:
  bind9:
    image: bind9:latest
    container_name: bind9
    ports:
      - '53:53/udp'
      - '53:53/tcp'
    restart: unless-stopped
    volumes:
      - ./bind9/etc/bind/named.conf.local:/etc/bind/named.conf.local
      - ./bind9/etc/bind/named.conf.options:/etc/bind/named.conf.options
      - ./bind9/etc/bind/db.homelab.lan:/etc/bind/db.homelab.lan
      - ./bind9/etc/bind/db.192.168:/etc/bind/db.192.168
  cadvisor:
    image: gcr.io/cadvisor/cadvisor:latest
    container_name: cadvisor
    ports:
      - '8080:8080'
    restart: unless-stopped
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
      - /dev/disk/:/dev/disk:ro
  node_exporter:
    image: quay.io/prometheus/node-exporter:latest
    container_name: node_exporter
    command:
      - '--path.rootfs=/host'
    network_mode: host
    pid: host
    restart: unless-stopped
    volumes:
      - '/:/host:ro,rslave'
  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    ports:
      - '9090:9090'
    restart: unless-stopped
    volumes:
      - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus-data:/prometheus
    command:
      - '--web.enable-lifecycle'
      - '--config.file=/etc/prometheus/prometheus.yml'
  grafana:
    image: grafana/grafana:latest
    container_name: grafana
    ports:
      - '3000:3000'
    restart: unless-stopped
    volumes:
      - grafana-data:/var/lib/grafana
  alertmanager:
    image: quay.io/prometheus/alertmanager:latest
    container_name: alertmanager
    ports:
      - "9093:9093"
    restart: unless-stopped
    volumes:
      - ./alertmanager/alertmanager.yml:/etc/alertmanager/alertmanager.yml
      - alertmanager-data:/data
  api:
    image: phpserver:latest
    container_name: api
    ports:
      - '443:443'
      - '80:80'
    restart: unless-stopped
    networks:
      homelab:
        ipv4_address: 192.168.1.151
    volumes:
      - /var/www/html/vendor
      - ./php/api:/var/www/html
  portainer:
    image: portainer/portainer-ce:latest
    ports:
      - 9443:9443
    volumes:
      - data:/data
      - /var/run/docker.sock:/var/run/docker.sock
    restart: unless-stopped
networks:
  homelab:
    driver: macvlan
    driver_opts:
      parent: enp1s0f0
    ipam:
      driver: default
      config:
        - subnet: 192.168.1.144/28
          gateway: 192.168.1.150

Any support, pointers or help would be highly appreciated.

2

The compose file is wild :smiley:

Why would there be unrelated services in the same compose file?
Why would every service publish ports, even when it is not using a bridge network?

Are you aware of the macvlan constraint, that no direction communication between macvlan parent interfaces (enp1s0f0) and macvlan child interfaces (what the containers use) is possible.

In other words, containers not attached to the macvlan network can’t communicate with those that are only attached to the macvlan network. It works from every client in the network, and from every macvlan container to every other macvlan container. A container can be attached to a bridge and macvlan network at the same time. A container using host networking won’t be able to communicate with containers attached to the macvlan network, either.

And why would the docker host be the router for the macvlan network? This can not work because of the macvlan parent/child restriction. It must be a router in your network - which preferable is in the same subnet, but not in the same ip_range you use for the macvlan network.

Note: published ports are not used when a containr uses the host networking, or uses the macvlan network. It is only relevant when the bridge network is used.

1 Like
3

Hi,

The state of the file is solely down to following a couple of tutorials so there is room for improvement on it.

If I understood what you was saying, all the containers need to be added to the macvlan and the subnet and gateway details need to be changed to the router’s information?

4

Whichever container is supposed to communicate with other containers that are only attached to the macvlan network, needs to be attached to macvlan as well. Containers only attached to a bridge network, or use host networking, are affected by the macvlan parent/child restrictions.

I am not sure what “the subnet and gateway details need to be changed to the router’s information” is supposed to mean. But the gateway ip should point to the ip of the router in the subnet that is actually responsible to route between subnets.

5

This fixed my issue, changing the gateway IP and setting the network_mode: host on some of the other containers was exactly what I needed, thank you