Docker service gaining access to main / host network

Docker Engine Swarm
1

Hi,

I have a docker swarm and trying to setup a docker service that, instead of being configured like most of my services with the overlay driver, I would like this particular service to be able to scan my main network. The service in question is NetAlertX. So far I have tried everything suggested in this documentation Docker Compose - NetAlertX Docs but it does not work in the context of a docker swarm.

I was wondering if there is a way to assign an IP address and mac address to a docker service that is part of a swarm and communicates directly with the firewall on the network? All that through a docker compose file? This is the config I have so far but I struggle to find what to do with the network part that will actually work for a swarm.

# NetAlertX
  netalertx:
    deploy:
      placement:
        constraints:
          - node.hostname==PTR1-PrimeSvr
      replicas: 1
      restart_policy:
        condition: any
        max_attempts: 100
        delay: 60s
        window: 60s
    image: jokobsk/netalertx:latest
    read_only: true
    cap_drop:
      - ALL
    cap_add:
      - NET_ADMIN
      - NET_RAW
      - NET_BIND_SERVICE
    environment:
      - PUID=${PUID:-500}
      - PGID=${PGID:-500}
      - TZ=${TIMEZONE:-America/Toronto}
      - LISTEN_ADDR=${LISTEN_ADDR:-0.0.0.0}  
      - GRAPHQL_PORT=${GRAPHQL_PORT:-20212}
    ports:
      - 2021:20211
    volumes:
      - netalertx-data:/data:rw
      - netalertx-tmp:/tmp:rw
    networks:
      - gateway-net
2

Since swarm services were built with replicas in mind, it was deemed irrelevant, see: [epic] add more options to `service create` / `service update` · Issue #25303 · moby/moby · GitHub. It didn’t change since then.

With endpoint mode vip (=default), the service’s service discovery name will resolve to the service’s vip, which load balances the traffic amongst the swarm tasks (aka replicas). The swarm tasks will have an arbitrary ip from the networks subnet and an arbitrary mac address. The same is true if the endpoint mode dnsrr is used, except there is no service vip, and the service name resolves to a multi-value dns result showing the ip addresses of each service task.

Sadly, this is not going to work with a swarm service. Though, you could run a plain container and attach it to an attachable overlay network, which according to the docs is always the case:

Overlay networks are always created as attachable . You can optionally set the attachable property to false .

source: https://docs.docker.com/compose/how-tos/networking/#multi-host-networking

3

I’m not sure if I understand everything here but from the sound of it, it looks like because I’m using a swarm there is no way to set a service to connect to the main LAN even if I set the service to have a replica of 1 and on a specific node?

4

Correct. network_mode: host does not work with Swarm services.

5

This is slightly ambiguous … :

  • “I’m using a swarm” as in “swarm cluster” or “swarm service”?
    • a swarm cluster can still run normal containers on the nodes, which allow configuring the ip and mac address. Normal containers can be attached to overlay networks if necessary (in case the container needs to communicate with swarm services in the overlay network).
  • What does “connect to main LAN” mean? That the service tasks is able to communicate with the LAN, or that it gets an ip from your LAN?
6

Hi there @meyay

  1. I have a swarm cluster. Most of my services are setup to start on a specific machine with no replicas. I set it out this way because it was easier to manage and deploy services to specific servers from the leader. How can I setup a normal container then on a server that isn’t the leader, as I tried that and every time it needs to be handled by the leader?

  2. What I mean by that is.. Docker uses its own VLAN and I can access the services through a defined PORT. All of these instances live inside a VLAN that is not directly accessible from my “main” network. For more services that is fine. However, NETALERTX, which is the service I need to deploy, needs to scan my LAN to detect servers and devices connected to the network that may be down. So, I need to make sure that in this case, that service has access to my LAN and not just the docker VLAN.

7

Can you share more info about how you tried to create the container? A swarm node is a machine where you can run docker containers. The ability to run swarm services comes with swarm, but you should be able to use docker compose or docker run commands to create a container.

9

So I cannot give you an example of me starting a container but I have been using docker-compose to stack deploys. For example here is the NetAlertX service I tried to setup :

# NetAlertX
  netalertx:
    deploy:
      placement:
        constraints:
          - node.hostname==ServerWhereIWantToDeploy
      replicas: 1
      restart_policy:
        condition: any
        max_attempts: 100
        delay: 60s
        window: 60s
    image: jokobsk/netalertx:latest
    read_only: true
    cap_drop:
      - ALL
    cap_add:
      - NET_ADMIN
      - NET_RAW
      - NET_BIND_SERVICE
    environment:
      - PUID=${PUID:-500}
      - PGID=${PGID:-500}
      - TZ=${TIMEZONE:-America/Toronto}
      - LISTEN_ADDR=${LISTEN_ADDR:-0.0.0.0}  
      - GRAPHQL_PORT=${GRAPHQL_PORT:-20212}
    ports:
      - 2021:20211
    volumes:
      - netalertx-data:/data:rw
      - netalertx-tmp:/tmp:rw
    networks:
      - gateway-net

Keep in mind this is still work in progress and just meant to show as an example.

To deploy that, I would execute this :

docker stack deploy --with-registry-auth --resolve-image=always -d -c compose.yaml sysmon

All this is executed from the leader node.

Now what you are proposing I think is a bit different. I would need to go to the actual node and create the service locally. I have not tried that but, that would mean, I would lose the ability to manage that “service” from the leader node and would need to manage it directly on the node itself.

10

Correct, this command deploys a swarm stack, which deploys swarm services. It is not used to deploy plain containers.

On any of your swarm nodes, regardless whether manager or worker, you can still deploy and run plain containers using the docker compose command. Unlike the deprecated docker-compose command, the cli plugin will ignore swarm specific configuration properties of your compose file.

Note: the docker compose cli plugin is installed with recent docker versions. The old docker-compose command was deprecated almost 2,5 yeas ago.

11

It’s just too bad that when a service is set to replicas = 1 and a specific node is selected that a host network can’t be selected because it almost is the same as deploying something as a container in a manner of speaking and allows to manage from one specific location, the leader.

I will give your solution a try and see how it goes

12

@meyay

That worked, thanks for your help.

Just a question. After I start the container with

docker compose start

and setting the restart condition to “unless-stopped”, am I correct in assuming that if the server shuts down and restarts that the container will start without me having to do so manually?