Port Mapping with custom Host IP on Linux

[bernhardkrop]

Hi!

I have problems creating a container with a custom host IP (other than “localhost” or “127.0.0.1”; e.g. “192.168.59.123”) on Linux. If using “localhost”, all works well.

Last year, I had a MacBook and had the same problem, but found a solution (I also created a topic). This year, I got a new notebook from my company: an Ubuntu 24.04. Unfortunately, my solution from last year on Mac does not work.

I found a lot of topics (and documentation), which suggests using mcvlan as a network driver would be the solution. But it does not work (or I am doing something wrong).

Could somebody help me getting it to work, please?

Thank you in advance,
Bernhard

Linux: Ubuntu 24.04.2

Docker Desktop:

docker-compose.yml

# Note: Values have been anonymized.
name: my-project-name
networks:
    database:
services:
    mariadb:
        build: /my/build/path
        container_name: my-project-name-mariadb
        env_file: /my/env_file/path
        networks:
            database:
        ports:
            - host_ip: 192.168.59.123
              mode: host
              protocol: tcp
              published: 1234
              target: 1234
        volumes:
            - read_only: true
              source: db
              target: /my/volume/target/path
              type: volume
            - source: /my/bind/source/path
              target: /my/bind/target/path
              type: bind
volumes:
    db:

Dockerfile:

FROM mariadb:latest

SHELL ["/bin/bash", "-ec"]

[rimelek (Ákos Takács)]

@bernhardkrop

I found a lot of topics (and documentation), which suggests using mcvlan as a network driver would be the solution. But it does not work (or I am doing something wrong).

@deanayalon in the topic you linked just wrote in this last response that MacVLAN in Docker Desktop will not work, and it is true.

I read in the linkeed topic that you didn’t even had the IP on the host from which you wanted to forward a port (maybe I misunderstood it). You say the solution that worked then, didn’t work now, but unless you share what you did exactly, I don’t think there is much we can say.

Please share what the error message is, or what you do to confirm what you want doesn’t work.

[bernhardkrop]

@rimelek

You say the solution that worked then, didn’t work now, but unless you share what you did exactly, I don’t think there is much we can say.

My solution on Mac was to create a new network in the network manager GUI (I can’t show a screenshot of it, because I no longer have a Mac). I had one network per project (because each project has its own host IP) and could easily turn them on / off per switch toggle (important, because having them on breaks the other technology, we are still using in most projects).

I tried the same on my new Ubuntu (screenshot below), but regardless of what I try, the Ubuntu is then using the new network for all internet connections; rendering me offline (another problem). I couldn’t get it to bind on to that created network, either (main problem). Both problems might be connected and could just be caused by wrong configurations. But all I tried didn’t work.

@rimelek

Please share what the error message is, or what you do to confirm what you want doesn’t work.

I write the host IP and port (e.g. http://192.168.59.123:1234/) into my browser and press enter. My browser says then, that there’s no connection. If I use localhost in my docker-compose.yml and go to http://localhost:1234/, then I reach the database.

[rimelek (Ákos Takács)]

So if I understand it correctly, the problem is that the request is sent to the port but the response is routed through a network which is isolated from your machine. At least I could imagine something like that. If you are not sure, you could try monitoring packages using tshark on the server for example and filter to the source IP

If you need to route the traffic from a container through a specific IP on an interface, you can check what I wrote here:

Force all traffic for a speicifc container to go through a specific interface

For outgoing traffic you can use the com.docker.network.host_ipv4 driver options. networks: net_192_168_218_9: driver_opts: com.docker.network.host_ipv4: 192.168.218.9

But I think the response should always be routed through the same interface so this might not be a solution for you. Checking packages with tshark could still help.

[bernhardkrop]

@rimelek

If you are not sure, you could try monitoring packages using tshark on the server for example and filter to the source IP

I am not using any server. The docker containers are running locally on my Ubuntu notebook. The request is not reaching into the container. All logs remain empty. So I don’t think it’s a routing problem.

As I wrote in my original post above, according to documentation and my internet research, the solution would be using the mcvlan driver. But every time I try using it, the container is no longer starting and provides me this error message:

[meyay (Metin Y.)]

@bernhardkrop

according to documentation and my internet research, the solution would be using the mcvlan driver.

This is true with docker-ce, but does not work with Docker Desktop

With Docker Desktop, the utility vm that runs the Docker Engine itself is in a Natted network. So intead of having a macvlan with your host’s network, it is able to create a mavlan attached to the natted network of the utility vm. This does not help your situation.

The easiest and cleanest solution would be to use docker-ce directly on the host.

[rimelek (Ákos Takács)]

@bernhardkrop

I am not using any server.

Sorry, I try to keep up with the topics and sometimes I forget important details which I knew minutes ago… But at that point it was not actually important as “server” meant only the machine on which you have the port that you want to acces. If it is the same machine on which the Docker client and your browser is, it is just easier, but you can still check what happens with the packages. If you don’t even see incoming packages, then something indeed blocks the traffic even before reaching your machine which could indicate that the IP is not right, since you are on the same machine. We haven’t seen that you actually have that IP on the machine this time after you wrote it was missing when you had a similar issue on Mac. I assume you have it now, but I still have to mention that in case you accidentally incorrectly added the IP which didn’t work. You can check it by running any service directly on the host on that IP like a simple http server using python:

python3 -m http.server --bind IPADDRESS 8888

Then you can try to access it from a browser or from a terminal.

If that works, you can try Docker. And if you use tshark and see packages are arriving but rejected or no response can go back, then you know what to investigate next.

Regarding maclan, now three of us told you the same

@rimelek

@deanayalon in the topic you linked just wrote in this last response that MacVLAN in Docker Desktop will not work, and it is true.

@meyay

This is true with docker-ce, but does not work with Docker Desktop

And it’s true, you can switch to Docker CE on Linux if you want to and allowed to, but it would be just a workaround, and if you are not familiar with MacVLAN, a way to other potential issues.

[bernhardkrop]

I did some research about Docker CE (which seems to be using Docker Engine, directly; without Docker Desktop).

I installed it and tried to use it. Unfortunately, the container is no longer starting due to permission errors. I did some research and the solution seems to be using the volume modifier “z”. But it doesn’t work.

Can you help me on this new problem, too?

[bluepuma77]

Share your full modified compose file.

[rimelek (Ákos Takács)]

It doesn’t seem to be related to the topic we discussed so far, so we should probably move it to a separate topic, but first, please also try the error message. It would be really hard to help without knowing what you already know. The actual error message.

[bernhardkrop]

@rimelek

It doesn’t seem to be related to the topic we discussed so far, so we should probably move it to a separate topic,

Yes, that’s right. I will open a new topic. We should continue in this topic, once my new problem is fixed.

@rimelek

but first, please also try the error message. It would be really hard to help without knowing what you already know. The actual error message.

I will provide the actual error message in the new topic. I’ll edit my reply and provide a link to it, after I created it.

EDIT:

Here is the new topic.

[bernhardkrop]

I got a solution for my new problem. They container is starting again, but I still get an error message. It’s different from the one before:

EDIT:

The image seems to be too small. Therefore, here is the text:

“Error response from daemon: failed to set up container networking: driver failed programming external connectivity on endpoint […] (b829af7787867258dd2495d7e822ad6cac8273a8d21fe117cfe67986294976cc): failed to bind host port for 192.168.[…]:3306:172.18.0.2:3306/tcp: cannot assign requested address
docker-compose process finished with exit code 1”

[bluepuma77]

The error is about ports, can you share the compose file?

[rimelek (Ákos Takács)]

@bernhardkrop

cannot assign requested address

This is what you get when you try to port forward from an IP that does not exist on the host. So

@rimelek

We haven’t seen that you actually have that IP on the machine this time after you wrote it was missing when you had a similar issue on Mac. I assume you have it now, but I still have to mention that in case you accidentally incorrectly added the IP which didn’t work.

docker run --rm -it -p 192.168.9.9:80:80 nginx

Output:

docker: Error response from daemon: failed to set up container networking: driver failed programming external connectivity on endpoint gracious_heisenberg (5a466f67fb88bf0c01f56ab694787355512da8511a9fdac82ba81b017ccc719b): failed to bind host port for 192.168.9.9:80:172.16.0.2:80/tcp: cannot assign requested address

Have you tried running a server with python as I suggested? I don’t think that would work either

[bernhardkrop]

@rimelek

Have you tried running a server with python as I suggested?

No, I totally forgot about it, due to the new problem. I did it right now and this is the output:

I guess it’s the same problem I had on my old Macbook. But I am not able to create a new network on my Ubuntu like on my Macbook. I thought, that using the macvlan driver would listen on that IP and port automatically. At least, that’s what I understood from reading the documentation.

@bluepuma77

The error is about ports, can you share the compose file?

My current compose file looks like this:

# Note: Values have been anonymized.
name: my-project-name
networks:
    database:
    project:
        driver: macvlan
        driver_opts:
            parent: my-network-interface
        ipam:
            config:
                - gateway: 192.168.59.1
                  subnet : 192.168.59.0/24
services:
    mariadb:
        build:
            context: /my/build/context/path
            dockerfile: /my/dockerfile/path
        container_name: my-project-name-mariadb
        entrypoint: /my/entrypoint/path
        env_file: /my/env_file/path
        networks:
            database:
            project:
                ipv4_address: 192.168.59.123
        ports:
            - host_ip: 192.168.59.123
              mode: host
              protocol: tcp
              published: 1234
              target: 1234
        volumes:
            - source: ./database
              target: /my/bind/database/target/path
              type: bind
            - bind:
                  create_host_path: true
              source: ./database/data
              target: /my/bind/database/data/target/path
              type: bind
            - bind:
                  create_host_path: true
              source: ./log
              target: /my/bind/log/target/path
              type: bind

[meyay (Metin Y.)]

@bernhardkrop

        ports:
            - host_ip: 192.168.59.123
              mode: host
              protocol: tcp
              published: 1234
              target: 1234

Remove this part form your compose file, and it will work.The container ip is already the ipv4 you want to use. There is no port publishing involved.

Due to a kernel security restriction, communication between the macvlan parent interface (my-network-interface in your compose file) and the macvlan child interfaces (=the ones the containers use) is not possible. Every other host in your 192.168.59.0/24 subnet can access the container port using 192.168.59.123:1234. Note: There is a workaround to bypass the mentioned restriction: please use the forum search or google for “macvlan-shim”, and you will know what to do.

[bernhardkrop]

@meyay

Remove this part form your compose file, and it will work.

Unfortunately, it does not work.

The database is no longer available via localhost:3306 (as expected). Entering 192.168.59.123:3306 into my browser seems to reach the container (because I get a timeout instead of no connection), but not the database (logs remain empty). But the real problem is, that I cannot use other containers with the same IP (but other ports). That was possible on my Macbook using the ports part.

At least, the container started. I didn’t come so far, yet. But it seems, using macvlan is not the solution I hoped. Is there any other way to get several Docker containers to listen on different ports of the same IP?

[rimelek (Ákos Takács)]

You have to decide what you want. You either have macvlan and the container will get the IP (only one) or configure the network on the host.. MacVLAN can work only if you are already in the network on the host, so you have the interface on the host and you can add any number of new IP addresses in the same subnet. But if you choose MacVLAN, as @meyay pointed out, port forward cannot be used so localhost will not work either.

how you add a second IP address on the host depends on your host and the network manager software on the host. On a Desktop it is usually NetworkManager on Ubuntu, and it is netplan on a server. I haven’t configured network without netplan for a long time, but I’m sure you could find an example if you search for the right tool.

I guess you used something like this on Mac as I have an SSH tunnel script that uses it to add new IPs to the loopback interface:

sudo ifconfig lo0 alias "127.0.6.$i" up

On recent Ubuntu versions, you can use the ip command if you want to add the ip temporarily.

ip addr add 192.168.218.14/24 dev enp0s1

This adds 192.168.218.14 to the interface called enp0s1. Now this is how my interface looks like in my Ubuntu VM:

2: enp0s1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:78:14:95 brd ff:ff:ff:ff:ff:ff
    inet 192.168.218.13/24 metric 100 brd 192.168.218.255 scope global dynamic enp0s1
       valid_lft 2195sec preferred_lft 2195sec
    inet 192.168.218.14/24 scope global secondary enp0s1
       valid_lft forever preferred_lft forever
    inet6 fd37:8bbf:f970:731:5054:ff:fe78:1495/64 scope global dynamic mngtmpaddr noprefixroute
       valid_lft 2591981sec preferred_lft 604781sec
    inet6 fe80::5054:ff:fe78:1495/64 scope link
       valid_lft forever preferred_lft forever

and I can make a process listen on the secondary interface as well and I can access it from outside the VM (from macOS host).

[bernhardkrop]

@rimelek

On recent Ubuntu versions, you can use the ip command if you want to add the ip temporarily.

ip addr add 192.168.218.14/24 dev enp0s1

This adds 192.168.218.14 to the interface called enp0s1.

Thank you very much! This was exactly what I was looking for. Now it works as intended.

EDIT:

I am not able to mark your answer as a solution. But I was able to in my other post. Does anyone know, why I cannot choose a solution?

[rimelek (Ákos Takács)]

@bernhardkrop

I am not able to mark your answer as a solution. But I was able to in my other post. Does anyone know, why I cannot choose a solution?

It was not alloed in this category. I know it is not enabled everywhere and it should be in some categories, so we will need to review that but I enabled it in this category now. Thank you for pointing out it was not enabled.