Unable to connect to host service from inside Docker container

I am running a service on my docker host that I am trying to access from a docker container. The service is accessible from the host, but I cannot access the service from inside my Docker container. I have tried using both the host private IP address (10.0.0.60) and the IP address of the docker0 bridge interface (172.17.0.1), and neither work. I have tried binding the service on the host to both 0.0.0.0 and 172.17.0.1, and neither makes the service visible to the container.

Running curl localhost:9090 on the host works correctly, but running curl 172.17.0.1:9090 inside the docker container returns curl: (7) Failed to connect to 172.17.0.1 port 9090 after 1 ms: Couldn't connect to server. Same happens when I try to use 10.0.0.60.

Running netstat -tulpn on the host gives tcp6 0 0 :::9090 :::* LISTEN 210664/java, and running the same command in the docker container does not show any service running on port 9090.

The only thing I can suspect would be causing this would be a firewall, however I have tried adding several rules to UFW and even disabling the firewall outright, and nothing seems to allow the docker container to access the service on the host. My docker-compose file is as follows:

version: '3.8'
services:
  app:
    image: 'jc21/nginx-proxy-manager:latest'
    restart: unless-stopped
    ports:
      - '80:80'
      - '81:81'
      - '443:443'
    volumes:
      - ./data:/data
      - ./letsencrypt:/etc/letsencrypt

I am using Docker Engine 27.4.1 on Ubuntu 24.04.1 Arm64

Unfortunately, a hostname alias host.docker.internal is set to the host only when using Docker Desktop

Using the host machineā€™s private/public IP SHOULD work - what error do you get with 10.0.0.60?

Please share the output of these commands (execute as root), so we get a better idea of whatā€™s happening:

docker info

ip address show scope global

netstat -tulpn | grep ':9090 '

Result of docker info:

Client: Docker Engine - Community
 Version:    27.4.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.19.3
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.32.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 3
  Running: 3
  Paused: 0
  Stopped: 0
 Images: 3
 Server Version: 27.4.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 88bf19b2105c8b17560993bee28a01ddc2f97182
 runc version: v1.2.2-0-g7cb3632
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.8.0-1018-oracle
 Operating System: Ubuntu 24.04.1 LTS
 OSType: linux
 Architecture: aarch64
 CPUs: 4
 Total Memory: 23.42GiB
 Name: instance-20240302-2000
 ID: 01a02367-6e8e-4fb6-8649-cf6af1baa135
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Result of ip address show scope global:

2: enp0s6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
    link/ether 02:00:17:02:b1:08 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.60/24 metric 100 brd 10.0.0.255 scope global noprefixroute enp0s6
       valid_lft forever preferred_lft forever
3: tailscale0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1280 qdisc fq_codel state UNKNOWN group default qlen 500
    link/none
    inet 100.125.248.30/32 scope global tailscale0
       valid_lft forever preferred_lft forever
    inet6 fd7a:115c:a1e0::3401:f81e/128 scope global
       valid_lft forever preferred_lft forever
4: br-bc1f42053a90: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:cb:64:1d:8d brd ff:ff:ff:ff:ff:ff
    inet 172.21.0.1/16 brd 172.21.255.255 scope global br-bc1f42053a90
       valid_lft forever preferred_lft forever
5: br-34ed8c59fd45: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:06:8d:cf:2a brd ff:ff:ff:ff:ff:ff
    inet 172.22.0.1/16 brd 172.22.255.255 scope global br-34ed8c59fd45
       valid_lft forever preferred_lft forever
6: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:91:de:ab:43 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
43: br-70edc0a12346: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:3a:78:23:77 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.1/24 brd 192.168.0.255 scope global br-70edc0a12346
       valid_lft forever preferred_lft forever

Result of netstat -tulpn | grep ':9090 '

tcp6 0 0 :::9090 :::* LISTEN 1012491/java

Based on the output you shared, itā€™s safe to say that this should have worked:

The ip of the docker0 interface should be accessible from any container. I assume your java process does listen on ipv4 and ipv6 and therefor should also bind to the ip of docker0.

Your os uses a non default kernel. Is your Linux running in some sort of os container technology like lxc container?

Update:
What is the result, if you execute the command on the host itself? curl 172.17.0.1:9090

Iā€™m unsure of the exact technology used for containerization, but my host machine is a VPS running on Oracleā€™s OCI plaftorm. Fastfetch labels my host as KVM Virtual Machine (virt-4.2), so itā€™s definitely running under some sort of virtualization technology.

Running curl 172.17.0.1:9090 on the host returns nothing, however looking at the logs of the application I can see a valid HTTP GET request reaching the server. The exact response from the server logs are:

2025.01.09 14:11:40:0000 [io-comp...] [INFO ] org.http4s.server.middleware.Logger - HTTP/1.1 GET /
2025.01.09 14:11:40:0001 [io-comp...] [INFO ] org.http4s.server.middleware.Logger - HTTP/1.1 303 See Other

It canā€™t be just a kvm vm, as there must be a reason why your os doesnā€™t use a default ubuntu kernel. Though, that should be irrelevant (at least I donā€™t see why it should be).

Your server already returns ā€œ303 See Otherā€. So we know it binds to the ipv4 address. Though, It is still an unusual redirect http code. Usually people use 301 or 302. You can make curl follow the redirect: curl -L 172.17.0.1:9090.

Note: If you canā€™t get it to work on the host itself, it wonā€™t be able to work from a container either.

Update:
To make sure that your application isnā€™t the problem, you can do following test.

Run a ad-hoc webserver on the host:

python3 -m http.server 8080

Note change the port from 8080 to another free port, if port 8080 is already used on your host.

Then open a second terminal and test whether its possible to reach the web server from a container:

docker run --rm alpine wget -O - http://172.17.0.1:8080

Following the redirect using curl -L returns the HTML for the expected page. I am also able to access the application using my Tailscale VPN and accessing port 9090 from the browser. Accessing the application from the browser or using curl -L also ends with the application returning a HTTP 200 OK response.

Running the python webserver on the host and attempting to connect through the alpine container also results in the container not being able to access the host. Result of docker run --rm alpine wget -O - http://172.17.0.1:8080:

Connecting to 172.17.0.1:8080 (172.17.0.1:8080)
wget: can't connect to remote host (172.17.0.1): Host is unreachable

However, running wget -O - http://172.17.0.1:8080 on the host returns a successful response:

--2025-01-09 14:40:41--  http://172.17.0.1:8080/
Connecting to 172.17.0.1:8080... connected.
HTTP request sent, awaiting response... 200 OK
Length: 187 [text/html]
Saving to: ā€˜STDOUTā€™

-                                             0%[                                                                                           ]       0  --.-KB/s               <!DOCTYPE HTML>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Directory listing for /</title>
</head>
<body>
<h1>Directory listing for /</h1>
<hr>
<ul>
</ul>
<hr>
</body>
</html>
-                                           100%[==========================================================================================>]     187  --.-KB/s    in 0s

2025-01-09 14:40:41 (47.9 MB/s) - written to stdout [187/187]

You get really strange results. I concur with you conclusion that some sort of firewall must prevent it. I never run docker on a system which establishes a vpn connection, but I can imagine that it messes around with routes, the default gateway of the host and might even tinker with firewall rules.

Nothing I can realy help with. I hope you do find a solution!

Thank you so much for your support! Iā€™m wondering if my issue is somehow caused by the hosting configuration, as Iā€™m running an almost identical setup on a physical Debian server and that system has no issues accessing the host from inside containers. Iā€™ll try asking on the Oracle forums to see if thereā€™s any advice they can offer there.

Found a solution! To anyone in the future who stumbles on this post looking for answers, it seems that the default Ubuntu image for Oracle Cloud has strange firewall rules by default, running the command iptables -I INPUT -j ACCEPT as specified in this Reddit post solved my problem!

1 Like

I am glad you pinpointed the problem and found a solution. Thank you for sharing it!