It’s not so complicated but still I’ve been stuck with an unknown problem for quite a few days which is constantly refusing to be solved. 
Basically, on a high-level, I did a fresh install of docker engine on Ubuntu VDI using this ref, add customized daemon configuration /etc/docker/daemon.json, create a user-defined network and lastly try to connect the containers from both using containers’ hostname as well as containers’ ip.
Host OS (Virtual Machine connecting through RDP)
$ sudo lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.6 LTS
Release: 18.04
Codename: bionic
$ uname -a
Linux IS*******1 4.15.0-189-generic #200-Ubuntu SMP Wed Jun 22 19:53:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
HOST Network Interfaces
$ docker network ls
NETWORK ID NAME DRIVER SCOPE
31ad3a96c5cc br-net bridge local
83bfcacaf953 bridge bridge local
969622e0a27b host host local
7dd1f34794f4 none null local
$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether e2:d1:68:5c:b5:41 brd ff:ff:ff:ff:ff:ff
inet 10.54.62.14/24 brd 10.54.62.255 scope global dynamic noprefixroute eth0
valid_lft 653308sec preferred_lft 653308sec
inet6 fe80::5527:ec7:24a3:6325/64 scope link noprefixroute
valid_lft forever preferred_lft forever
39: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:fa:5c:8c:01 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
40: br-31ad3a96c5cc: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:89:b4:03:4a brd ff:ff:ff:ff:ff:ff
inet 172.18.0.1/16 brd 172.18.255.255 scope global br-31ad3a96c5cc
valid_lft forever preferred_lft forever
56: veth16f5946@if55: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-31ad3a96c5cc state UP group default
link/ether 7e:a3:b9:fd:bc:ca brd ff:ff:ff:ff:ff:ff link-netnsid 0
58: veth2164bfb@if57: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-31ad3a96c5cc state UP group default
link/ether 46:d5:4e:63:14:a3 brd ff:ff:ff:ff:ff:ff link-netnsid 1
HOST iptables -L
$ sudo iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
DOCKER-USER all -- anywhere anywhere
DOCKER-ISOLATION-STAGE-1 all -- anywhere anywhere
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
DOCKER all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
DOCKER all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain DOCKER (2 references)
target prot opt source destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target prot opt source destination
DOCKER-ISOLATION-STAGE-2 all -- anywhere anywhere
DOCKER-ISOLATION-STAGE-2 all -- anywhere anywhere
RETURN all -- anywhere anywhere
Chain DOCKER-ISOLATION-STAGE-2 (2 references)
target prot opt source destination
DROP all -- anywhere anywhere
DROP all -- anywhere anywhere
RETURN all -- anywhere anywhere
Chain DOCKER-USER (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere
HOST iptables -S
$ sudo iptables -S
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N DOCKER
-N DOCKER-ISOLATION-STAGE-1
-N DOCKER-ISOLATION-STAGE-2
-N DOCKER-USER
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o br-31ad3a96c5cc -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o br-31ad3a96c5cc -j DOCKER
-A FORWARD -i br-31ad3a96c5cc ! -o br-31ad3a96c5cc -j ACCEPT
-A FORWARD -i br-31ad3a96c5cc -o br-31ad3a96c5cc -j ACCEPT
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i br-31ad3a96c5cc ! -o br-31ad3a96c5cc -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o br-31ad3a96c5cc -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
HOST ip route
$ ip route
default via 10.54.62.254 dev eth0 proto dhcp metric 100
10.54.62.0/24 dev eth0 proto kernel scope link src 10.54.62.14 metric 100
169.254.0.0/16 dev eth0 scope link metric 1000
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
172.18.0.0/16 dev br-31ad3a96c5cc proto kernel scope link src 172.18.0.1
Content of /etc/docker/daemon.json
$ cat /etc/docker/daemon.json
{
"data-root" : "/opt/docker-data",
"debug": true,
"exec-opts": ["native.cgroupdriver=systemd"],
"log-level": "debug",
"log-driver": "json-file",
"log-opts": {
"max-size": "10m",
"max-file": "20"
},
"storage-driver": "overlay2"
}
Details of Docker Ingine
$ docker info
Client: Docker Engine - Community
Version: 26.1.0
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.14.0
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.26.1
Path: /usr/libexec/docker/cli-plugins/docker-compose
Server:
Containers: 4
Running: 4
Paused: 0
Stopped: 0
Images: 4
Server Version: 26.1.0
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: e377cd56a71523140ca6ae87e30244719194a521
runc version: v1.1.12-0-g51d5e94
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: builtin
Kernel Version: 5.4.0-177-generic
Operating System: Ubuntu 20.04.6 LTS
OSType: linux
Architecture: x86_64
CPUs: 6
Total Memory: 31.33GiB
Name: IXXXXXX001
ID: b8ca102c-8e16-4f87-a0bf-beb74cd6296f
Docker Root Dir: /opt/docker-data
Debug Mode: true
File Descriptors: 74
Goroutines: 105
System Time: 2024-04-25T00:27:48.270028804+03:00
EventsListeners: 0
Username: ghoshsayak
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: No swap limit support
Details of Docker Components
$ docker network create br-net
Create Containers:
$ docker run -d --hostname debugger --network br-net --name debugger ghoshsayak/local-docker-env:debugger-20240420
$ docker run -d --hostname sqlserver --network br-net --name sqlserver --env MSSQL_SA_PASSWORD='xxxxx' --env ACCEPT_EULA='Y' --env TZ='Asia/Jerusalem' mcr.microsoft.com/mssql/server:2019-CU18-ubuntu-20.04
Container “debugger” and “sqlserver” is running properly.
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
be15cc347760 mcr.microsoft.com/mssql/server:2019-CU18-ubuntu-20.04 "/opt/mssql/bin/perm…" About an hour ago Up About an hour 1433/tcp sqlserver
9a908ca983c9 ghoshsayak/local-docker-env:debugger-20240420 "python3 /server.py" About an hour ago Up About an hour 5000/tcp debugger
Preflight Network tests on “debugger” container
$ docker exec -it debugger /bin/bash
root@debugger:/# nc -v -l -p 5000
nc: Address already in use
root@debugger:/#
root@debugger:/# hostname
debugger
root@debugger:/# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
55: eth0@if56: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:ac:12:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 172.18.0.2/16 brd 172.18.255.255 scope global eth0
valid_lft forever preferred_lft forever
root@debugger:/#
root@debugger:/# netstat -plntu | grep LISTEN
tcp 0 0 127.0.0.11:37411 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:5000 0.0.0.0:* LISTEN 1/python3
root@debugger:/#
root@debugger:/# curl http://localhost:5000
Hostname: debugger
root@debugger:/#
root@debugger:/# curl http://172.18.0.2:5000 -v
* Trying 172.18.0.2:5000...
* Immediate connect fail for 172.18.0.2: Connection refused
* Closing connection 0
curl: (7) Couldn't connect to server
root@debugger:/#
root@debugger:/# curl http://debugger:5000 -v
* Trying 172.18.0.2:5000...
* Immediate connect fail for 172.18.0.2: Connection refused
* Closing connection 0
curl: (7) Couldn't connect to server
root@debugger:/#
root@debugger:/# nslookup debugger
Server: 127.0.0.11
Address: 127.0.0.11#53
Non-authoritative answer:
Name: debugger
Address: 172.18.0.2
root@debugger:/#
root@debugger:/# nslookup sqlserver
Server: 127.0.0.11
Address: 127.0.0.11#53
Non-authoritative answer:
Name: sqlserver
Address: 172.18.0.3
root@debugger:/#
root@debugger:/#
root@debugger:/# telnet 172.18.0.3 1433
Trying 172.18.0.3...
telnet: Unable to connect to remote host: Connection refused
root@debugger:/#
root@debugger:/# telnet sqlserver 1433
Trying 172.18.0.3...
telnet: Unable to connect to remote host: Connection refused
root@debugger:/#
root@debugger:/#
root@debugger:/# ip route
default via 172.18.0.1 dev eth0
172.18.0.0/16 dev eth0 proto kernel scope link src 172.18.0.2
root@debugger:/#
root@debugger:/# traceroute debugger
traceroute to debugger (172.18.0.2), 30 hops max, 60 byte packets
send: Connection refused
root@debugger:/#
root@debugger:/# traceroute 172.18.0.2
traceroute to 172.18.0.2 (172.18.0.2), 30 hops max, 60 byte packets
send: Connection refused
root@debugger:/#
root@debugger:/# traceroute sqlserver
traceroute to sqlserver (172.18.0.3), 30 hops max, 60 byte packets
send: Connection refused
root@debugger:/#
root@debugger:/# traceroute 172.18.0.3
traceroute to 172.18.0.3 (172.18.0.3), 30 hops max, 60 byte packets
send: Connection refused
root@debugger:/#
Preflight Network tests on “sqlserver” container
SAME AS ABOVE BEHAVIOUR.
I tried to provide as much as information possible.
Now hoping for someone who can really help me out with!
Thanks.