What could be preventing SYN-ACK response?

Docker Engine General
1

Hello,

I have a networking issue that I think is docker-specific due to tcpdump output. Below is a detailed description but the summary is:

I have a privoxy server that is not reachable from a specific IP range because it will not send SYN-ACK responses and TCP connections time out

Here is why I think docker may be causing this:

Inside the docker container:

First this is what the compose layout looks like:

CONTAINER ID   IMAGE                                 COMMAND                  CREATED          STATUS                    PORTS                                                                          NAMES

24dc02575ee7   alpine-privoxy/latest                 "privoxy-start.sh"       31 minutes ago   Up 31 minutes                                                                                            privoxy2

b680bd9748e9   
             "/init"                  31 minutes ago   Up 31 minutes (healthy)   0.0.0.0:3128->8118/tcp, [::]:3128->8118/tcp                                    nordlynx

The simple idea is that the proxy allows you to use the VPN without running it on your own PC. The docker compose file exposes port 3128 which is mapped to 8118 inside the container where privoxy listens for connections and services them via the VPN container called nordlynx. This works in general for subnet 192.168.9.0/24 which is my main subnet, but does not work for other subnets like 192.168.43.0/24 which time out trying to connect to 3128.

Here is my running nc -vz 192.168.40.3 3128 on 192.168.43.101 to simply see if I can connect to privoxy, while tcpdump inside the container shows what happens to the traffik:

dockerapps@docker ~$ docker exec -it privoxy2 tcpdump -n 'not host 192.168.9.174 and port 8118'
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes

15:54:10.016706 IP 192.168.43.101.48078 > 172.20.0.2.8118: Flags [S], seq 3997310274, win 64240, options [mss 1460,sackOK,TS val 2548798020 ecr 0,nop,wscale 10], length 0

15:54:11.080435 IP 192.168.43.101.48078 > 172.20.0.2.8118: Flags [S], seq 3997310274, win 64240, options [mss 1460,sackOK,TS val 2548799083 ecr 0,nop,wscale 10], length 0
... (repeated many times)

The connection times out and as you can see the SYN packet is coming in from 192.168.43.101 but there is no SYN-ACK reply going back… Machines from other subnets like 192.168.9.0/24 can reach the privoxy service without issues…

Now, AFAIK there is no way to prevent the OS from generating a SYN-ACK at the application level, so this is likely a docker/linux issue?

Outside docker, inside the LXC running docker:

I know it is not recommended to run docker nested inside an LXC, but that seems to be the case here. I thought it is some iptables problem filtering out traffic, so here is the output from inside the LXC that runs docker:

root@docker ~# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
f2b-SSH tcp -- anywhere anywhere tcp dpt:ssh

Chain FORWARD (policy DROP)
target prot opt source destination
DOCKER-USER all -- anywhere anywhere
DOCKER-FORWARD all -- anywhere anywhere

Chain OUTPUT (policy ACCEPT)
target prot opt source destination

Chain DOCKER (4 references)
target prot opt source destination
ACCEPT tcp -- anywhere 172.20.0.2 tcp dpt:8118
ACCEPT tcp -- anywhere 172.19.0.4 tcp dpt:https
ACCEPT tcp -- anywhere 172.19.0.4 tcp dpt:http
DROP all -- anywhere anywhere
DROP all -- anywhere anywhere
DROP all -- anywhere anywhere
DROP all -- anywhere anywhere

Chain DOCKER-BRIDGE (1 references)
target prot opt source destination
DOCKER all -- anywhere anywhere
DOCKER all -- anywhere anywhere
DOCKER all -- anywhere anywhere
DOCKER all -- anywhere anywhere

Chain DOCKER-CT (1 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED

Chain DOCKER-FORWARD (1 references)
target prot opt source destination
DOCKER-CT all -- anywhere anywhere
DOCKER-ISOLATION-STAGE-1 all -- anywhere anywhere
DOCKER-BRIDGE all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target prot opt source destination
DOCKER-ISOLATION-STAGE-2 all -- anywhere anywhere
DOCKER-ISOLATION-STAGE-2 all -- anywhere anywhere
DOCKER-ISOLATION-STAGE-2 all -- anywhere anywhere
DOCKER-ISOLATION-STAGE-2 all -- anywhere anywhere

Chain DOCKER-ISOLATION-STAGE-2 (4 references)
target prot opt source destination
DROP all -- anywhere anywhere
DROP all -- anywhere anywhere
DROP all -- anywhere anywhere
DROP all -- anywhere anywhere

Chain DOCKER-USER (1 references)
target prot opt source destination

Chain f2b-SSH (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere

I am not well versed in iptables, but it seems to me that port 8118 should accept connections from anywhere? Why are SYN packets from 192.168.43.101 not being responded to?

2

This looks wrong. Is it safe to assume that you build this image yourself? Since the image is not publicly available we can not check it’s description.

Never the less, how is the container started? Since the privoxy container has no mappings on it’s own, it looks like it uses the network namespace of the nordlynx container.

All vpn containers I came across in this forum had an environment variable to configure the subnets that are allowed to access it. The output of docker ps does not show the used image for whatever reason.

3

Thank you @meyay for this guidance.

EDIT: accidentally pressed Ctrl-Enter and didn’t type the complete response. See below

Indeed there is a compose file with an extra container for nordvpn (where I specify the network you correctly said is not shown here) and indeed there is a variable for specifying allowed IPs to access that VPN contaienr. The problem was that I had mistyped the 192.168.43.0/24 with a comma instead of a dot between 43 and 0, which apparently caused all this pain.

I sincerely appreciate your response, I had given up on finding this. I had actually checked this already but the dot/comma for my age looks identical. After your suggestion I paid close attention and spotted my typo. Marking this as solved by your reply.

4

Sometimes things can be so easy, but still hard to spot :smiley:

Glad you identified the problem and applied the solution!