Well, it was something related to the firewall, indeed, but I don’t really fully understand it…
I’ll try to explain it.
In the first situation, with no manually defined network, I have set up a firewall rule (ufw) to match the docker default range on the host:
5432 ALLOW IN 172.17.0.0/16
So, as the container is having the randomly assigned 172.20.0.1 Gateway, this seems logic to me that it cannot reach the local postgres service using the default one 172.17.0.1 (confirmation at the very bottom of this post).
(( also, listen_addresses = '*' in postgresql.conf and I allowed an even larger set of connections from this range 172.0.0.0/8 in pg_hba.conf on the local postgresql. These are the only modifications. ))
In the second example, when actually using the custom network, I have to change the firewall rule so that it matches the network, as follow (otherwise it doesn’t work):
5432 ALLOW IN 172.20.0.0/16
Now it works, either with 172.17.0.1 or 172.20.0.1… but…
…after some more investigations, I finally noticed in this second situation, that I can successfully connect to the host postgres with every single Gateway address of every running services; this includes 5 other IPs which are actually not listed in ufw (but these are covered by the large range in pg_hba.conf).
Fun fact: this is totally random, but I do have an other service running on this Gateway 192.168.208.1 and I can connect to the host postgres using this IP even so there is strictly no firewall rule for this IP, neither an entry in pg_hba.conf. So this one should be blocked, but it’s not…
Finally, in the first situation, if I change the ufw rule to match the randomly assigned Gateway (let’s say .22.) as follows: ALLOW IN 172.22.0.0/16 I can also reach the local postgres using whatever Gateway of any of the running services, including the default Gateway, exactly as in the previously described situation.
So to make it work, I have to allow an ufw rule matching the actual container Gateway, not the default 172.17.0.0/16, in order to access postgres. And in that case, pg is accessible through every single existing docker Gateway (even so they are not explicitly allowed by ufw (!!) Therefore, I don’t fully understand what exactly happens, this is some kind of black magic to me for the moment…) !
Lastly, if I set up a more restrictive 172.17.0.0/16 in pg_hba.conf and trying to connect, it confirm what I originally thought:
# notice the .17. here, but the current container Gateway is .22.
root@postgres:/# psql -d postgres://postgres@172.17.0.1:5432/postgres
psql: error: connection to server at "172.22.0.1", port 5432 failed: FATAL: no pg_hba.conf entry for host "172.22.0.2", user "postgres", database "postgres", SSL on
connection to server at "172.22.0.1", port 5432 failed: FATAL: no pg_hba.conf entry for host "172.22.0.2", user "postgres", database "postgres", SSL off
so pg refuses the connection from the actual container Gateway, not the docker default, which I gave.