I am running Open Media Vault (OMV) 5, which looks to be Debian 10. I am running on an old rack mounted server that I picked up a few years ago. My router is running DDWRT. I am a software developer with a working knowledge of linux (enough to comfortably follow most guides).
I’ve been using Open Media Vault for several years (going back to R4 of OMV) and I’ve had this problem since the beginning. I have never been able to forward a port to any apps running on Docker. I am talking about the standard containers - SABNZBD, PiHole, Sonarr, etc. Everything works fine internally - I can access the containers via the port in the browser, my kodi boxes can access shared library in MariaDB, etc. The issue only seems to affect Docker - when I was running on R4 I had a windows instance running in VirtualBox and I had no problems passing ports through to programs running there. I have also had no problems passing ports through to other windows boxes in my environment. I am not doing anything fancy at the network level.
So as part of my debugging over in the OMV forum the mods had me shut down Docker and then forward port 80 to the OMV box. Sure enough, the OMV UI came up. The instant I started Docker inside of OMV I could no longer access the OMV UI from the outside. At that point they said I should hop over here to seek help.
As I said earlier I don’t do this in my day job but I am happy to provide any other debugging information that I can provide. Thank you very much in advance!
I don’t use OMV but I can try to give you some advice based on your described experience. If I can’t help, hopefully someone else will see your question.
My first guess is almost always the firewall. I know ubuntu has “ufw” but I am not sure about Debian 10. Even if Debian 10 does not have it by default, it could be installed on OMV. Since you mentioned that the OMV UI works only when Docker is not running and Docker also uses iptables, it might somehow interfere with the rules set by an other service on OMV.
Do you know what is the firewall in OpenMediaVault?
Update:
I was curious, so I installed openmediavault in UTM on Mac. Then I installed openmediavault extras and Docker. Finally I run a docker container on port 8080 forwarded to the container’s port 80. In addition I had to forward ports from the Mac host to OMV. It works. Docker and the OMV UI together. I also rebooted it and still works.
How did you install Docker?
I can also see that OMV has firewall settings in System / Network / Firewall
Thank you for the reply @rimelek, I really appreciate it!
No, I have no firewall entries at all set in OMV nor do I have any kind of specialized network settings on my network. I know that this works for most people but I also came across others that had the same issue that I am having. As you said, it seems to do with the iptables but that is well beyond my understanding. Any other thoughts?
I don’t have other ideas than trying to disable any firewall from terminal. You can try to empty the iptables rules, but you should save the current rules before that. You can save it using iptables-save
Then you can try to clear the rules: linux - best way to clear all iptables rules - Server Fault
If you can, try it on a test virtual machine before OMV.
Thanks so much. I ran the command and rebooted a couple of times - no joy. I ran an online scan and every port forward to a windows machine works and every pointed at a docker container did not. Any other thoughts?
Oh, I thought that this was a permanent thing. Just to be clear, after reboot the containers loaded up and appeared normal. The issue is that I can’t forward ports to them from my ddwrt router. You should I should do it again but not reboot?
EDIT: I tried it again but this time did not reboot. Same issue - I can hit the containers from the ports internally but not if I pass them through ddwrt firewall.
So Open Media Vault is being served on port 80. If I forward port 80 from my ddwrt router the port shows as filtered and I can’t hit it from the outside. The second I stop the docker service the port shows as open and I can hit the OMV admin console from the outside on port 80. What does that tell me?
you can only have 1 service using/forwarding the port (also aplies to the DDWRT ), so some docker app is also using port 80, either change the OMV admin service portnumber with omv-firstaid,
or
use vlan bridges, an elegant way would be iobroker
Can you give us more precise details, like the run commands or the compose files that you use to crate the containers, so we can see that everything is okay with how the ports are published. Next step can we see details about the portforwarding in your router?
Just to be sure, this is for ipv4, isn’t it? Ipv6 would be a whole different game.
also, are you sure in DD-WRT it is sufficient just to add portforwarding? Wasn’t it one of those Router OS’es that additional require the firewall to be opened for that particular port additionaly?
Thank you so much for the responses! Let me try to give some more information.
IP4 only. No firewalls or other configurations.
I’ve had these containers set up for at least a couple of years. They were created via the Open Media Vault UI and I spent a fair amount of time in their forums trying to fix the issue before we realized that the issue has to be specific to Docker (see next section).
Just to be clear, each container (SABNzbd, Sonarr, etc) have their own port and I can access them internally with no issue. I mentioned the part about OMV running on port 80 to point out that I can’t even forward to that port if Docker is running. If I stop the Docker service then it is immediately available. Of course, that’s not the issue I am trying to solve - I was just adding this in for more information.
For apps that I am trying to serve externally, I’ve never ever had a problem forwarding a port to a windows pc on my network. I’ve also never had a problem forwarding to a windows instance that was running on virtualbox inside of OMV on the same IP address (OMV has since dropped support for virtualbox). I only have issues if I am trying to forward to containers in docker.
So the one thing I have tried so far is running iptables -F and then checking the ports. I would be delighted to gather any other information that might help to solve this issue. Thanks!
Edit: forgot to add a screen cap of the port forwarding. I’ve tried both ranges and without ranges and the result was the same. I read in the ddwrt forum that ranges are recommended. As you can see I am currently forwarding to the windows pc without issue. There are no firewalls enabled in OMV and the admins there said that they could not find anything wrong with my setup (I was following accepted guides when I made the containers).
I am still completely in the dark how you run your conatiners. Do you use macvlan or bridged networks with published ports?
I just want to be sure to understand it properly, because I have seen people adding the docker host as gateway to bridged networks… you’d be surprised with what of kind solutions people can come up with.
after your update: is 192.168.1.206 your docker-host?
To be honest I don’t have a great understanding of how all of that works. I generally just follow the guides and that doesn’t always give you a deeper dive into what all of the settings mean. I know for a fact I am not running macvlan because I had previously set one up for PiHole (no longer in use).
.206 is my windows pc and .100 is OMV. Obviously things are currently pointed towards the windows pc.
Here is my plex setup (obviously it doesn’t tell you much). That was one of the first clues that I had an issue as Plex’s dashboard does a good job telling us if it can connect to the outside. It knows everything is fine when I point to .206 and not fine when pointed to .100.
I am very sorry, I should have been clearer. As I said in my reply, all of my primary containers were made via the Open Media Vault UI by following guides. This means that there isn’t a yml file, docker compose, etc. You enter a few parameters and hit submit and the work is done for you. I recently did spin up a container using a yml file but I had the same result with the port forwarding. If there is one that you think would be good to try I would be happy to do so since I’ve finally been learning more about creating them by hand.
I am playing around with his a little more today in the hopes of finding some more information. I’ve attached my docker info information below. I am trying to add the unify-controller to docker via command compose. I am getting this error:
starting container failed: error creating external connectivity network: could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network
I did some searching - apparently docker prune fixes this for many people but it did not for me (I also rebooted a couple of times).
The compose file looks fine, though the error message you get does absolutly not. Is your compose file just an example or did you actualy create the container using docker-compose? This makes a huge difference - as compose will create a user defined network, while docker run or an admin ui might just add the container to the “default” bridge network (which is considered lagacy anyways).
Anyway, you shouldn’t get such an error, with the default settings in /etc/docker/daemon.json (which also apply if the file is missing). Can you share the content of /etc/docker/daemon.json, just to be sure it does not contain settings that lead to what you experience.
At least the error would explain why containers are not reachable from outside.
Thank you again for all of your help! I think I found out the culprit. While researching this error I came across an entry in a forum where it says that having openvpn installed in one of the dockers can cause an issue like I reported earlier today. So I set the container to not restart and re-started OMV. Sure enough, I was able to process the yml file that I included in my last post. So I hopped over to my router and forwarded the ports back to docker - they are now working as well!
So back to the culprit - I installed a docker that has both openvpn and transmission together and it is set to have openvpn keep a constant link. Was this spoiling all of the network traffic on that node?
