I’m super ignorant. With that in mind…
I have Portainer running on a NixOS machine. I have the CloudflareD daemon running in a docker container on that same machine. I configured the non-container-Clourdflare-web-console-side correctly (correct IP address and port number). I can hit the intended domain, the auth-protections I put in place in the Cloudflare web console work, but then I get a 502 bad gateway error when the Cloudflare container attempts to reach the service I want to expose. The service I’m trying to access through the CloudflareD daemon is up and running. It’s available and working correctly when I access it from another machine on the local network that the NixOS machine is connected to.
The issue appears to be that the CloudflareD container cannot talk to the other containers. I setup multiple subdomains on the Clourdflare web console side to confirm that this issue was not just a single container issue. The CloudflareD container cannot talk any other containers.
Both containers, the CloudflareD container and the service I’m trying to expose, are on the “bridge (System)” network. “enable_icc” is true on that network. So they share a network.
I tried to connect to the console of the CloudflareD container via Portainer (using /bin/bash in Containers > cloudflared-tunnel > Console) and nothing happens. When I click the Connect button in that page it flashes “Exec into container as default user using command bash” and then it goes right back to the Containers > cloudflared-tunnel > Console page. No notification, no error message. Checking the logs for that container, there are no new logs generated from that console-connection-attempt.
So I SSH’d to the NixOS machine and tried running the docker commands directly. Running docker container exec -it cloudflared-tunnel bash returns this error: OCI runtime exec failed: exec failed: unable to start container process: exec: "bash": executable file not found in $PATH: unknown (remember I don’t know anything so…) I believe that means bash isn’t found in the $PATH ENV variable in Containers > cloudflared-tunnel under the “Container details” section. Here is what is listed there: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin .
My hope was to get into the container to see if I could ping the IP address of the container that is running the service I’m trying to expose. Then I learned you can do that with this command: docker exec cloudflared-tunnel ping [the name of the container you are trying to ping] -c2 . Running that command returns this error: OCI runtime exec failed: exec failed: unable to start container process: exec: "ping": executable file not found in $PATH: unknown .
I keep running into road blocks when I try to troubleshoot. I think those road blocks are actually the same issue over and over again (something wrong with the $PATH I guess). I’m just not knowledgeable enough to know what the problem is, where to look for signs of the problem(s), or how to Google-foo my way to an answer, apparently.
Edit: Still unresolved but here’s a touch more info. I thought maybe since its a NixOS machine, docker was looking in a specific place for bash (likely “/bin/bash”). So I ran whereis bash on the machine and it returned /nix/store/(hidden_for_privacy_reasons)-system-path/bin/bash. I was hoping that maybe adding this path to the $PATH would resolve the issue. So I did that via Portainer, restarted the container and still nothing. I also went back to the Containers > cloudflared-tunnel > Console page and entered that /nix/store/ path as a “custom path” for bash and that also didn’t work.
I’m not even sure if thats the issue now. I would think that Nix would create a symlink from “/bin/bash” to that “/nix/store/-system-path/bin/bash” location so that things looking for /bin/bash wouldn’t run into issues. But I could be wrong about that (or literally all of this lol).