Docker Compose through SSH failing and referring to docker.example.com

Hello!
I’m done with configuring Dockerfile and Docker Compose for my deployment and whenever I test it on the local machine it is perform cool - everything loads properly.

But when I try to deploy using ssh, I’m getting this very strange error:

error during connect: Post "http://docker.example.com/v1.24/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&labels=%7B%7D&memory=0&memswap=0&networkmode=default&rm=1&shmsize=0&t=project_web&target=&ulimits=null&version=1": write |1: broken pipe

Why does it try to post on docker.example.com? Where is my mistake?
Docker-compose and docker are running properly on remote machine… So how to get rid of this strange exception?

What command are you using for that?

I’m using this: docker-compose up -d
I’ve added DOCKER_HOST variable to my environment variables, so it uses needed machine as I can see.

Just to be 100% sure, for others who may be able to help: so you’re running docker-compose up -d on your local machine to start the services on the remote machine? And all you’ve configured is DOCKER_HOST? Which, I assume, does not refer to example.com, and the error message is really mentioning that domain?

Yus, exactly as you’ve said!

I am having a similar problem, but with setting a context. I have a Virtualbox VM (Ubuntu 20) running on my Mac. In the VM I have Docker installed rootlessly. The VM is called vmname and the user is username. Virtualbox exposes the VM port 22 on Mac port 3022. Docker client and server are both version 20.10.10.

I have this in my ~/.ssh/config file:

Host vmname
  HostName localhost
  User username
  Port 3022

So ssh works correctly:

mac$ ssh vmname

works correctly and gives me the vm console prompt.

vmname$ docker info

works correctly, giving me client and server info. Going back to the Mac,

mac$ unset DOCKER_HOST
mac$ docker context create vmname --docker "host=ssh://vmname"
mac$ docker context use vmname

seems to work correctly, because:

mac$ docker context ls
NAME                   TYPE          DOCKER ENDPOINT
vmname *             moby          ssh://vmname

But here’s the problem:

mac$ docker info
Client:
 Context:    vmname
 Debug Mode: false
 Plugins:
  buildx: Build with BuildKit (Docker Inc., v0.6.3)
  compose: Docker Compose (Docker Inc., v2.1.1)
  scan: Docker Scan (Docker Inc., 0.9.0)

Server:
ERROR: Cannot connect to the Docker daemon at http://docker.example.com. Is the docker daemon running?
errors pretty printing info

There seems to be a clue on Github in docker/engine/client/errors.go; it looks as though some doRequest() invocation is confused about the host:

// Error returns a string representation of an errConnectionFailed
func (err errConnectionFailed) Error() string {
	if err.host == "" {
		return "Cannot connect to the Docker daemon. Is the docker daemon running on this host?"
	}
	return fmt.Sprintf("Cannot connect to the Docker daemon at %s. Is the docker daemon running?", err.host)
}

If I power down the VM, I get a different error, which seems to imply that when the VM is running the ssh connection is getting through, but the Docker server is getting confused?

mac$ docker info
Client:
  <same>
Server:
ERROR: error during connect: Get "http://docker.example.com/v1.24/info": command [ssh -- vmname docker system dial-stdio] has exited with exit status 255, please make sure the URL is valid, and Docker 18.09 or later is installed on the remote host: stderr=ssh: connect to host localhost port 3022: Connection refused

I can’t find any other reports of this problem on Google, so I assume I am doing something very stupid. Any idea what?

OK, as I suspected, I was doing something stupid. For rootless operation, I needed to set the DOCKER_HOST environment variable globally on the Docker server.

I added the following line to /etc/environment and restarted the server:

DOCKER_HOST=unix:///run/user/1000/docker.sock

and everything works great now.

** UPDATE ** see below - as Ákos Takács points out, you don’t need to set DOCKER_HOST on the server for rootless mode to work. You just have to docker context use rootless on the server if it is not the default context.

I didn’t set DOCKER_HOST and it worked for me switching to the rootless context:

docker context use rootless

If it doesn’t work check the available contexts first

docker context ls

* in the output means that context is used

But if DOCKER_HOST works for you, that is also good :slight_smile: Keep in mind though, when you create additional contexts you won’t be able to switch between them until you delete the variable.

OK, it is a confusion between the context on the server and the context on the client.

Thanks Ákos, you are right - I don’t need to set DOCKER_HOST on any machine when in rootless mode.

On my local development system I can use docker context use <name> to switch between target remote servers.

On every remote server I use docker context use rootless

The ‘rootless’ context is installed on the server by the dockerd-rootless-setuptool.sh script.

I was getting a similar error using a Mac with macOS Monterrey:

Server:
ERROR: error during connect: Get "http://docker.example.com/v1.24/info": command [ssh -l myremoteuser -- REMOTE_IP docker system dial-stdio] has exited with exit status 255, please make sure the URL is valid, and Docker 18.09 or later is installed on the remote host: stderr=Permission denied, please try again.

previously, I generated and installed the public RSA key via the commands:

ssh-keygen
ssh-copy-id myremoteuser@REMOTE_IP

and using the ssh myremoteuser@REMOTE_IP command, it asked for the passphrase and connected without problems, also my remote user was able to run docker commands.

The problem was when I tried to connect using a docker context created with:

docker context create \
    --docker host=ssh://myremoteuser@REMOTE_IP \
    --description="Remote engine at REMOTE_IP" \
    my-remote-engine

then I selected the context with docker context use my-remote-engine and when I tried to run docker info, I got the Permission denied error, but it did not ask for the passphrase.

So, it seems that docker does not ask for the passphrase and we need to use an ssh-agent to load the key and docker can use it. That will be done with the command:

ssh-add

just doing that let me run the docker info command without problems, additionally to persist this behavior we need to add the following lines to our ssh config file ~/.ssh/config

Host *
   AddKeysToAgent yes
   UseKeychain yes

More details at:

3 Likes