Docker Beta not visible to docker-machine

Expected behavior

I’d expect to do a docker-machine ls and see the newly-installed docker machine.

Actual behavior

Using either the app-installed installed CLI version or the brew installed docker-machine cli, I get only the machines that previously existed, and not the fancy new xhyve host:

$ /Applications/Docker.app/Contents/Resources/bin/docker-machine ls
NAME            ACTIVE   DRIVER       STATE     URL                                 SWARM   DOCKER    ERRORS
dev             -        virtualbox   Stopped                                               Unknown   
devdepot        -        generic      Running   tcp://XXXX:2376           Unknown   Unable to query docker version: Cannot connect to the docker engine endpoint
docker-machine ls
NAME       ACTIVE   DRIVER       STATE     URL                                 SWARM   DOCKER    ERRORS
dev        -        virtualbox   Stopped                                               Unknown   
devdepot   -        generic      Running   tcp://XXXX:2376           Unknown   Unable to query docker version: Cannot connect to the docker engine endpoint
$ which docker-machine
/usr/local/bin/docker-machine
$ readlink /usr/local/bin/docker-machine
/Users/XXXX/Library/Group Containers/group.com.docker/bin/docker-machine

Information

Docker’s pinata is apparently bugged as well:

$ `pinata diagnose -u` 
-bash: OS: command not found

But I can see the docker processes running:

/Applications/Docker.app/Contents/MacOS/com.docker.hyperkit

OSX 10.10.5, docker OSX beta version:
Version 1.11.1-beta13.1 (build: 8193)
e9f58bbe60f0debccc097b7d27cad200556ec6f5

Hi @perilousapricot,

Please take a moment and review Docker for Mac vs. Docker Toolbox:

I think it might be a good refresher for you. Including these paragraphs:

  • Installing Docker for Mac does not affect machines you created with Docker Machine. The install offers to copy containers and images from your local default machine (if one exists) to the new Docker for Mac xhyve VM. If chosen, content from default is copied to the new Docker for Mac xhyve VM, and your original default machine is kept as is.
  • The Docker for Mac application does not use docker-machine to provision that VM; but rather creates and manages it directly.

I hope this helps,
Alexandre

I understand that’s how it’s implemented, but the “expected behavior” is what I … “expect” to happen.

Especially for a product that’s considered to be “beta”, I don’t (and I’m sure others) want to take the plunge and move my whole workflow over to something that’s not considered solid yet.

Unless the docker toolbox will be forever deprecated, having Docker for Mac be treated differently is un-orthogonal and confusing.

That’s how it’s documented, therefore it is the expected behavior :wink: If it was undocumented or if the documentation said the opposite, then you could rightly expect it to work otherwise.

My two cents here: If you are not ready to change your workflow, especially since things might change and break in the upcoming months, that is totally acceptable and I fully understand that. However, if such is your position, maybe you should keep your automation far from Docker Beta for the moment. A Beta is usually contains all the features of the target release, but is prone to contains bugs or undefined behavior that will be fix as the beta progress. This means some things might work differently between the beta and the release (an example: the /Mac mount won’t be available on release)

It is Docker goal to provide the same user experience on Linux, MacOS and Windows. docker-machine, while necessary, was an obstacle to that goal. It is not Docker for Mac that behaves differently (it is very close to its Linux counterpart), it was the docker-machine that treated Mac user differently.

The tool existed in order to allow Mac and Windows users to install and run Docker. If the tool is not required anymore, it can and should be discard (in my opinion) in order to put the emphasis on the native Docker client. I believe I saw someone from Docker saying something along those lines. If I can find it, I’ll let you know so you can know their (and not my) position on the matter.

1 Like

We could argue semantics all day about what “expected” means, it is the internet, afterall.

The developers (or at least the one on documentation duty) obviously expect something different than what I, as a user, expect. To me, it’s non-orthogonal to have a different set of tooling for one case compared to the others. Maybe it makes more sense from the dev’s side, but from this user’s standpoint, where I could before have “dev” and “prod” machine(s) and treat them identically, that no longer becomes the case. This is a feedback forum for beta software, they can take the suggestion or not.

I understand what it means to run beta software! I’m willing to take the time to run non-real things with the software to try and shake out bugs (I’m a good open-source citizen). As is, however, it’s hard to “dip my toes in”. Like any other major upgrade, I want to slowly edge my way up as I get more experience/confidence.

That’s not been my experience. I have my development laptop and a number of beefy servers. If I work locally, or remotely, running docker-machine gives me (and the rest of my team) a unified interface so I can truly develop small, then scale to larger instances frictionless-ly.

Indeed, we could argue forever. Long live Internet arguments! :wink:

In that regard, I can understand your use case and, yes, it is shared by a lot of people. I know that the Docker staff is aware the the docker-machine is an important tool for many.

I am not using docker-machine myself, but I’ve seen a few threads about just being able to connect the docker-machine to the local machine created by Docker for Mac. I don’t know what the status of those issues.

Regardless of my lack of appreciation for that tool, I would expect (yes!) docker-machine to be able to connect to any machine using the socket file or the API (:2376).

I believe it is fair to ask for this integration. I am unsure how much Docker for Mac and docker-machine have to change in order to allow that.

It’s fun and useful to set up a bunch of VMs on my laptop, particularly to debug cluster deploys.

Since Docker.app includes a docker-machine it would be nice if it had better support. Forgetting about the integration with the docker default VM for the moment, I’d like to be able to use the same xhyve driver and boot2docker image (or equivalent) that Docker.app is using. Any way to do this?

I can install an xhyve driver of my own, e.g. via

brew install docker-machine-driver-xhyve

and then

docker-machine create --driver xhyve foo

will create a VM, however this is not using the same xhyve driver or boot2docker image, which seems like a missed opportunity for debugging.

I think the intent of docker for mac is to appear no different to the user from docker engine on a Linux machine. To that end having the hyve VM not show up in docker-machine makes perfect sense to me.

Whether it should be listed as a docker-machine is entirely a matter of opinion. On a Linux machine I don’t expect the docker host/engine to show up as a docker-machine by default, although it’s entirely possible to point docker-machine to the machine with the generic driver. Why one would do that, I don’t know, but the option is there.

Also note that with a docker-machine provisioned VM or machine (whether remote or local) the interface binding is not to your local interface, it is to that VM/machine’s interface (virtual or otherwise). Thus when a container is exposed through the docker host’s interface, you cannot reach that service via localhost/127.0.0.1. Contrast to that, with Docker for Mac when you expose a container it’s accessible via localhost/127.0.0.1.

So, to me, Docker for Mac behaves much more like Docker would locally on a Linux machine than a VM that’s being managed/provisioned by docker-machine.

1 Like

I, for one, would like to eventually see docker-machine and docker-swarm added to Docker for Mac. I do realize that this is a non-trivial task and that you might not have the resources to implement this anytime soon, but I do want to make the request to try and get it added to the roadmap.

Swarm is part of Docker now as of 1.12, not a standalone tool, and docker-machine has been available for Mac for quite a while now.