Docker Community Forums

Share and learn in the Docker community.

Docker EE with UCP in a Linux VM in Azure - /etc/kubernetes/azure.json file missing

enterprise
azure
ucp

(Rabbyte) #1

Hi everybody,

I’ve been trying to install Docker Enterprise Edition (trial subscription) with UCP in a Linux VM (Ubuntu 18.04) in Azure.

I’ve followed this posts:

To install Docker EE

To install UCP on Azure

And here comes the problems.
The Docker EE installed just fine, but when I try to install the UCP i get an error saying the azure.json file is missing.

Here’s the output:

docker container run --rm -it
–name ucp
-v /var/run/docker.sock:/var/run/docker.sock
docker/ucp:3.1.0 install
–host-address 172.16.96.4
–interactive
–swarm-port 3376
–pod-cidr 172.16.96.0/24
–cloud-provider Azure

WARN[0034] None of the hostnames we’ll be using in the UCP certificates [UCP1 127.0.0.1 172.17.0.1 172.16.96.4] contain a domain component. Your generated certs may fail TLS validation unless you only use one of these shortnames or IPs to connect. You can use the --san flag to add more aliases
You may enter additional aliases (SANs) now or press enter to proceed with the above list.
Additional aliases: ucp1.docker.lab
INFO[0000] Using value of --cloud-provider flag instead of the value specified in the config
INFO[0025] Installing UCP with host address 172.16.96.4 - If this is incorrect, please specify an alternative address with the ‘–host-address’ flag
INFO[0025] Deploying UCP Service…
ERRO[0102] Unable to successfully setup local node. Run “docker logs ucp-reconcile” for more details
FATA[0102] reconcile exited with non-zero status: 1

Then I toke a look at the logs:

dockeradmin@UCP1:~$ docker logs ucp-reconcile

{"level":"info","msg":"Configuring node as agent with the following SANs: [kubernetes.default.svc.cluster.local ucp-controller.kube-system.svc.cluster.local proxy.local b7 xrmxd031r21713u0rjbw34p kubernetes 127.0.0.1 172.16.96.4 localhost kubernetes.default kubernetes.default.svc ucp1 172.17.0.1 dckr.poc 10.96.0.1 compose-api.kube-system.svc ucp1.docker.lab]","time":"2018-12-10T17:14:28Z"}

{"level":"info","msg":"Reconciling state of component Docker Proxy","time":"2018-12-10T17:14:28Z"}

{"level":"info","msg":"Reconciling state of component Certificates","time":"2018-12-10T17:14:28Z"}

{"level":"info","msg":"Reconciling state of component Concurrent [Client CA Cluster CA Analytics Kubelet Kubernetes Proxy legacymetrics Concurrent [ucp-agent-service ucp-a gent-win-service ucp-agent-s390x-service] interlockservice [etcd Exclusive RethinkDB Concurrent [eNZi Secret Kubernetes API Server] Concurrent [Swarm-Classic Manager Concu rrent [eNZi API x86_64 service eNZi API s390x service] Concurrent [eNZi Worker x86_64 service eNZi Worker s390x service] Kubernetes Scheduler Kubernetes Controller Manager ]]]","time":"2018-12-10T17:14:28Z"}

{"level":"info","msg":"Reconciling components [etcd Exclusive RethinkDB Concurrent [eNZi Secret Kubernetes API Server] Concurrent [Swarm-Classic Manager Concurrent [eNZi A PI x86_64 service eNZi API s390x service] Concurrent [eNZi Worker x86_64 service eNZi Worker s390x service] Kubernetes Scheduler Kubernetes Controller Manager]]","time":"2 018-12-10T17:14:28Z"}

{"level":"info","msg":"Reconciling state of component etcd","time":"2018-12-10T17:14:28Z"}

{"level":"info","msg":"Reconciling state of component Exclusive RethinkDB","time":"2018-12-10T17:14:28Z"}

{"level":"info","msg":"Reconciling state of component Concurrent [eNZi Secret Kubernetes API Server]","time":"2018-12-10T17:14:28Z"}

{"level":"info","msg":"Container ucp-kube-apiserver is desired to be running but is not running","time":"2018-12-10T17:14:28Z"}

{"level":"info","msg":"successfully reconciled state of Client CA component","time":"2018-12-10T17:14:28Z"}

{"level":"info","msg":"successfully reconciled state of Cluster CA component","time":"2018-12-10T17:14:28Z"}

{"level":"info","msg":"Container ucp-kubelet is desired to be running but is not running","time":"2018-12-10T17:14:28Z"}

{"level":"info","msg":"successfully reconciled state of eNZi Secret component","time":"2018-12-10T17:14:28Z"}

{"level":"info","msg":"successfully reconciled state of Kubernetes Proxy component","time":"2018-12-10T17:14:28Z"}

{"level":"info","msg":"successfully reconciled state of legacymetrics component","time":"2018-12-10T17:14:28Z"}

{"level":"info","msg":"successfully reconciled state of ucp-agent-s390x-service component. This component will enable UCP on s390x linux nodes if they are added to the clu ster","time":"2018-12-10T17:14:28Z"}

{"level":"info","msg":"successfully reconciled state of interlockservice component. This component will enable the interlock load balancing solution on the UCP cluster."," time":"2018-12-10T17:14:28Z"}

{"level":"info","msg":"successfully reconciled state of ucp-agent-service component. This component will enable UCP on x86_64 linux nodes if they are added to the cluster" ,"time":"2018-12-10T17:14:28Z"}

{"level":"info","msg":"successfully reconciled state of ucp-agent-win-service component. This component will enable UCP on x86_64 windows nodes if they are added to the cl uster","time":"2018-12-10T17:14:28Z"}

{"level":"info","msg":"successfully reconciled state of Concurrent [ucp-agent-service ucp-agent-win-service ucp-agent-s390x-service] component","time":"2018-12-10T17:14:28 Z"}

{"level":"info","msg":"Starting up ucp-kube-apiserver container","time":"2018-12-10T17:14:28Z"}

{"level":"warning","msg":"Error deploying ucp-kube-apiserver: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /etc/kub ernetes/azure.json","time":"2018-12-10T17:14:28Z"}

{"level":"error","msg":"unable to reconcile state of Kubernetes API Server component: could not deploy the kube api server. Most likely, you are missing the /etc/kubernete s/azure.json file (Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /etc/kubernetes/azure.json)","time":"2018-12-10T17: 14:28Z"}

{"level":"error","msg":"unable to reconcile state of [etcd Exclusive RethinkDB Concurrent [eNZi Secret Kubernetes API Server] Concurrent [Swarm-Classic Manager Concurrent [eNZi API x86_64 service eNZi API s390x service] Concurrent [eNZi Worker x86_64 service eNZi Worker s390x service] Kubernetes Scheduler Kubernetes Controller Manager]] com ponent: unable to reconcile state of Concurrent [eNZi Secret Kubernetes API Server] component: unable to reconcile state of Kubernetes API Server component: could not depl oy the kube api server. Most likely, you are missing the /etc/kubernetes/azure.json file (Error response from daemon: invalid mount config for type "bind": bind source p ath does not exist: /etc/kubernetes/azure.json)","time":"2018-12-10T17:14:28Z"}

{"level":"info","msg":"successfully reconciled state of Analytics component","time":"2018-12-10T17:14:28Z"}

{"level":"info","msg":"systemd-resolved is running, so using --resolv-conf=/run/systemd/resolve/resolv.conf for kubelet","time":"2018-12-10T17:14:29Z"}

{"level":"info","msg":"Starting up ucp-kubelet container","time":"2018-12-10T17:14:29Z"}

{"level":"warning","msg":"Error deploying ucp-kubelet: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /etc/kubernetes /azure.json","time":"2018-12-10T17:14:29Z"}

{"level":"error","msg":"unable to reconcile state of Kubelet component: could not deploy the kubelet. Most likely, you are missing the /etc/kubernetes/azure.json file (Err or response from daemon: invalid mount config for type "bind": bind source path does not exist: /etc/kubernetes/azure.json)","time":"2018-12-10T17:14:29Z"}

{"level":"fatal","msg":"unable to reconcile state of Concurrent [Client CA Cluster CA Analytics Kubelet Kubernetes Proxy legacymetrics Concurrent [ucp-agent-service ucp-ag ent-win-service ucp-agent-s390x-service] interlockservice [etcd Exclusive RethinkDB Concurrent [eNZi Secret Kubernetes API Server] Concurrent [Swarm-Classic Manager Concur rent [eNZi API x86_64 service eNZi API s390x service] Concurrent [eNZi Worker x86_64 service eNZi Worker s390x service] Kubernetes Scheduler Kubernetes Controller Manager] ]] component: unable to reconcile state of [etcd Exclusive RethinkDB Concurrent [eNZi Secret Kubernetes API Server] Concurrent [Swarm-Classic Manager Concurrent [eNZi API x86_64 service eNZi API s390x service] Concurrent [eNZi Worker x86_64 service eNZi Worker s390x service] Kubernetes Scheduler Kubernetes Controller Manager]] component: un able to reconcile state of Concurrent [eNZi Secret Kubernetes API Server] component: unable to reconcile state of Kubernetes API Server component: could not deploy the kub e api server. Most likely, you are missing the /etc/kubernetes/azure.json file (Error response from daemon: invalid mount config for type "bind": bind source path does n ot exist: /etc/kubernetes/azure.json)","time":"2018-12-10T17:14:29Z"}

bind source path does not exist: /etc/kubernetes /azure.json

Any idea on what it’s happening? Why is it asking for a kubernetes files if I’ve never used it?
Has anyone succesfully installed Docker EE and UCP on Linux in Azure?

Thanks in advanced,

Arnau Valls Cunillera


(Hleclerc) #2

Hello,

Look at : https://github.com/docker/docker.github.io/blob/master/ee/ucp/admin/install/install-on-azure.md
installation of UCP 3.1 is different from previous version (k8s part). You have to deploy the file /etc/kubernetes/azure.json on each nodes of the ucp cluster with the correct parameters to have an automatic ipam pod management.

Regards


(Rabbyte) #3

Hi herveleclerc,

Thank you for the link!
That helped me move forward and solve the missing file issue :slight_smile:

Now I’m stuck with the following:

$ docker container run --rm -it --name ucp -v /var/run/docker.sock:/var/run/docker.sock docker/ucp:3.1.0 install --host-address 10.0.0.4 --interactive --swarm-port 3376 --pod-cidr 10.0.0.0/16 --cloud-provider Azure

INFO[0000] Your engine version 18.09.0, build 33a45cd (4.15.0-1035-azure) is compatible with UCP 3.1.0 (7196b47)
Admin Username: dockadmin
Admin Password:
Confirm Admin Password:
WARN[0010] None of the hostnames we’ll be using in the UCP certificates [docker-enterprise 127.0.0.1 172.17.0.1 10.0.0.4] contain a domain component. Your generated certs may fail TLS validation unless you only use one of these shortnames or IPs to connect. You can use the --san flag to add more aliases

You may enter additional aliases (SANs) now or press enter to proceed with the above list.
Additional aliases: azure.docker.lab
INFO[0000] Using value of --swarm-port flag instead of the value specified in the config
INFO[0000] Using value of --pod-cidr flag instead of the value specified in the config
INFO[0000] Using value of --cloud-provider flag instead of the value specified in the config
INFO[0013] Installing UCP with host address 10.0.0.4 - If this is incorrect, please specify an alternative address with the ‘–host-address’ flag
INFO[0013] Deploying UCP Service…
ERRO[0675] Unable to successfully setup local node. Run “docker logs ucp-reconcile” for more details
FATA[0675] reconcile exited with non-zero status: 1

If we take a look a the logs:

$ docker logs ucp-reconcile
{“level”:“info”,“msg”:“Configuring node as agent with the following SANs: [10.0.0.4 ucp-controller.kube-system.svc.cluster.local kk6076305u1wgahifoxypy9wp compose-api.kube-system.svc 10.96.0.1 127.0.0.1 kubernetes.default.svc localhost proxy.local docker-enterprise azure.docker.lab kubernetes 172.17.0.1 kubernetes.default kubernetes.default.svc.cluster.local]”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“Reconciling state of component Docker Proxy”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“Reconciling state of component Certificates”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“Reconciling state of component Concurrent [Client CA Cluster CA Analytics Kubelet Kubernetes Proxy legacymetrics Concurrent [ucp-agent-service ucp-agent-win-service ucp-agent-s390x-service] interlockservice [etcd Exclusive RethinkDB Concurrent [eNZi Secret Kubernetes API Server] Concurrent [Swarm-Classic Manager Concurrent [eNZi API x86_64 service eNZi API s390x service] Concurrent [eNZi Worker x86_64 service eNZi Worker s390x service] Kubernetes Scheduler Kubernetes Controller Manager]]]”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of Kubelet component”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of Client CA component”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of Cluster CA component”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of legacymetrics component”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of Kubernetes Proxy component”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of ucp-agent-s390x-service component. This component will enable UCP on s390x linux nodes if they are added to the cluster”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of ucp-agent-service component. This component will enable UCP on x86_64 linux nodes if they are added to the cluster”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of ucp-agent-win-service component. This component will enable UCP on x86_64 windows nodes if they are added to the cluster”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of Concurrent [ucp-agent-service ucp-agent-win-service ucp-agent-s390x-service] component”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of interlockservice component. This component will enable the interlock load balancing solution on the UCP cluster.”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“Reconciling components [etcd Exclusive RethinkDB Concurrent [eNZi Secret Kubernetes API Server] Concurrent [Swarm-Classic Manager Concurrent [eNZi API x86_64 service eNZi API s390x service] Concurrent [eNZi Worker x86_64 service eNZi Worker s390x service] Kubernetes Scheduler Kubernetes Controller Manager]]”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“Reconciling state of component etcd”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“Reconciling state of component Exclusive RethinkDB”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“Reconciling state of component Concurrent [eNZi Secret Kubernetes API Server]”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of Kubernetes API Server component”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of eNZi Secret component”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“Reconciling state of component Concurrent [Swarm-Classic Manager Concurrent [eNZi API x86_64 service eNZi API s390x service] Concurrent [eNZi Worker x86_64 service eNZi Worker s390x service] Kubernetes Scheduler Kubernetes Controller Manager]”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of Kubernetes Controller Manager component”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of Swarm-Classic Manager component”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of eNZi API s390x service component. This component will enable eNZi API servers on s390x linux nodes if they are added to the cluster”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of Analytics component”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of eNZi API x86_64 service component. This component will enable eNZi API servers on x86_64 linux nodes if they are added to the cluster”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of eNZi Worker s390x service component. This component will enable eNZi workers on s390x linux nodes if they are added to the cluster”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of Concurrent [eNZi API x86_64 service eNZi API s390x service] component”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of Kubernetes Scheduler component”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of eNZi Worker x86_64 service component. This component will enable eNZi workers on x86_64 linux nodes if they are added to the cluster”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of Concurrent [eNZi Worker x86_64 service eNZi Worker s390x service] component”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“successfully reconciled state of [etcd Exclusive RethinkDB Concurrent [eNZi Secret Kubernetes API Server] Concurrent [Swarm-Classic Manager Concurrent [eNZi API x86_64 service eNZi API s390x service] Concurrent [eNZi Worker x86_64 service eNZi Worker s390x service] Kubernetes Scheduler Kubernetes Controller Manager]] component”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“Reconciling state of component UCP Controller”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“Reconciling state of component Kubernetes CNI Plugin”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“Deploying addon calico”,“time”:“2018-12-13T10:46:04Z”}
{“level”:“info”,“msg”:“Waiting for kubernetes node docker-enterprise to become ready”,“time”:“2018-12-13T10:46:09Z”}

It’s stucked at "Waiting for kubernetes node docker-enterprise to become ready"

docker-enterprise is the name of the machine.


(Rabbyte) #4

If anyone is having the same problem, here’s is the SOLUTION:

https://success.docker.com/article/kubernetes-unable-to-register-node-in-ucp

As my azure vm is located in West Europe, my azure.json file contains:

location:“West Europe”

but spaces are not allowed, instead type:

location:“westeurope”