Modifying the apiserver on Kubernetes on Docker

I’m trying to test out OpenID Connect User authentication on Kubernetes using Docker for Windows (though it should be the same on Docker for Mac) on the edge channel.

In order to enable OpenID Connect auth for Kubernetes, you need to specify arguments on the api server such as --oidc-client-id and --oidc-groups-claim etc. This is trivial if you set up a kubernetes cluster by hand or if you are using minikube, it is done by setting the --extra-config parameters (example).

However, with Kubernetes for Docker on Windows, the apiserver is run in a docker container that is setup by magic as all I can do is tick the button to enable Kubernetes.

Does anyone on here know how the magic works to set up these kube-system docker containers such as the apiserver and how to modify/configure them? Apart from that, the new kubernetes support in Docker-edge is great!

1 Like

I have this question too. I need to use PodPresets which requires attaching extra parameters to the api-server container command. Did you find any solution for this?
I read this in the documentation:

Docker for Mac 17.12 CE Edge includes a standalone Kubernetes server and client, as well as Docker CLI integration. The Kubernetes server runs locally within your Docker instance, is not configurable, and is a single-node cluster.

I didn’t find a decent solution, but I did find a hacky workaround.

found the container running the apiserver in it (you need to tick the box to show kubernetes internal containers when using the docker commands), then did docker exec -it <containerId> bash to exec into it.
From there, I found the process that was running (/usr/lib/bin/kube-apiserver I think), and moved this to kube-apiserver.real).
In it’s place It then created a simple shell script (called kube-apiserver) which calls kube-apiserver.real with the appropriate arguments you want (and then $@ to add all other command line arguments) and chmod’ed it to 755 to make it runnable.

Then I exited the container and restarted it. It’s a dirty hack, but works (if for any reason it destroys the container, you need to do it all again though)

Aha, I see what you did. Probably, when you uncheck the “Enable Kubernetes” the container will be recreated.
Thanks for the workaround.

Thanks for the workaround.
For others hunting Istio’s automatic proxy injection requires similar modification to apiserver admission-control flags.

From the Istio docs [to help the google point people here]:
“verify that the kube-apiserver process has the admission-control flag set with the MutatingAdmissionWebhook and ValidatingAdmissionWebhook admission controllers added and listed in the correct order.”

@Docker - we’re going to need a better way to control/configure this!

You can always connect into the “VM” with:
screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty

change the files in the /etc/kubernetes/manifests folder

and then restart docker. But i think it applies to docker for mac osx only.
With this approach, we were able to enable podpresets.

For Windows, you can access the DockerDesktopVM with this apporach by @ailurarctos:

When you are logged in, just add or edit the kube-apiserver parameters in /etc/kubernetes/manifests/kube-apiserver.yaml