Custom DNS for Virtual Machines

Docker Desktop for Windows
1

Issue type: creating virtual machines
OS: Windows 10 Professional 1903 64 bit
App version: Server Version: 18.09.2
Product License: Community Engine

I’m trying to get started with Hyper-V virtural machines and Docker on Windows 10 Professional.

I’m following the Getting Started - Swarms tutorial. All other tutorials in that series have been run successfully.

I’m using a private registry that I’ve created running on an OSS Nexus 3 (3.16.1-02) server. I’ve successfully logged in, pushed, and pulled images from the hosted repository using a self-signed certificate.

My first problem is in creating virtual machines that use my internal DNS server as the name server.

Things I’ve tried:

  1. Change my Windows 10 Professional machine to use the internal DNS server
  2. Change the virtual switch to use the internal DNS server
  3. Change the docker for windows configuration to use the internal DNS server
  4. Create the virtual machine with:

docker-machine create -d hyperv --hyperv-virtual-switch “myswitch” --engine-opt “dns=internal_ip_address” myvm1

docker-machine create -d hyperv --hyperv-virtual-switch “myswitch” --engine-opt dns=internal_ip_address myvm1

However, when I execute the following:

docker-machine ssh myvm1 “cat /etc/resolv.conf”, it always returns with:

search home
nameserver 192.168.0.1

This is my router, and also the default DNS server provided by DHCP. This means that I cannot connect to my private docker registry.

The only way I’ve found to get around this is the following:

docker-machine ssh myvm1
$ sudo -i
# echo "search home" > /etc/resolv.conf
# echo "nameserver internal_ip_address" >> /etc/resolv.conf
# /etc/init.d/docker restart
# exit
$ exit

This is annoying and doesn’t survive reboots.

I still can’t connect with docker login to my private registry, because I haven’t figured out where to put the self-signed certificate. I’ve generated the PEM file, but docker login still thinks it’s not a trusted connection.

I would really like to create virtual machines using the specified DNS server in /etc/resolv.conf.

Pointers would be greatly appreciated.

2

This is going to be long, but hopefully helpful.

This covers how to successfully do the Swarms getting started exercise on Windows 10 Professional with the following conditions.

A. Private secure registry using a self-signed certificate
B. Internal DNS server

Note that this all has to be done from a PowerShell window that has administrator privileges.

This may not be the best way to do it. In particular, setting a non-DHCP provided DNS server doesn’t seem to be optimal. However, this all works.

I will not cover how to set up a private repository with a self-signed certificate here.

After the virtual machines are created, do the following.

  1. Import the self-signed certificate

a. Obtain the .pem file that contains the certificate and key
b. make sure it’s named the same as your private repository host name

for example: repo.mydomain.org.pem for a repository on repo.mydomain.org

c. scp the pem file to the virtual machine

docker-machine scp repo.mydomain.org.pem virtual_machine_name:

Note that on Windows 10 this appears to copy scp.exe

d. SSH to the virtual machine and do some cleanup

docker-machine ssh virtual_machine_name
$ rm scp.exe
$ sudo -i
# mkdir /var/lib/boot2docker/certs
# mv /home/docker/repo.mydomain.org.pem /var/lib/boot2docker/certs/
# exit
$ exit

Note, you could do this by sending commands directly via ssh.

e. restart the virtual machine

docker-machine restart virtual_machine_name

  1. Set up DNS

a. SSH to the virtual machine

docker-machine ssh virtual_machine_name

b. Create the desired resolv.conf file

$ sudo -i
# cd /etc
# echo "search home" > resolv.conf
# echo "nameserver nameserver_ip_address" >> resolv.conf

c. Restart docker

# /etc/init.d/docker restart

d. Leave

# exit
$ exit

Note, DO NOT RESTART the virtual machine. While the certificate will hang around, the changes to /etc/resolv.conf will not.

Now you should be able to do the following:

docker-machine env virtual_machine_name | Invoke-Expression
docker login -u username --password-stdin repo.mydomain.org:port
Password: account_password
Login Succeeded

I really wish that --engine-opt “dns=nameserver_ip_address” worked.