Insecure, private v2 registry not working with 1.9.x engine?

Goal: To have a private, insecure v2 registry available internally for the development of a POC.

Overview:
I have an instance of Ubuntu 14.04 LTS Server tipped up in an OpenStack private cloud. Docker has been installed and is verified running version 1.9.0.

Following the documentation at https://docs.docker.com/registry/insecure/ I have added the following line in /etc/default/docker:

DOCKER_OPTS="--insecure-registry localhost:5000"

The docker service is then restarted with:

$ service docker stop && service docker start

This alone, so far as I can determine, does not start any registry. As verified by:

root@docker-registry-0:/etc/default# curl --include --request GET http://localhost:5000/v2/
curl: (7) Failed to connect to localhost port 5000: Connection refused

Looking at other bits of the docs, I can start up a registry as a container:

root@docker-registry-0:/etc/default# docker run -d -p 5000:5000 --restart=always --name registry registry:2
0f3b177c79fc90725db7486c4076556e7596dc65ae3caf3e12b6f154b1c46e68
root@docker-registry-0:/etc/default# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
0f3b177c79fc        registry:2          "/bin/registry /etc/d"   4 seconds ago       Up 4 seconds        0.0.0.0:5000->5000/tcp   registry
root@docker-registry-0:/etc/default# curl --include --request GET http://localhost:5000/v2/
HTTP/1.1 200 OK
...
Docker-Distribution-Api-Version: registry/2.0

This allows me to pull, tag, & push local on the server where this is all running. However, what I need is to be able to do is pull and push from another server on our network. With port 5000 verified open, trying to access the registry always results in the same error:

user@machina:~/dev/play/template$ docker push 172.22.0.126:5000/myubuntu
The push refers to a repository [172.22.0.126:5000/myubuntu] (len: 1)
unable to ping registry endpoint https://172.22.0.126:5000/v0/
v2 ping attempt failed with error: Get https://172.22.0.126:5000/v2/: tls: oversized record received with length 20527
v1 ping attempt failed with error: Get https://172.22.0.126:5000/v1/_ping: tls: oversized record received with length 20527

Which is better than when the registry is not running at all which results in:

user@machina:~/dev/play/pi-template$ docker push 172.22.0.126:5000/myubuntu
The push refers to a repository [172.22.0.126:5000/myubuntu] (len: 1)
unable to ping registry endpoint https://172.22.0.126:5000/v0/
v2 ping attempt failed with error: Get https://172.22.0.126:5000/v2/: dial tcp 172.22.0.126:5000: connection refused
v1 ping attempt failed with error: Get https://172.22.0.126:5000/v1/_ping: dial tcp 172.22.0.126:5000: connection refused

I have tried a variety of combinations in both /etc/default/docker and /etc/init.d/docker [exclusive] and have have had no success in getting a insecure registry to run when the daemon starts up. After changes to /etc/defaults/docker the docker service was stopped and started. After changes to /etc/init.d/docker the box was restarted. Some of combinations tried are:

#DOCKER_OPTS="--insecure-registry localhost:5000"
#DOCKER_OPTS="--insecure-registry=localhost:5000"
#DOCKER_OPTS="--insecure-registry 172.22.0.126:5000"
#DOCKER_OPTS="--insecure-registry=172.22.0.126:5000"
#DOCKER_OPTS="--insecure-registry 172.22.0.126:5000"
#DOCKER_OPTS="--insecure-registry=172.22.0.126:5000"
#DOCKER_OPTS="--insecure-registry=172.22.0.126"
#DOCKER_OPTS="--insecure-registry 172.22.0.126"
#DOCKER_OPTS="--insecure-registry=http://172.22.0.126:5000"
#DOCKER_OPTS="--insecure-registry http://172.22.0.126:5000"

…and likely a few other transient combinations that didn’t make the list of commented out values.

So I am hoping I am missing something simple to get an insecure private registry up and visible using docker 1.9.0 and registry v2. Thoughts or links to the right docs would be appreciated. Again my hope is to put together a POC for an on prem dockerized SDLC. I do not wish to violate the will of the community and use an insecure registry once the concepts are proven. Thanks much.

I’m real new to docker but just went through this exercise so if i’m following you I think you need to create your own certs so that you can access the server remotely.


"Running a domain registry

While running on localhost has its uses, most people
want their registry to be more widely available. To do so, the Docker
engine requires you to secure it using TLS, which is conceptually very
similar to configuring your web server with SSL. "

follow the rest of the directions and should be fine.

So your command should look like this:
docker run -d -p 5000:5000 --restart=always --name yourreg
-v pwd/data:/var/lib/registry
-v pwd/certs:/certs
-e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/yourdomain.crt
-e REGISTRY_HTTP_TLS_KEY=/certs/yourdomain.key
registry:2

So are we theorizing that an insecure registry cannot work beyond localhost? It would be great if there was a definitive answer to this question as the docs use broad strokes and leave a few gaps. If a private registry must be secured with TLS to be accessed from outside, so be it. But since I will likely create and destroy dozens of these it would be nice to skip those steps if it is possible.

Thanks for your input.

Brad,
I’m not theorizing at least I don’t think so lol. What I have above is a cut and paste from that link, it’s not my words but dockers. My read of is it that a private registry must have TLS running thus certs. I did it as per directions and it worked.

Gotcha - it’s just that the focus of my question is the creation of an insecure registry. We will need a secure registry at some point and it is good to know that this is simple, but that is not my current use case. So what I am trying to determine is:

  1. Can an insecure registry be created beyond local host?
  2. If so, how?

It is again worth noting that I would like to do this on ephemeral nodes that won’t have a domain configured and I’d like to access the registry with the IP of box.

I suppose the other question is whether my case is even appropriate beyond very initial experimentation and POC work? I will admit that this is entirely possible as my newness to Docker is likely very evident.

Thanks.

Brad,
I’m very new to docker also but to answer your questions to the best of my ability here.

  1. No. an insecure registry is only accessible from local host. You need to set-up TLS/Certs to access beyond local host. It’s not a big deal for experimentation just generate self signed certs.
  2. Can’t be done to the best of my knowledge.

Now this is just a thought and not sure it would work. But if you don’t really care about security and running TLS properly then why don’t you create a dokerfile that follows the steps outlined to generate self signed certs and make a secure registry? This way you could configure it like I did but in the dockerfile and not on the command line. It’s not the right way but would probably get you around this issue. Now that the containers would be ephemeral I guess you have a plan on how to have clients connect to those containers. I saw this that might help you create that virtual domain if that’s how you where planning to do it: http://tech.mybuilder.com/virtual-hosts-with-docker/

v/r,
Frank

Word. OK, going TLS and here is how I manually put this together to get a registry up on a node. Perhaps this will help others as there was one gotcha that let me say I learned something (else) today.

OK, on the registry host - 14:04 LTS

  1. Follow the directions for a standard Docker Engine install.

(Now, since this is just a node (ie ephemeral) w/o a domain - before we create our self signed certs, you will need to modify your /etc/ssl/openssl.cnf file. This step is for people who just want to hit a registry on the IP of the box.)

  1. In /etc/ssl/openssl.cnf find section [v3_ca] and add the following line:
    subjectAltName = IP:[YOUR_REG_BOX_IP_ADDR]
    Example:
    subjectAltName = IP:175.24.12.27

If you do not do this, when you push to your registry you will see an error like:

v2 ping attempt failed with error: Get https://175.23.12.27:5000/v2/: x509: cannot validate certificate for 175.23.12.27 because it doesn't contain any IP SANs

Moving on…

  1. Create your cert:

    $ openssl req -newkey rsa:4096 -nodes -sha256 -keyout certs/domain.key -x509 -days 365 -out certs/domain.crt


Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:Minnesota
Locality Name (eg, city) []:Minneapolis
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Foo
Organizational Unit Name (eg, section) []:RND
Common Name (e.g. server FQDN or YOUR name) []:175.24.12.27
Email Address []:me@myself.andi

  1. Secure copy your cert/domain.crt file to any Docker Engine you want to use with this registry and copy the file to:

    /etc/docker/certs.d/175.24.12.27:5000/ca.crt
    Example:
    $ cp certs/domain.crt /etc/docker/certs.d/175.24.12.27:5000/ca.crt

Doing this should eliminate the pesky:
v2 ping attempt failed with error: Get https://175.23.12.27:5000/v2/: x509: certificate signed by unknown authority

  1. Restart docker on registry server and all engines.

  2. Start your registry, perhaps like:
    $ docker run -d -p 5000:5000 --restart=always --name registry -v pwd/certs:/certs -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key registry:2

  3. Test
    $ docker pull ubuntu
    $ docker tag ubuntu 175.24.12.27:5000/myubuntu
    $ docker push 175.24.12.27:5000/myubuntu
    $ docker pull 175.24.12.27:5000/myubuntu

Everything should work swimmingly:
(from my local client)
bkarels@ubuntu:~$ docker push 175.24.12.27:5000/myubuntu
The push refers to a repository [175.24.12.27:5000/myubuntu] (len: 1)
e9ae3c220b23: Pushed
a6785352b25c: Pushed
0998bf8fb9e9: Pushed
0a85502c06c9: Pushed
latest: digest: sha256:a04b913296a0bae19dae62f85a3cd531c752e7a655d15090ff240bd64ba98cbe size: 6776

So there’s what I know/learned - hope it helps someone. Of course open to better, more simple, faster solutions.

Thanks all.

1 Like

Hi. Forgive me for my bad english.

I’m trying to find a way to resolve the “x509: certificate signed by unknown authority” error.
I have started a new registry server using a self signed certificates:
#!/bin/bash

CONT_NAME="registry"

sudo docker run -d -p 443:5000 --restart=always --name $CONT_NAME \
  -v /opt/$CONT_NAME/certs:/certs \
  -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/docker-registry.test.local.crt \
  -e REGISTRY_HTTP_TLS_KEY=/certs/docker-registry.test.local.key \
  registry:2

If I test using curl, it-s ok:
# curl --cacert ./ca.crt https://docker-registry.test.local/v2/
{}[root

Now I try to access to that registry using docker on same server after placing the certificate in :
# ll /etc/docker/certs.d/docker-registry.test.local.crt/
total 4
-rw------- 1 root root 2269 Jan 11 12:19 ca.crt

My version:
# docker version
Client:
Version: 1.9.1
API version: 1.21
Go version: go1.4.2
Git commit: a34a1d5
Built: Fri Nov 20 13:25:01 UTC 2015
OS/Arch: linux/amd64

Server:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.4.2
 Git commit:   a34a1d5
 Built:        Fri Nov 20 13:25:01 UTC 2015
 OS/Arch:      linux/amd64

But the error persists. Any idea ?

Ok it was just a mistake when I created the directory in /etc:

I moved /etc/docker/certs.d/docker-registry.test.local.crt to /etc/docker/certs.d/docker-registry.test.local
and it’s working now.

I am seeing the below error on Docker version 1.9.1, build a34a1d5

docker pull xx.xxx.xx.x:8080/image_name:latest
Error response from daemon: unable to ping registry endpoint https://xx.xxx.xx.x:8080/v0/
v2 ping attempt failed with error: Get https://xx.xxx.xx.x:8080/v2/: EOF

I have added the below line in /etc/sysconfig/docker
INSECURE_REGISTRY=’–insecure-registry xx.xxx.xx.x:8080’

Restarted docker but still not able to pull the images. Any thoughts?

I have the same issue and don’t know if anyone fixes it. :frowning:

Hi, I followed your steps line by line. But I am seeing the below errors :frowning:
[root@testbox certs.d]# docker push 139.23.49.195:5000/mynginx
The push refers to a repository [139.23.49.195:5000/mynginx]
Get https://139.23.49.195:5000/v1/_ping: x509: cannot validate certificate for 139.23.49.195 because it doesn’t contain any IP SANs
I also tried with the hostname. But encountered a different error.
[root@testbox certs.d]# docker tag nginx testbox.mydomain. com:5000/mynginx
[root@testbox certs.d]# docker push testbox.mydomain. com:5000/mynginx
The push refers to a repository [testbox.mydomain. com : 5000/mynginx]
Get https :// testbox.mydomain. com : 5000/v1/_ping: x509: certificate signed by unknown authority
I have added respective folders and placed the ca.crt file under /etc/docker/certs.d folder.
Please let me know if this is a known issue in 17.03.1-ce.

Hi @bradkarels, I followed your steps and I got the same error : Get https://129.X.X.X:5000/v1/_ping: x509: cannot validate certificate for 129.X.X.X because it doesn’t contain any IP SANs. I checked selinux and found it to be in enforcing mode. # sestatus
SELinux status: enabled
SELinuxfs mount: /selinux
Current mode: enforcing
Mode from config file: enforcing
Policy version: 29
Policy from config file: targeted

Ran the command # setenforce 0

Tested docker push and it worked for me. Just adding this note here so that it helps someone in future :slight_smile:

Hi @cbananth, I also ran into the same issue after following the exact steps from Brad. I was able to fix it locally (push/pull works locally on docker host) with setting selinux to permissive on my docker host. However, when I tried pushing the image from a remote machine to my registry, I ran into this " cannot validate certificate for X.X.X.X because it doesn’t contain any IP SANs:

Checking this further I nailed down the problem. In my case this was what happened.

  1. I generated a self signed cert and on verifying the cert, I found that it did not have Subject Alternative Name:
    IP Address:X.X.X.X in it.

Steps I did to resolve the issue:

  1. Added the line subjectAltName=IP[YOUR_REG_BOX_IP_ADDR] in openssl.cnf under the section [v3_ca] as Brad stated above.

  2. Generated Self Signed cert using this config file -

openssl req -x509 -newkey rsa:4096 -sha256 -nodes -keyout certs/instance1.key -out certs/instance1.crt -days 365 -config /etc/pki/tls/openssl.cnf`< add the correct path of your openssl.cnf based on your distro>

  1. Verify subjectAltName is seen in your newly generated cert using the command -

#openssl x509 -in certs/instance1.crt -text -noout|less

You’d see something like -

X509v3 extensions:
X509v3 Subject Alternative Name:
IP Address: <[YOUR_REG_BOX_IP_ADDR]>

  1. Run the commands - #update-ca-trust enable and #service docker restart

  2. Start a registry container using the new certs that you’ve generated

  3. On the remote machine, create -

mkdir -p /etc/docker/certs.d/[YOUR_REG_BOX_IP_ADDR]:5000
cp /ca.crt /etc/docker/certs.d/[YOUR_REG_BOX_IP_ADDR]:5000/ca.crt
update-ca-trust enable
service docker restart

Test push /pull to the registry from your remote machine. This worked for my case :):smiley: