Cannot get Trust Delegation to work (Notary v0.3.0) (SOLVED)

I’m trying to get Content Trust Delegation to work as specified here: Docker Docs but am having some difficulties.

I’m using:

  • Docker version 1.10.2 (commit c3959b1)
  • Notary version v0.3.0 (commit 658a25c) (both client and server)
  • Registry (commit e430d773)
  • completely clean and freshly compiled Notary and Registry servers and Notary client

I’m trying to get Trust delegation working so I can push with Content Trust to the same repo from different clients that share the necessary certificates.

I set up a “pusher” Docker image with a private key imported into Notary thus:

cd /root

openssl genrsa -out delegation.key 2048

openssl req -new -sha256 -key delegation.key -out delegation.csr -subj “/C=XX/ST=XX/L=XX/O=XX/OU=XX/CN=XX”

openssl x509 -req -days 365 -in delegation.csr -signkey delegation.key -out delegation.crt

export NOTARY_DELEGATION_PASSPHRASE=“notary_delegation_passphrase”

notary -v -D -d /root/.docker/trust key import /root/delegation.key --role targets/user

I then created two separate “pusher” Docker containers, each running this image.

The first container is able to push successfully

export DOCKER_CONTENT_TRUST=1

export DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE=targets

export DOCKER_CONTENT_TRUST_ROOT_PASSPHRASE=root

export DOCKER_CONTENT_TRUST_SERVER=https://notary-server:4443

export NOTARY_DELEGATION_PASSPHRASE=notary_delegation_passphrase

export NOTARY_ROOT_PASSPHRASE=root

export NOTARY_SNAPSHOT_PASSPHRASE=targets

export NOTARY_TARGETS_PASSPHRASE=targets

docker tag good_image registry-server:5000/good_image:latest

docker push registry-server:5000/good_image:latest

The push refers to a repository [registry-server:5000/good_image]

latest: digest: sha256:10c47e5524e9ce57d9669ba543907e73b90ec463fd7040779335e280dae4a694 size: 920
Signing and pushing trust metadata
Finished initializing “registry-server:5000/good_image”

notary -s https://notary-server:4443 -d /root/.docker/trust key rotate registry-server:5000/good_image snapshot -r

notary -s https://notary-server:4443 -d /root/.docker/trust delegation add registry-server:5000/good_image targets/releases /root/delegation.crt --all-paths
Addition of delegation role targets/releases with keys [2313c659904e7933be605f657d4177e8f94c27d18298bcef6b83b7671b83ab4a], with paths [“” ], to repository “registry-server:5000/good_image” staged for next publish.

notary -s https://notary-server:4443 -d /root/.docker/trust publish registry-server:5000/good_image
Pushing changes to registry-server:5000/good_image

However, push from the second container fails:

export DOCKER_CONTENT_TRUST=1

export DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE=targets

export DOCKER_CONTENT_TRUST_ROOT_PASSPHRASE=root

export DOCKER_CONTENT_TRUST_SERVER=https://notary-server:4443

export NOTARY_DELEGATION_PASSPHRASE=notary_delegation_passphrase

export NOTARY_ROOT_PASSPHRASE=root

export NOTARY_SNAPSHOT_PASSPHRASE=targets

export NOTARY_TARGETS_PASSPHRASE=targets

docker tag good_image registry-server:5000/good_image:latest

docker -D push registry-server:5000/good_image:latest
The push refers to a repository [registry-server:5000/good_image]
time=“2016-06-02T00:07:30Z” level=debug msg=“reading certificate directory: /root/.docker/tls/notary-server:4443”
latest: digest: sha256:10c47e5524e9ce57d9669ba543907e73b90ec463fd7040779335e280dae4a694 size: 920
Signing and pushing trust metadata
time=“2016-06-02T00:07:31Z” level=debug msg=“Adding cert with certID: e8ca6db6c4f5119a89e2b2f4e8acb1bbf30fd919064e275da38cf2077b477582”
time=“2016-06-02T00:07:31Z” level=debug msg=“Making dir path: /root/.docker/trust/tuf/registry-server:5000/good_image/changelist”
time=“2016-06-02T00:07:31Z” level=debug msg=“Adding target "latest" with sha256 "10c47e5524e9ce57d9669ba543907e73b90ec463fd7040779335e280dae4a694" and size 920 bytes.\n”
time=“2016-06-02T00:07:31Z” level=debug msg=“entered ValidateRoot with dns: registry-server:5000/good_image”
time=“2016-06-02T00:07:31Z” level=debug msg=“found the following root keys: [e8ca6db6c4f5119a89e2b2f4e8acb1bbf30fd919064e275da38cf2077b477582]”
time=“2016-06-02T00:07:31Z” level=debug msg=“found 1 valid leaf certificates for registry-server:5000/good_image”
time=“2016-06-02T00:07:31Z” level=debug msg=“found 1 valid root certificates for registry-server:5000/good_image”
time=“2016-06-02T00:07:31Z” level=debug msg=“entering root certificate rotation for: registry-server:5000/good_image”
time=“2016-06-02T00:07:31Z” level=debug msg=“Adding cert with certID: e8ca6db6c4f5119a89e2b2f4e8acb1bbf30fd919064e275da38cf2077b477582”
time=“2016-06-02T00:07:31Z” level=debug msg=“ignoring certificate addition to: registry-server:5000/good_image”
time=“2016-06-02T00:07:31Z” level=debug msg=“Root validation succeeded for registry-server:5000/good_image”
time=“2016-06-02T00:07:31Z” level=debug msg=“200 when retrieving metadata for root”
time=“2016-06-02T00:07:31Z” level=debug msg=“Adding key 2e39bf5891d74eea14d7b545780134645d08cb68b56be527dc2f7065224cc27a”
time=“2016-06-02T00:07:31Z” level=debug msg=“Adding key 35caeb34dd00363b7b94f8a438ada2ea410a0ce31f9c6133c80521e5cb786f04”
time=“2016-06-02T00:07:31Z” level=debug msg=“Adding key dec3f878777f0ce0e1117a552bd394dfd469c0be8070059dc164516437f1f3e0”
time=“2016-06-02T00:07:31Z” level=debug msg=“Adding key e8ca6db6c4f5119a89e2b2f4e8acb1bbf30fd919064e275da38cf2077b477582”
time=“2016-06-02T00:07:31Z” level=debug msg=“Adding role targets with keys 35caeb34dd00363b7b94f8a438ada2ea410a0ce31f9c6133c80521e5cb786f04”
time=“2016-06-02T00:07:31Z” level=debug msg=“Adding role timestamp with keys dec3f878777f0ce0e1117a552bd394dfd469c0be8070059dc164516437f1f3e0”
time=“2016-06-02T00:07:31Z” level=debug msg=“Adding role root with keys e8ca6db6c4f5119a89e2b2f4e8acb1bbf30fd919064e275da38cf2077b477582”
time=“2016-06-02T00:07:31Z” level=debug msg=“Adding role snapshot with keys 2e39bf5891d74eea14d7b545780134645d08cb68b56be527dc2f7065224cc27a”
time=“2016-06-02T00:07:31Z” level=debug msg=“updating TUF client”
time=“2016-06-02T00:07:31Z” level=debug msg=“Downloading Timestamp…”
time=“2016-06-02T00:07:31Z” level=debug msg=“200 when retrieving metadata for timestamp”
time=“2016-06-02T00:07:31Z” level=debug msg=“timestamp role has key IDs: dec3f878777f0ce0e1117a552bd394dfd469c0be8070059dc164516437f1f3e0”
time=“2016-06-02T00:07:31Z” level=debug msg=“verifying signature for key ID: dec3f878777f0ce0e1117a552bd394dfd469c0be8070059dc164516437f1f3e0”
time=“2016-06-02T00:07:31Z” level=debug msg=“successfully verified timestamp”
time=“2016-06-02T00:07:31Z” level=debug msg=“Downloading Snapshot…”
time=“2016-06-02T00:07:31Z” level=debug msg=“using cached snapshot”
time=“2016-06-02T00:07:31Z” level=debug msg=“snapshot role has key IDs: 2e39bf5891d74eea14d7b545780134645d08cb68b56be527dc2f7065224cc27a”
time=“2016-06-02T00:07:31Z” level=debug msg=“verifying signature for key ID: 2e39bf5891d74eea14d7b545780134645d08cb68b56be527dc2f7065224cc27a”
time=“2016-06-02T00:07:31Z” level=debug msg=“successfully verified snapshot”
time=“2016-06-02T00:07:31Z” level=debug msg=“Downloading Targets…”
time=“2016-06-02T00:07:31Z” level=debug msg=“using cached targets”
time=“2016-06-02T00:07:31Z” level=debug msg=“targets role has key IDs: 35caeb34dd00363b7b94f8a438ada2ea410a0ce31f9c6133c80521e5cb786f04”
time=“2016-06-02T00:07:31Z” level=debug msg=“verifying signature for key ID: 35caeb34dd00363b7b94f8a438ada2ea410a0ce31f9c6133c80521e5cb786f04”
time=“2016-06-02T00:07:31Z” level=debug msg=“successfully verified targets”
Enter passphrase for targets/user key with ID 2313c65:
time=“2016-06-02T00:07:31Z” level=debug msg=“Making dir path: /root/.docker/trust/tuf/registry-server:5000/good_image/changelist”
time=“2016-06-02T00:07:31Z” level=debug msg=“changelist add: latest”
time=“2016-06-02T00:07:31Z” level=error msg=“couldn’t add target to targets/releases: could not find necessary signing keys, at least one of these keys must be available: 6347d8bd80fedcf50ccc14b1f1f62951799585eaf0adaec91dc12fdb81a8bbb6”
time=“2016-06-02T00:07:31Z” level=debug msg=“Error applying changelist”
Error: could not find signing keys for remote repository registry-server:5000/good_image, or could not decrypt signing key: could not find necessary signing keys, at least one of these keys must be available: 6347d8bd80fedcf50ccc14b1f1f62951799585eaf0adaec91dc12fdb81a8bbb6

Can anyone tell me what I’m doing wrong? Thanks!

PS: I tried importing the key with role “user” instead of “targets/user” but still got the same problem.

UPDATE:

It looks as if the notary delegation passphrase is not being read from the environment variable NOTARY_DELEGATION_PASSPHRASE during Docker push. Is there any way to supply this passphrase noninteractively?

SECOND UPDATE:

Looks as if I can supply the passphrase on standard input – seems to be working now.

Hi, @dwake! I am trying to do the exact thing. What kind of certificates did you shared in order to be able to see the notary server on the second machine? I am not able to push signed images from one machine to another because of the notary server service. Thank you in advance!

Hi @lenutatode Can you show the exact commands you ran and error messages you saw?

https://blog.docker.com/2016/03/notary-0-2/ … I wanna do this tutorial. On machine 2 for the second trying of pushing trusted images with DOCKER CONTENT TRUST enabled, I receive: “Error: trust data missing for remote repository mydomain.com:5000/ubuntu or remote repository not found: timestamp key trust data unavailable. Has a notary repository been initialized?” or “error contacting notary server”.

All good now. Thank you!