Chapter 3. Setting up the Registry

List the current pods to find the pod name of your Docker registry:

# oc get pods

Then, use oc describe to find the host name for the node running the container:

# oc describe pod <pod_name>

Log into the desired node:

# ssh node.example.com

List the running containers on the node host and identify the container ID for the Docker registry:

# docker ps | grep ose-docker-registry

List the registry contents using the docker exec command:

# docker exec -it 4c01db0b339c find /registry
/registry/docker
/registry/docker/registry
/registry/docker/registry/v2
/registry/docker/registry/v2/blobs 1
/registry/docker/registry/v2/blobs/sha256
/registry/docker/registry/v2/blobs/sha256/ed
/registry/docker/registry/v2/blobs/sha256/ed/ede17b139a271d6b1331ca3d83c648c24f92cece5f89d95ac6c34ce751111810
/registry/docker/registry/v2/blobs/sha256/ed/ede17b139a271d6b1331ca3d83c648c24f92cece5f89d95ac6c34ce751111810/data 2
/registry/docker/registry/v2/blobs/sha256/a3
/registry/docker/registry/v2/blobs/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
/registry/docker/registry/v2/blobs/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4/data
/registry/docker/registry/v2/blobs/sha256/f7
/registry/docker/registry/v2/blobs/sha256/f7/f72a00a23f01987b42cb26f259582bb33502bdb0fcf5011e03c60577c4284845
/registry/docker/registry/v2/blobs/sha256/f7/f72a00a23f01987b42cb26f259582bb33502bdb0fcf5011e03c60577c4284845/data
/registry/docker/registry/v2/repositories 3
/registry/docker/registry/v2/repositories/p1
/registry/docker/registry/v2/repositories/p1/pause 4
/registry/docker/registry/v2/repositories/p1/pause/_manifests
/registry/docker/registry/v2/repositories/p1/pause/_manifests/revisions
/registry/docker/registry/v2/repositories/p1/pause/_manifests/revisions/sha256
/registry/docker/registry/v2/repositories/p1/pause/_manifests/revisions/sha256/e9a2ac6418981897b399d3709f1b4a6d2723cd38a4909215ce2752a5c068b1cf
/registry/docker/registry/v2/repositories/p1/pause/_manifests/revisions/sha256/e9a2ac6418981897b399d3709f1b4a6d2723cd38a4909215ce2752a5c068b1cf/signatures 5
/registry/docker/registry/v2/repositories/p1/pause/_manifests/revisions/sha256/e9a2ac6418981897b399d3709f1b4a6d2723cd38a4909215ce2752a5c068b1cf/signatures/sha256
/registry/docker/registry/v2/repositories/p1/pause/_manifests/revisions/sha256/e9a2ac6418981897b399d3709f1b4a6d2723cd38a4909215ce2752a5c068b1cf/signatures/sha256/ede17b139a271d6b1331ca3d83c648c24f92cece5f89d95ac6c34ce751111810
/registry/docker/registry/v2/repositories/p1/pause/_manifests/revisions/sha256/e9a2ac6418981897b399d3709f1b4a6d2723cd38a4909215ce2752a5c068b1cf/signatures/sha256/ede17b139a271d6b1331ca3d83c648c24f92cece5f89d95ac6c34ce751111810/link 6
/registry/docker/registry/v2/repositories/p1/pause/_uploads 7
/registry/docker/registry/v2/repositories/p1/pause/_layers 8
/registry/docker/registry/v2/repositories/p1/pause/_layers/sha256
/registry/docker/registry/v2/repositories/p1/pause/_layers/sha256/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
/registry/docker/registry/v2/repositories/p1/pause/_layers/sha256/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4/link 9
/registry/docker/registry/v2/repositories/p1/pause/_layers/sha256/f72a00a23f01987b42cb26f259582bb33502bdb0fcf5011e03c60577c4284845
/registry/docker/registry/v2/repositories/p1/pause/_layers/sha256/f72a00a23f01987b42cb26f259582bb33502bdb0fcf5011e03c60577c4284845/link

Fetch the service IP and port of the registry:

$ oc get svc/docker-registry
NAME              LABELS                                    SELECTOR                  IP(S)            PORT(S)
docker-registry   docker-registry=default                   docker-registry=default   172.30.124.220   5000/TCP

You can use an existing server certificate, or create a key and server certificate valid for specified IPs and host names, signed by a specified CA. To create a server certificate for the registry service IP and the docker-registry.default.svc.cluster.local host name:

$ oadm ca create-server-cert \
    --signer-cert=/etc/origin/master/ca.crt \
    --signer-key=/etc/origin/master/ca.key \
    --signer-serial=/etc/origin/master/ca.serial.txt \
    --hostnames='docker-registry.default.svc.cluster.local,docker-registry.default.svc,172.30.124.220' \
    --cert=/etc/secrets/registry.crt \
    --key=/etc/secrets/registry.key

If the router will be exposed externally, add the public route host name in the --hostnames flag:

--hostnames='mydocker-registry.example.com,docker-registry.default.svc.cluster.local,172.30.124.220 \

See Redeploying Registry and Router Certificates for additional details on updating the default certificate so that the route is externally accessible.

Create the secret for the registry certificates:

$ oc secrets new registry-secret \
    /etc/secrets/registry.crt \
    /etc/secrets/registry.key

Add the secret to the registry pod’s service accounts (including the default service account):

$ oc secrets link registry registry-secret
$ oc secrets link default  registry-secret

Add the secret volume to the registry deployment configuration:

$ oc volume dc/docker-registry --add --type=secret \
    --secret-name=registry-secret -m /etc/secrets

Enable TLS by adding the following environment variables to the registry deployment configuration:

$ oc set env dc/docker-registry \
    REGISTRY_HTTP_TLS_CERTIFICATE=/etc/secrets/registry.crt \
    REGISTRY_HTTP_TLS_KEY=/etc/secrets/registry.key

See more details on overriding registry options.

Update the scheme used for the registry’s liveness probe from HTTP to HTTPS:

$ oc patch dc/docker-registry -p '{"spec": {"template": {"spec": {"containers":[{
    "name":"registry",
    "livenessProbe":  {"httpGet": {"scheme":"HTTPS"}}
  }]}}}}'

If your registry was initially deployed on OpenShift Container Platform 3.2 or later, update the scheme used for the registry’s readiness probe from HTTP to HTTPS:

$ oc patch dc/docker-registry -p '{"spec": {"template": {"spec": {"containers":[{
    "name":"registry",
    "readinessProbe":  {"httpGet": {"scheme":"HTTPS"}}
  }]}}}}'

Validate the registry is running in TLS mode. Wait until the latest docker-registry deployment completes and verify the Docker logs for the registry container. You should find an entry for listening on :5000, tls.

$ oc logs dc/docker-registry | grep tls
time="2015-05-27T05:05:53Z" level=info msg="listening on :5000, tls" instance.id=deeba528-c478-41f5-b751-dc48e4935fc2

Copy the CA certificate to the Docker certificates directory. This must be done on all nodes in the cluster:

$ dcertsdir=/etc/docker/certs.d
$ destdir_addr=$dcertsdir/172.30.124.220:5000
$ destdir_name=$dcertsdir/docker-registry.default.svc.cluster.local:5000

$ sudo mkdir -p $destdir_addr $destdir_name
$ sudo cp ca.crt $destdir_addr    1
$ sudo cp ca.crt $destdir_name

When using authentication, some versions of docker also require you to configure your cluster to trust the certificate at the OS level.

Remove the --insecure-registry option only for this particular registry in the /etc/sysconfig/docker file. Then, reload the daemon and restart the docker service to reflect this configuration change:

$ sudo systemctl daemon-reload
$ sudo systemctl restart docker

Validate the docker client connection. Running docker push to the registry or docker pull from the registry should succeed. Make sure you have logged into the registry.

$ docker tag|push <registry/image> <internal_registry/project/image>

For example:

$ docker pull busybox
$ docker tag docker.io/busybox 172.30.124.220:5000/openshift/busybox
$ docker push 172.30.124.220:5000/openshift/busybox
...
cf2616975b4a: Image successfully pushed
Digest: sha256:3662dd821983bc4326bee12caec61367e7fb6f6a3ee547cbaff98f77403cab55

Create a passthrough route via the oc create route passthrough command, specifying the registry as the route’s service. By default, the name of the created route is the same as the service name.

For example:

$ oc get svc
NAME              CLUSTER_IP       EXTERNAL_IP   PORT(S)                 SELECTOR                  AGE
docker-registry   172.30.69.167    <none>        5000/TCP                docker-registry=default   4h
kubernetes        172.30.0.1       <none>        443/TCP,53/UDP,53/TCP   <none>                    4h
router            172.30.172.132   <none>        80/TCP                  router=router             4h

$ oc create route passthrough    \
    --service=docker-registry    \1
    --hostname=<host>
route "docker-registry" created     2

$ oc get route/docker-registry -o yaml
apiVersion: v1
kind: Route
metadata:
  name: docker-registry
spec:
  host: <host> 1
  to:
    kind: Service
    name: docker-registry 2
  tls:
    termination: passthrough 3

Next, you must trust the certificates being used for the registry on your host system. The certificates referenced were created when you secured your registry.

$ sudo mkdir -p /etc/docker/certs.d/<host>
$ sudo cp <ca certificate file> /etc/docker/certs.d/<host>
$ sudo systemctl restart docker

Log in to the registry using the information from securing the registry. However, this time point to the host name used in the route rather than your service IP. You should now be able to tag and push images using the route host.

$ oc get imagestreams -n test
NAME      DOCKER REPO   TAGS      UPDATED

$ docker pull busybox
$ docker tag busybox <host>/test/busybox
$ docker push <host>/test/busybox
The push refers to a repository [<host>/test/busybox] (len: 1)
8c2e06607696: Image already exists
6ce2e90b0bc7: Image successfully pushed
cf2616975b4a: Image successfully pushed
Digest: sha256:6c7e676d76921031532d7d9c0394d0da7c2906f4cb4c049904c4031147d8ca31

$ docker pull <host>/test/busybox
latest: Pulling from <host>/test/busybox
cf2616975b4a: Already exists
6ce2e90b0bc7: Already exists
8c2e06607696: Already exists
Digest: sha256:6c7e676d76921031532d7d9c0394d0da7c2906f4cb4c049904c4031147d8ca31
Status: Image is up to date for <host>/test/busybox:latest

$ oc get imagestreams -n test
NAME      DOCKER REPO                       TAGS      UPDATED
busybox   172.30.11.215:5000/test/busybox   latest    2 seconds ago

Make a note of the clusterIP for the service:

$ oc get svc/docker-registry -o yaml | grep clusterIP:

Delete the service:

$ oc delete svc/docker-registry dc/docker-registry

Create the registry definition in registry.yaml, replacing <options> with, for example, those used in step 3 of the instructions in the Non-Production Use section:

$ oadm registry <options> -o yaml > registry.yaml

Create the registry using the modified registry.yaml:

$ oc create -f registry.yaml

Edit the /etc/sysconfig/docker file to block all registries:

BLOCK_REGISTRY='--block-registry=all'

You may need to uncomment the BLOCK_REGISTRY line.

In the same file, add registries to which you want to allow access:

ADD_REGISTRY='--add-registry=<registry1> --add-registry=<registry2>'

ADD_REGISTRY='--add-registry=registry.access.redhat.com'

This example would restrict access to images available on the Red Hat Customer Portal.

Edit the registry configuration file locally as needed. The initial YAML file deployed on the registry is provided below. Review supported options.

version: 0.1
log:
  level: debug
http:
  addr: :5000
storage:
  cache:
    blobdescriptor: inmemory
  filesystem:
    rootdirectory: /registry
  delete:
    enabled: true
auth:
  openshift:
    realm: openshift
middleware:
  registry:
    - name: openshift
  repository:
    - name: openshift
      options:
        acceptschema2: false
        pullthrough: true
        enforcequota: false
        projectcachettl: 1m
        blobrepositorycachettl: 10m
  storage:
    - name: openshift

Create a new secret called registry-config from your custom registry configuration file you edited locally:

$ oc secrets new registry-config config.yml=</path/to/custom/registry/config.yml>

Add the registry-config secret as a volume to the registry’s deployment configuration to mount the custom configuration file at /etc/docker/registry/:

$ oc volume dc/docker-registry --add --type=secret \
    --secret-name=registry-config -m /etc/docker/registry/

Update the registry to reference the configuration path from the previous step by adding the following environment variable to the registry’s deployment configuration:

$ oc set env dc/docker-registry \
    REGISTRY_CONFIGURATION_PATH=/etc/docker/registry/config.yml

Delete the registry-config secret:

$ oc delete secret registry-config

Recreate the secret to reference the updated configuration file:

$ oc secrets new registry-config config.yml=</path/to/custom/registry/config.yml>

Redeploy the registry to read the updated configuration:

$ oc deploy docker-registry --latest

Ensure that the sessionAffinity of your docker-registry service is set to ClientIP:

$ oc get svc/docker-registry --template='{{.spec.sessionAffinity}}'

This should return ClientIP, which is the default in recent OpenShift Container Platform versions. If not, change it:

$ oc get -o yaml svc/docker-registry | \
      sed 's/\(sessionAffinity:\s*\).*/\1ClientIP/' | \
      oc replace -f -