Chapter 28. Persistent Storage Examples

1. Install the latest ceph-common package:

   yum install -y ceph-common

   Note

   The ceph-common library must be installed on all schedulable OpenShift Container Platform nodes.

2. From an administrator or MON node, create a new pool for dynamic volumes, for example:

   $ ceph osd pool create kube 1024
   $ ceph auth get-or-create client.kube mon 'allow r, allow command "osd blacklist"' osd 'allow class-read object_prefix rbd_children, allow rwx pool=kube' -o ceph.client.kube.keyring

   Note

   Using the default pool of RBD is an option, but not recommended.

1. Generate the client.admin base64-encoded key:

   $ ceph auth get client.admin

   Ceph secret definition example

   apiVersion: v1
   kind: Secret
   metadata:
     name: ceph-secret
     namespace: kube-system
   data:
     key: QVFBOFF2SlZheUJQRVJBQWgvS2cwT1laQUhPQno3akZwekxxdGc9PQ== 1
   type: kubernetes.io/rbd 2

   1

   This base64 key is generated on one of the Ceph MON nodes using the ceph auth get-key client.admin | base64 command, then copying the output and pasting it as the secret key’s value.

   2

   This value is required for Ceph RBD to work with dynamic provisioning.

2. Create the Ceph secret for the client.admin:

   $ oc create -f ceph-secret.yaml
   secret "ceph-secret" created

3. Verify that the secret was created:

   $ oc get secret ceph-secret
   NAME          TYPE                DATA      AGE
   ceph-secret   kubernetes.io/rbd   1         5d

4. Create the storage class:

   $ oc create -f ceph-storageclass.yaml
   storageclass "dynamic" created

   Ceph storage class example

   apiVersion: storage.k8s.io/v1beta1
   kind: StorageClass
   metadata:
     name: dynamic
     annotations:
       storageclass.kubernetes.io/is-default-class: "true"
   provisioner: kubernetes.io/rbd
   parameters:
     monitors: 192.168.1.11:6789,192.168.1.12:6789,192.168.1.13:6789 1
     adminId: admin 2
     adminSecretName: ceph-secret 3
     adminSecretNamespace: kube-system 4
     pool: kube  5
     userId: kube  6
     userSecretName: ceph-user-secret 7

   1

   A comma-delimited list of IP addresses Ceph monitors. This value is required.

   2

   The Ceph client ID that is capable of creating images in the pool. The default is admin.

   3

   The secret name for adminId. This value is required. The secret that you provide must have kubernetes.io/rbd.

   4

   The namespace for adminSecret. The default is default.

   5

   The Ceph RBD pool. The default is rbd, but this value is not recommended.

   6

   The Ceph client ID used to map the Ceph RBD image. The default is the same as the secret name for adminId.

   7

   The name of the Ceph secret for userId to map the Ceph RBD image. It must exist in the same namespace as the PVCs. Unless you set the Ceph secret as the default in new projects, you must provide this parameter value.

5. Verify that the storage class was created:

   $ oc get storageclasses
   NAME                TYPE
   dynamic (default)   kubernetes.io/rbd

6. Create the PVC object definition:

   PVC object definition example

   kind: PersistentVolumeClaim
   apiVersion: v1
   metadata:
     name: ceph-claim-dynamic
   spec:
     accessModes:  1
       - ReadWriteOnce
     resources:
       requests:
         storage: 2Gi 2

   1

   The accessModes do not enforce access rights but instead act as labels to match a PV to a PVC.

   2

   This claim looks for PVs that offer 2Gi or greater capacity.

7. Create the PVC:

   $ oc create -f ceph-pvc.yaml
   persistentvolumeclaim "ceph-claim-dynamic" created

8. Verify that the PVC was created and bound to the expected PV:

   $ oc get pvc
   NAME        STATUS  VOLUME                                   CAPACITY ACCESSMODES  AGE
   ceph-claim  Bound   pvc-f548d663-3cac-11e7-9937-0024e8650c7a 2Gi      RWO          1m

9. Create the pod object definition:

   Pod object definition example

   apiVersion: v1
   kind: Pod
   metadata:
     name: ceph-pod1 1
   spec:
     containers:
     - name: ceph-busybox
       image: busybox 2
       command: ["sleep", "60000"]
       volumeMounts:
       - name: ceph-vol1 3
         mountPath: /usr/share/busybox 4
         readOnly: false
     volumes:
     - name: ceph-vol1
       persistentVolumeClaim:
         claimName: ceph-claim-dynamic 5

   1

   The name of this pod as displayed by oc get pod.

   2

   The image run by this pod. In this case, busybox is set to sleep.

   3

   The name of the volume. This name must be the same in both the containers and volumes sections.

   4

   The mount path in the container.

   5

   The PVC that is bound to the Ceph RBD cluster.

10. Create the pod:

    $ oc create -f ceph-pod1.yaml
    pod "ceph-pod1" created

11. Verify that the pod was created:

    $ oc get pod
    NAME        READY     STATUS   RESTARTS   AGE
    ceph-pod1   1/1       Running  0          2m

1. To enable static provisioning, first create a GlusterFS volume. See the Red Hat Gluster Storage Administration Guide for information on how to do this using the gluster command-line interface or the heketi project site for information on how to do this using heketi-cli. For this example, the volume will be named myVol1.

2. Define the following Service and Endpoints in gluster-endpoints.yaml:

   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: glusterfs-cluster 1
   spec:
     ports:
     - port: 1
   ---
   apiVersion: v1
   kind: Endpoints
   metadata:
     name: glusterfs-cluster 2
   subsets:
     - addresses:
         - ip: 192.168.122.221 3
       ports:
         - port: 1 4
     - addresses:
         - ip: 192.168.122.222 5
       ports:
         - port: 1 6
     - addresses:
         - ip: 192.168.122.223 7
       ports:
         - port: 1 8

   1 2

   These names must match.

   3 5 7

   The ip values must be the actual IP addresses of a Red Hat Gluster Storage server, not hostnames.

   4 6 8

   The port number is ignored.

3. From the OpenShift Container Platform master host, create the Service and Endpoints:

   $ oc create -f gluster-endpoints.yaml
   service "glusterfs-cluster" created
   endpoints "glusterfs-cluster" created

4. Verify that the Service and Endpoints were created:

   $ oc get services
   NAME                       CLUSTER_IP       EXTERNAL_IP   PORT(S)    SELECTOR        AGE
   glusterfs-cluster          172.30.205.34    <none>        1/TCP      <none>          44s
   
   $ oc get endpoints
   NAME                ENDPOINTS                                               AGE
   docker-registry     10.1.0.3:5000                                           4h
   glusterfs-cluster   192.168.122.221:1,192.168.122.222:1,192.168.122.223:1   11s
   kubernetes          172.16.35.3:8443                                        4d

   Note

   Endpoints are unique per project. Each project accessing the GlusterFS volume needs its own Endpoints.

5. In order to access the volume, the container must run with either a user ID (UID) or group ID (GID) that has access to the file system on the volume. This information can be discovered in the following manner:

   $ mkdir -p /mnt/glusterfs/myVol1
   
   $ mount -t glusterfs 192.168.122.221:/myVol1 /mnt/glusterfs/myVol1
   
   $ ls -lnZ /mnt/glusterfs/
   drwxrwx---. 592 590 system_u:object_r:fusefs_t:s0    myVol1 1 2

   1

   The UID is 592.

   2

   The GID is 590.

6. Define the following PersistentVolume (PV) in gluster-pv.yaml:

   apiVersion: v1
   kind: PersistentVolume
   metadata:
     name: gluster-default-volume 1
     annotations:
       pv.beta.kubernetes.io/gid: "590" 2
   spec:
     capacity:
       storage: 2Gi 3
     accessModes: 4
       - ReadWriteMany
     glusterfs:
       endpoints: glusterfs-cluster 5
       path: myVol1 6
       readOnly: false
     persistentVolumeReclaimPolicy: Retain

   1

   The name of the volume.

   2

   The GID on the root of the GlusterFS volume.

   3

   The amount of storage allocated to this volume.

   4

   accessModes are used as labels to match a PV and a PVC. They currently do not define any form of access control.

   5

   The Endpoints resource previously created.

   6

   The GlusterFS volume that will be accessed.

7. From the OpenShift Container Platform master host, create the PV:

   $ oc create -f gluster-pv.yaml

8. Verify that the PV was created:

   $ oc get pv
   NAME                     LABELS    CAPACITY     ACCESSMODES   STATUS      CLAIM     REASON    AGE
   gluster-default-volume   <none>    2147483648   RWX           Available                       2s

9. Create a PersistentVolumeClaim (PVC) that will bind to the new PV in gluster-claim.yaml:

   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: gluster-claim  1
   spec:
     accessModes:
     - ReadWriteMany      2
     resources:
        requests:
          storage: 1Gi    3

   1

   The claim name is referenced by the pod under its volumes section.

   2

   Must match the accessModes of the PV.

   3

   This claim will look for PVs offering 1Gi or greater capacity.

10. From the OpenShift Container Platform master host, create the PVC:

    $ oc create -f gluster-claim.yaml

11. Verify that the PV and PVC are bound:

    $ oc get pv
    NAME         LABELS    CAPACITY   ACCESSMODES   STATUS      CLAIM          REASON    AGE
    gluster-pv   <none>    1Gi        RWX           Available   gluster-claim            37s
    
    $ oc get pvc
    NAME            LABELS    STATUS    VOLUME       CAPACITY   ACCESSMODES   AGE
    gluster-claim   <none>    Bound     gluster-pv   1Gi        RWX           24s

1. Create the pod object definition:

   apiVersion: v1
   kind: Pod
   metadata:
     name: hello-openshift-pod
     labels:
       name: hello-openshift-pod
   spec:
     containers:
     - name: hello-openshift-pod
       image: openshift/hello-openshift
       ports:
       - name: web
         containerPort: 80
       volumeMounts:
       - name: gluster-vol1
         mountPath: /usr/share/nginx/html
         readOnly: false
     volumes:
     - name: gluster-vol1
       persistentVolumeClaim:
         claimName: gluster1 1

   1

   The name of the PVC created in the previous step.

2. From the OpenShift Container Platform master host, create the pod:

   # oc create -f hello-openshift-pod.yaml
   pod "hello-openshift-pod" created

3. View the pod. Give it a few minutes, as it might need to download the image if it does not already exist:

   # oc get pods -o wide
   NAME                               READY     STATUS    RESTARTS   AGE       IP               NODE
   hello-openshift-pod                          1/1       Running   0          9m        10.38.0.0        node1

4. oc exec into the container and create an index.html file in the mountPath definition of the pod:

   $ oc exec -ti hello-openshift-pod /bin/sh
   $ cd /usr/share/nginx/html
   $ echo 'Hello OpenShift!!!' > index.html
   $ ls
   index.html
   $ exit

5. Now curl the URL of the pod:

   # curl http://10.38.0.0
   Hello OpenShift!!!

6. Delete the pod, recreate it, and wait for it to come up:

   # oc delete pod hello-openshift-pod
   pod "hello-openshift-pod" deleted
   # oc create -f hello-openshift-pod.yaml
   pod "hello-openshift-pod" created
   # oc get pods -o wide
   NAME                               READY     STATUS    RESTARTS   AGE       IP               NODE
   hello-openshift-pod                          1/1       Running   0          9m        10.37.0.0        node1

7. Now curl the pod again and it should still have the same data as before. Note that its IP address may have changed:

   # curl http://10.37.0.0
   Hello OpenShift!!!

8. Check that the index.html file was written to GlusterFS storage by doing the following on any of the nodes:

   $ mount | grep heketi
   /dev/mapper/VolGroup00-LogVol00 on /var/lib/heketi type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
   /dev/mapper/vg_f92e09091f6b20ab12b02a2513e4ed90-brick_1e730a5462c352835055018e1874e578 on /var/lib/heketi/mounts/vg_f92e09091f6b20ab12b02a2513e4ed90/brick_1e730a5462c352835055018e1874e578 type xfs (rw,noatime,seclabel,nouuid,attr2,inode64,logbsize=256k,sunit=512,swidth=512,noquota)
   /dev/mapper/vg_f92e09091f6b20ab12b02a2513e4ed90-brick_d8c06e606ff4cc29ccb9d018c73ee292 on /var/lib/heketi/mounts/vg_f92e09091f6b20ab12b02a2513e4ed90/brick_d8c06e606ff4cc29ccb9d018c73ee292 type xfs (rw,noatime,seclabel,nouuid,attr2,inode64,logbsize=256k,sunit=512,swidth=512,noquota)
   
   $ cd /var/lib/heketi/mounts/vg_f92e09091f6b20ab12b02a2513e4ed90/brick_d8c06e606ff4cc29ccb9d018c73ee292/brick
   $ ls
   index.html
   $ cat index.html
   Hello OpenShift!!!

1. To enable dynamic provisioning, first create a StorageClass object definition. The definition below is based on the minimum requirements needed for this example to work with OpenShift Container Platform. See Dynamic Provisioning and Creating Storage Classes for additional parameters and specification definitions.

   kind: StorageClass
   apiVersion: storage.k8s.io/v1
   metadata:
     name: glusterfs
   provisioner: kubernetes.io/glusterfs
   parameters:
     resturl: "http://10.42.0.0:8080" 1
     restauthenabled: "false" 2

   1

   The heketi server URL.

   2

   Since authentication is not turned on in this example, set to false.

2. From the OpenShift Container Platform master host, create the StorageClass:

   # oc create -f gluster-storage-class.yaml
   storageclass "glusterfs" created

3. Create a PVC using the newly-created StorageClass. For example:

   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: gluster1
   spec:
     accessModes:
     - ReadWriteMany
     resources:
       requests:
         storage: 30Gi
     storageClassName: glusterfs

4. From the OpenShift Container Platform master host, create the PVC:

   # oc create -f glusterfs-dyn-pvc.yaml
   persistentvolumeclaim "gluster1" created

5. View the PVC to see that the volume was dynamically created and bound to the PVC:

   # oc get pvc
   NAME       STATUS   VOLUME                                     CAPACITY   ACCESSMODES   STORAGECLASS   AGE
   gluster1   Bound    pvc-78852230-d8e2-11e6-a3fa-0800279cf26f   30Gi       RWX           glusterfs      42s

1. Create the pod object definition:

   apiVersion: v1
   kind: Pod
   metadata:
     name: hello-openshift-pod
     labels:
       name: hello-openshift-pod
   spec:
     containers:
     - name: hello-openshift-pod
       image: openshift/hello-openshift
       ports:
       - name: web
         containerPort: 80
       volumeMounts:
       - name: gluster-vol1
         mountPath: /usr/share/nginx/html
         readOnly: false
     volumes:
     - name: gluster-vol1
       persistentVolumeClaim:
         claimName: gluster1 1

   1

   The name of the PVC created in the previous step.

2. From the OpenShift Container Platform master host, create the pod:

   # oc create -f hello-openshift-pod.yaml
   pod "hello-openshift-pod" created

3. View the pod. Give it a few minutes, as it might need to download the image if it does not already exist:

   # oc get pods -o wide
   NAME                               READY     STATUS    RESTARTS   AGE       IP               NODE
   hello-openshift-pod                          1/1       Running   0          9m        10.38.0.0        node1

4. oc exec into the container and create an index.html file in the mountPath definition of the pod:

   $ oc exec -ti hello-openshift-pod /bin/sh
   $ cd /usr/share/nginx/html
   $ echo 'Hello OpenShift!!!' > index.html
   $ ls
   index.html
   $ exit

5. Now curl the URL of the pod:

   # curl http://10.38.0.0
   Hello OpenShift!!!

6. Delete the pod, recreate it, and wait for it to come up:

   # oc delete pod hello-openshift-pod
   pod "hello-openshift-pod" deleted
   # oc create -f hello-openshift-pod.yaml
   pod "hello-openshift-pod" created
   # oc get pods -o wide
   NAME                               READY     STATUS    RESTARTS   AGE       IP               NODE
   hello-openshift-pod                          1/1       Running   0          9m        10.37.0.0        node1

7. Now curl the pod again and it should still have the same data as before. Note that its IP address may have changed:

   # curl http://10.37.0.0
   Hello OpenShift!!!

8. Check that the index.html file was written to GlusterFS storage by doing the following on any of the nodes:

   $ mount | grep heketi
   /dev/mapper/VolGroup00-LogVol00 on /var/lib/heketi type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
   /dev/mapper/vg_f92e09091f6b20ab12b02a2513e4ed90-brick_1e730a5462c352835055018e1874e578 on /var/lib/heketi/mounts/vg_f92e09091f6b20ab12b02a2513e4ed90/brick_1e730a5462c352835055018e1874e578 type xfs (rw,noatime,seclabel,nouuid,attr2,inode64,logbsize=256k,sunit=512,swidth=512,noquota)
   /dev/mapper/vg_f92e09091f6b20ab12b02a2513e4ed90-brick_d8c06e606ff4cc29ccb9d018c73ee292 on /var/lib/heketi/mounts/vg_f92e09091f6b20ab12b02a2513e4ed90/brick_d8c06e606ff4cc29ccb9d018c73ee292 type xfs (rw,noatime,seclabel,nouuid,attr2,inode64,logbsize=256k,sunit=512,swidth=512,noquota)
   
   $ cd /var/lib/heketi/mounts/vg_f92e09091f6b20ab12b02a2513e4ed90/brick_d8c06e606ff4cc29ccb9d018c73ee292/brick
   $ ls
   index.html
   $ cat index.html
   Hello OpenShift!!!

1. As the admin, create the service, endpoint object, and persistent volume:

   $ oc create -f gluster-endpoints-service.yaml
   $ oc create -f gluster-endpoints.yaml
   $ oc create -f gluster-pv.yaml

2. Verify that the objects were created:

   $ oc get svc
   NAME              CLUSTER_IP      EXTERNAL_IP   PORT(S)   SELECTOR   AGE
   gluster-cluster   172.30.151.58   <none>        1/TCP     <none>     24s

   $ oc get ep
   NAME              ENDPOINTS                           AGE
   gluster-cluster   192.168.59.102:1,192.168.59.103:1   2m

   $ oc get pv
   NAME                     LABELS    CAPACITY   ACCESSMODES   STATUS      CLAIM     REASON    AGE
   gluster-default-volume   <none>    2Gi        RWX           Available                       2d

1. As the admin, add a user to the SCC:

   $ oc adm policy add-scc-to-user privileged <username>

2. Log in as the regular user:

   $ oc login -u <username> -p <password>

3. Then, create a new project:

   $ oc new-project <project_name>

1. As a regular user, create the PersistentVolumeClaim to access the volume:

   $ oc create -f gluster-pvc.yaml -n <project_name>

2. Define your pod to access the claim:

   Example 28.10. Pod Definition

   apiVersion: v1
   id: gluster-S3-pvc
   kind: Pod
   metadata:
     name: gluster-nginx-priv
   spec:
     containers:
       - name: gluster-nginx-priv
         image: fedora/nginx
         volumeMounts:
           - mountPath: /mnt/gluster 1
             name: gluster-volume-claim
         securityContext:
           privileged: true
     volumes:
       - name: gluster-volume-claim
         persistentVolumeClaim:
           claimName: gluster-claim 2

   1

   Volume mount within the pod.

   2

   The gluster-claim must reflect the name of the PersistentVolume.

3. Upon pod creation, the mount directory is created and the volume is attached to that mount point.

   As regular user, create a pod from the definition:

   $ oc create -f gluster-S3-pod.yaml

4. Verify that the pod created successfully:

   $ oc get pods
   NAME                 READY     STATUS    RESTARTS   AGE
   gluster-S3-pod   1/1       Running   0          36m

   It can take several minutes for the pod to create.

Procedure

1. Edit your Docker’s systemd service file. Set MountFlags as follows:

   MountFlags=shared

   Alternatively, remove MountFlags=slave, if present.

2. Restart the Docker daemon:

   $ sudo systemctl daemon-reload
   $ sudo systemctl restart docker

1. To enable static provisioning, first create a GlusterFS volume. See the Red Hat Gluster Storage Administration Guide for information on how to do this using the gluster command-line interface or the heketi project site for information on how to do this using heketi-cli. For this example, the volume will be named myVol1.

2. Define the following Service and Endpoints in gluster-endpoints.yaml:

   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: glusterfs-cluster 1
   spec:
     ports:
     - port: 1
   ---
   apiVersion: v1
   kind: Endpoints
   metadata:
     name: glusterfs-cluster 2
   subsets:
     - addresses:
         - ip: 192.168.122.221 3
       ports:
         - port: 1 4
     - addresses:
         - ip: 192.168.122.222 5
       ports:
         - port: 1 6
     - addresses:
         - ip: 192.168.122.223 7
       ports:
         - port: 1 8

   1 2

   These names must match.

   3 5 7

   The ip values must be the actual IP addresses of a Red Hat Gluster Storage server, not hostnames.

   4 6 8

   The port number is ignored.

3. From the OpenShift Container Platform master host, create the Service and Endpoints:

   $ oc create -f gluster-endpoints.yaml
   service "glusterfs-cluster" created
   endpoints "glusterfs-cluster" created

4. Verify that the Service and Endpoints were created:

   $ oc get services
   NAME                       CLUSTER_IP       EXTERNAL_IP   PORT(S)    SELECTOR        AGE
   glusterfs-cluster          172.30.205.34    <none>        1/TCP      <none>          44s
   
   $ oc get endpoints
   NAME                ENDPOINTS                                               AGE
   docker-registry     10.1.0.3:5000                                           4h
   glusterfs-cluster   192.168.122.221:1,192.168.122.222:1,192.168.122.223:1   11s
   kubernetes          172.16.35.3:8443                                        4d

   Note

   Endpoints are unique per project. Each project accessing the GlusterFS volume needs its own Endpoints.

5. In order to access the volume, the container must run with either a user ID (UID) or group ID (GID) that has access to the file system on the volume. This information can be discovered in the following manner:

   $ mkdir -p /mnt/glusterfs/myVol1
   
   $ mount -t glusterfs 192.168.122.221:/myVol1 /mnt/glusterfs/myVol1
   
   $ ls -lnZ /mnt/glusterfs/
   drwxrwx---. 592 590 system_u:object_r:fusefs_t:s0    myVol1 1 2

   1

   The UID is 592.

   2

   The GID is 590.

6. Define the following PersistentVolume (PV) in gluster-pv.yaml:

   apiVersion: v1
   kind: PersistentVolume
   metadata:
     name: gluster-default-volume 1
     annotations:
       pv.beta.kubernetes.io/gid: "590" 2
   spec:
     capacity:
       storage: 2Gi 3
     accessModes: 4
       - ReadWriteMany
     glusterfs:
       endpoints: glusterfs-cluster 5
       path: myVol1 6
       readOnly: false
     persistentVolumeReclaimPolicy: Retain

   1

   The name of the volume.

   2

   The GID on the root of the GlusterFS volume.

   3

   The amount of storage allocated to this volume.

   4

   accessModes are used as labels to match a PV and a PVC. They currently do not define any form of access control.

   5

   The Endpoints resource previously created.

   6

   The GlusterFS volume that will be accessed.

7. From the OpenShift Container Platform master host, create the PV:

   $ oc create -f gluster-pv.yaml

8. Verify that the PV was created:

   $ oc get pv
   NAME                     LABELS    CAPACITY     ACCESSMODES   STATUS      CLAIM     REASON    AGE
   gluster-default-volume   <none>    2147483648   RWX           Available                       2s

9. Create a PersistentVolumeClaim (PVC) that will bind to the new PV in gluster-claim.yaml:

   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: gluster-claim  1
   spec:
     accessModes:
     - ReadWriteMany      2
     resources:
        requests:
          storage: 1Gi    3

   1

   The claim name is referenced by the pod under its volumes section.

   2

   Must match the accessModes of the PV.

   3

   This claim will look for PVs offering 1Gi or greater capacity.

10. From the OpenShift Container Platform master host, create the PVC:

    $ oc create -f gluster-claim.yaml

11. Verify that the PV and PVC are bound:

    $ oc get pv
    NAME         LABELS    CAPACITY   ACCESSMODES   STATUS      CLAIM          REASON    AGE
    gluster-pv   <none>    1Gi        RWX           Available   gluster-claim            37s
    
    $ oc get pvc
    NAME            LABELS    STATUS    VOLUME       CAPACITY   ACCESSMODES   AGE
    gluster-claim   <none>    Bound     gluster-pv   1Gi        RWX           24s

1. Create a new registry configuration file called registryconfig.yaml and add the following information:

   version: 0.1
   log:
     level: debug
   http:
     addr: :5000
   storage:
     cache:
       blobdescriptor: inmemory
     delete:
       enabled: true
     azure: 1
       accountname: azureblobacc
       accountkey:  azureblobacckey
       container: azureblobname
       realm: core.windows.net 2
   auth:
     openshift:
       realm: openshift
   middleware:
     registry:
       - name: openshift
     repository:
       - name: openshift
         options:
           acceptschema2: false
           pullthrough: true
           enforcequota: false
           projectcachettl: 1m
           blobrepositorycachettl: 10m
     storage:
       - name: openshift

   1

   Replace the values for accountname, acountkey, and container with storage account name, storage account key, and storage container name respectively.

   2

   If using Azure regional cloud, set to the desired realm. For example, core.cloudapi.de for the Germany regional cloud.

2. Create a new registry configuration:

   $ oc create secret generic registry-config --from-file=config.yaml=registryconfig.yaml

3. Add the secret:

   $ oc set volume dc/docker-registry --add --type=secret \
       --secret-name=registry-config -m /etc/docker/registry/

4. Set the REGISTRY_CONFIGURATION_PATH environment variable:

   $ oc set env dc/docker-registry \
       REGISTRY_CONFIGURATION_PATH=/etc/docker/registry/config.yaml

5. If you already created a registry configuration:

   1. Delete the secret:

      $ oc delete secret registry-config

   2. Create a new registry configuration:

      $ oc create secret generic registry-config --from-file=config.yaml=registryconfig.yaml

   3. Update the configuration by starting a new rollout:

      $ oc rollout latest docker-registry