Chapter 9. Updating the Registry with Red Hat Openshift Container Storage as the Storage Back-end

On the master or client, execute the following command to login as the cluster admin user:

# oc login

For example:

# oc login

Authentication required for https://master.example.com:8443 (openshift)
Username: <cluster-admin-user>
Password: <password>
Login successful.

You have access to the following projects and can switch between them with 'oc project <projectname>':

  * default
    management-infra
    openshift
    openshift-infra

Using project "default".

If you are not automatically logged into project default, then switch to it by executing the following command:

# oc project default

To verify that the pod is created, execute the following command:

# oc get pods

For example:

# oc get pods
NAME                       READY     STATUS    RESTARTS   AGE
docker-registry-2-mbu0u    1/1       Running   4          6d
docker-registry-2-spw0o    1/1       Running   3          6d
registry-console-1-rblwo   1/1       Running   3          6d

To verify that the endpoints are created, execute the following command:

# oc get endpoints

For example:

# oc get endpoints
NAME               ENDPOINTS                                                                  AGE
docker-registry    10.128.0.15:5000,10.129.0.9:5000                                           7d
kubernetes         192.168.234.143:8443,192.168.234.143:8053,192.168.234.143:8053             7d
registry-console   10.128.0.17:9090                                                           7d
router             192.168.234.144:443,192.168.234.145:443,192.168.234.144:1936 + 3 more...   7d

To verify that the persistent volume is created, execute the following command:

# oc get pv
NAME   CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS      CLAIM  REASON    AGE
registry-volume           5Gi        RWX           Retain          Bound       default/registry-claim             7d

To obtain the details of the persistent volume that was created for the NFS registry, execute the following command:

# oc describe pv registry-volume
Name:        registry-volume
Labels:        <none>
StorageClass:
Status:        Bound
Claim:        default/registry-claim
Reclaim Policy:    Retain
Access Modes:    RWX
Capacity:    5Gi
Message:
Source:
    Type:    NFS (an NFS mount that lasts the lifetime of a pod)
    Server:    cns30.rh73
    Path:    /exports/registry
    ReadOnly:    false
No events.

Login to the default project:

# oc project default

For example:

# oc project default
Now using project "default" on server "https://cns30.rh73:8443"

Execute the following command to create the gluster-registry-endpoints.yaml file:

 oc get endpoints <heketi-db-storage-endpoint-name> -o yaml --namespace=<project-name> >  gluster-registry-endpoints.yaml

Edit the gluster-registry-endpoints.yaml file. Change the name to gluster-registry-endpoints and remove all the other metadata, leaving everything else the same.

# cat gluster-registry-endpoints.yaml
apiVersion: v1
kind: Endpoints
metadata:
  name: gluster-registry-endpoints
subsets:
  - addresses:
      - ip: 192.168.124.114
      - ip: 192.168.124.52
      - ip: 192.168.124.83
    ports:
      - port: 1
protocol: TCP

Execute the following command to create the endpoint:

# oc create -f gluster-registry-endpoints.yaml
endpoints "gluster-registry-endpoints" created

To verify the creation of the endpoint, execute the following command:

# oc get endpoints
NAME                       ENDPOINTS                                                                 AGE
docker-registry            10.129.0.8:5000,10.130.0.5:5000                                           28d
gluster-registry-endpoints  192.168.124.114:1,192.168.124.52:1,192.168.124.83:1                       10s
kubernetes                 192.168.124.250:8443,192.168.124.250:8053,192.168.124.250:8053            28d
registry-console           10.131.0.6:9090                                                           28d
router                     192.168.124.114:443,192.168.124.83:443,192.168.124.114:1936 + 3 more...   28d

Execute the following command to create the gluster-registry-service.yaml file:

 oc get services <heketi-storage-endpoint-name> -o yaml --namespace=<project-name> >  gluster-registry-service.yaml

Edit the gluster-registry-service.yaml file. Change the name to gluster-registry-service and remove all the other metadata. Also, remove the specific cluster IP addresses:

# cat gluster-registry-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: gluster-registry-service
spec:
  ports:
    - port: 1
      protocol: TCP
      targetPort: 1
  sessionAffinity: None
  type: ClusterIP
status:
loadBalancer: {}

Execute the following command to create the service:

# oc create -f gluster-registry-service.yaml
services "gluster-registry-service" created

Execute the following command to verify if the service are running:

# oc get services
NAME                       CLUSTER-IP       EXTERNAL-IP   PORT(S)                   AGE
docker-registry            172.30.197.118   <none>        5000/TCP                  28d
gluster-registry-service   172.30.0.183     <none>        1/TCP                     6s
kubernetes                 172.30.0.1       <none>        443/TCP,53/UDP,53/TCP     29d
registry-console           172.30.146.178   <none>        9000/TCP                  28d
router                     172.30.232.238   <none>        80/TCP,443/TCP,1936/TCP   28d

Execute the following command to obtain the fsGroup GID of the existing docker-registry pods:

# export GID=$(oc get po --selector="docker-registry=default" -o go-template --template='{{printf "%.0f" ((index .items 0).spec.securityContext.fsGroup)}}')

Execute the following command to create a volume

# heketi-cli volume create --size=5 --name=gluster-registry-volume --gid=${GID}

Create the persistent volume file for the Red Hat Gluster Storage volume:

# cat gluster-registry-volume.yaml
kind: PersistentVolume
apiVersion: v1
metadata:
  name: gluster-registry-volume
  labels:
    glusterfs: registry-volume
spec:
  capacity:
    storage: 5Gi
  glusterfs:
    endpoints: gluster-registry-endpoints
    path: gluster-registry-volume
  accessModes:
    - ReadWriteMany
persistentVolumeReclaimPolicy: Retain

Execute the following command to create the persistent volume:

# oc create -f gluster-registry-volume.yaml

Execute the following command to verify and get the details of the created persistent volume:

# oc get pv/gluster-registry-volume
NAME                      CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS      CLAIM     REASON    AGE
gluster-registry-volume   5Gi        RWX           Retain          Available                       21m

Create a new persistent volume claim. Following is a sample Persistent Volume Claim that will be used to replace the existing registry-storage volume claim.

# cat gluster-registry-claim.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: gluster-registry-claim
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 5Gi
  selector:
    matchLabels:
glusterfs: registry-volume

Create the persistent volume claim by executing the following command:

# oc create -f gluster-registry-claim.yaml

For example:

# oc create -f gluster-registry-claim.yaml
persistentvolumeclaim "gluster-registry-claim" created

Execute the following command to verify if the claim is bound:

# oc get pvc/gluster-registry-claim

For example:

# oc get pvc/gluster-registry-claim
NAME                     STATUS    VOLUME                    CAPACITY   ACCESSMODES   AGE
gluster-registry-claim   Bound     gluster-registry-volume   5Gi        RWX           22s

Make the registry read-only by executing the following command:

# oc set env -n default dc/docker-registry 'REGISTRY_STORAGE_MAINTENANCE_READONLY={"enabled":true}'

To confirm the value is set to readonly, execute the following command:

# oc set env -n default dc/docker-registry --list

If you want to migrate the data from the old registry to the Red Hat Gluster Storage registry, then execute the following commands:

Replace the existing registry-storage volume with the new gluster-registry-claim PVC:

# oc set volume dc/docker-registry --add --name=registry-storage -t pvc --claim-name=gluster-registry-claim --overwrite

Make the registry read write by executing the following command:

# oc set env dc/docker-registry REGISTRY_STORAGE_MAINTENANCE_READONLY-

To validate if the setting is set to read write, execute the following command:

# oc set env -n default dc/docker-registry --list