Chapter 24. Configuring for Azure

Procedure

1. Using the Azure CLI, obtain the account subscription ID:

   # az account list
   [
   {
     "cloudName": "AzureCloud",
     "id": "<subscription>", 1
     "isDefault": false,
     "name": "Pay-As-You-Go",
     "state": "Enabled",
     "tenantId": "<tenant-id>",
     "user": {
       "name": "admin@example.com",
       "type": "user"
     }
   ]

   1

   The subscription ID to use to create the new permissions.

2. Create the service principal with the Microsoft Azure role of contributor and with the scope of the Microsoft Azure subscription and the resource group. Record the output of these values to be used when defining the inventory. Use the <subscription> value from the previous step in place of the value below:

   # az ad sp create-for-rbac --name openshiftcloudprovider \
        --password <secret> --role contributor \
        --scopes /subscriptions/<subscription>/resourceGroups/<resource-group>
   
   Retrying role assignment creation: 1/36
   Retrying role assignment creation: 2/36
   {
     "appId": "<app-id>",
     "displayName": "ocpcloudprovider",
     "name": "http://ocpcloudprovider",
     "password": "<secret>",
     "tenant": "<tenant-id>"
   }

1. Create an Azure storage account:

   az storage account create
   --name <account_name> \
   --resource-group <resource_group> \
   --location <location> \
   --sku Standard_LRS

   This creates an account key. To view the account key:

   az storage account keys list \
       --account-name <account-name> \
       --resource-group <resource-group> \
       --output table
   
   KeyName    Permissions    Value
   key1       Full           <account-key>
   key2       Full           <extra-account-key>

Procedure

1. Configure the Ansible inventory for the registry to use the storage account:

   [OSEv3:vars]
   # Azure Registry Configuration
   openshift_hosted_registry_replicas=1 1
   openshift_hosted_registry_storage_kind=object
   openshift_hosted_registry_storage_azure_blob_accountkey=<account_key> 2
   openshift_hosted_registry_storage_provider=azure_blob
   openshift_hosted_registry_storage_azure_blob_accountname=<account_name> 3
   openshift_hosted_registry_storage_azure_blob_container=<registry> 4
   openshift_hosted_registry_storage_azure_blob_realm=core.windows.net

   1

   The number of replicas to configure.

   2

   The account key associated with the <account-name>.

   3

   The storage account name.

   4

   Directory used to store the data. registry by default

Procedure

1. Export the current config.yml:

   $ oc get secret registry-config \
       -o jsonpath='{.data.config\.yml}' -n default | base64 -d \
     >> config.yml.old

2. Create a new configuration file from the old config.yml:

   $ cp config.yml.old config.yml

3. Edit the file to include the Azure parameters:

   storage:
     delete:
       enabled: true
     cache:
       blobdescriptor: inmemory
     azure:
       accountname: <account-name> 1
       accountkey: <account-key> 2
       container: registry 3
       realm: core.windows.net 4

   1

   Replace with the storage account name.

   2

   The account key associated to the <account-name>.

   3

   Directory used to store the data. registry by default

   4

   Storage realm core.windows.net by default

4. Delete the registry-config secret:

   $ oc delete secret registry-config -n default

5. Recreate the secret to reference the updated configuration file:

   $ oc create secret generic registry-config \
       --from-file=config.yml -n default

6. Redeploy the registry to read the updated configuration:

   $ oc rollout latest docker-registry -n default

Procedure

1. After a successful registry deployment, the registry deploymentconfig will always show that the registry is using an emptydir instead of Microsoft Azure blob storage:

   $ oc describe dc docker-registry -n default
   ...
   Mounts:
     ...
     /registry from registry-storage (rw)
   Volumes:
   registry-storage:
   Type:       EmptyDir 1
   ...

   1

   The temporary directory that shares a pod’s lifetime.

2. Check if the /registry mount point is empty. This is the volume Microsoft Azure storage will use:

   $ oc exec \
       $(oc get pod -l deploymentconfig=docker-registry \
       -o=jsonpath='{.items[0].metadata.name}')  -i -t -- ls -l /registry
   total 0

3. If it is empty, it is because the Microsoft Azure blob configuration is performed in the registry-config secret:

   $ oc describe secret registry-config
   Name:         registry-config
   Namespace:    default
   Labels:       <none>
   Annotations:  <none>
   
   Type:  Opaque
   
   Data
   ====
   config.yml:  398 bytes

4. The installer creates a config.yml file with the desired configuration using the extended registry capabilities as seen in Storage in the installation documentation. To view the configuration file, including the storage section where the storage bucket configuration is stored:

   $ oc exec \
       $(oc get pod -l deploymentconfig=docker-registry \
         -o=jsonpath='{.items[0].metadata.name}') \
     cat /etc/registry/config.yml
   
     version: 0.1
     log:
       level: debug
     http:
       addr: :5000
     storage:
       delete:
         enabled: true
       cache:
         blobdescriptor: inmemory
       azure:
         accountname: registry
         accountkey: uZekVBJBa6xzwAqK8EDz15/hoHUoc8I6cPfP31ZS+QOSxLfo7WT7CLrVPKaqvtNTMgztxH7CGjYfpFRNUhvMiA==
         container: registry
         realm: core.windows.net
     auth:
       openshift:
         realm: openshift
     middleware:
       registry:
       - name: openshift
       repository:
       - name: openshift
         options:
           pullthrough: True
           acceptschema2: True
           enforcequota: False
       storage:
       - name: openshift

   Or you can view the secret:

   $ oc get secret registry-config -o jsonpath='{.data.config\.yml}' | base64 -d
   version: 0.1
   log:
     level: debug
   http:
     addr: :5000
   storage:
     delete:
       enabled: true
     cache:
       blobdescriptor: inmemory
     azure:
       accountname: registry
       accountkey: uZekVBJBa6xzwAqK8EDz15/hoHUoc8I6cPfP31ZS+QOSxLfo7WT7CLrVPKaqvtNTMgztxH7CGjYfpFRNUhvMiA==
       container: registry
       realm: core.windows.net
   auth:
     openshift:
       realm: openshift
   middleware:
     registry:
     - name: openshift
     repository:
     - name: openshift
       options:
         pullthrough: True
         acceptschema2: True
         enforcequota: False
     storage:
     - name: openshift

Procedure

1. The following storageclass is created when you configure the Azure cloud provider at installation using the openshift_cloudprovider_kind=azure and openshift_cloud_provider_azure variables in the Ansible inventory:

   $ oc get --export storageclass azure-standard -o yaml
   apiVersion: storage.k8s.io/v1
   kind: StorageClass
   metadata:
     annotations:
       storageclass.kubernetes.io/is-default-class: "true"
     creationTimestamp: null
     name: azure-standard
   parameters:
     kind: Shared
     storageaccounttype: Standard_LRS
   provisioner: kubernetes.io/azure-disk
   reclaimPolicy: Delete
   volumeBindingMode: Immediate

   If you did not use Ansible to enable OpenShift Container Platform and Microsoft Azure integration, you can create the storageclass manually. See the Dynamic provisioning and creating storage classes section for more information.

2. Currently, the default storageclass kind is shared which means that the Microsoft Azure instances must use unmanaged disks. You can optionally modify this by allowing instances to use managed disks by providing the openshift_storageclass_parameters={'kind': 'Managed', 'storageaccounttype': 'Premium_LRS'} or openshift_storageclass_parameters={'kind': 'Managed', 'storageaccounttype': 'Standard_LRS'} variables in the Ansible inventory file at installation.

Procedure

1. Create a new application:

   $ oc new-app openshift/hello-openshift

2. Expose the load balancer service:

   $ oc expose dc hello-openshift --name='hello-openshift-external' --type='LoadBalancer'

   This creates a Loadbalancer service similar to the following:

   apiVersion: v1
   kind: Service
   metadata:
     labels:
       app: hello-openshift
     name: hello-openshift-external
   spec:
     externalTrafficPolicy: Cluster
     ports:
     - name: port-1
       nodePort: 30714
       port: 8080
       protocol: TCP
       targetPort: 8080
     - name: port-2
       nodePort: 30122
       port: 8888
       protocol: TCP
       targetPort: 8888
     selector:
       app: hello-openshift
       deploymentconfig: hello-openshift
     sessionAffinity: None
     type: LoadBalancer

3. Verify that the service has been created:

   $ oc get svc
   NAME                       TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)                         AGE
   hello-openshift            ClusterIP      172.30.223.255   <none>          8080/TCP,8888/TCP               1m
   hello-openshift-external   LoadBalancer   172.30.99.54     40.121.42.180   8080:30714/TCP,8888:30122/TCP   4m

   The LoadBalancer type and External-IP fields indicate that the service is using Microsoft Azure load balancers to expose the application.