Chapter 5. Disaster recovery with stretch cluster for OpenShift Data Foundation

Procedure

1. Log in to the OpenShift Web Console.
2. Click Operators OperatorHub.
3. Type local storage in the Filter by keyword​ box to find the Local Storage Operator from the list of operators, and click on it.

4. Set the following options on the Install Operator page:

   1. Update channel as stable.
   2. Installation mode as A specific namespace on the cluster.
   3. Installed Namespace as Operator recommended namespace openshift-local-storage.
   4. Update approval as Automatic.
5. Click Install.

Procedure

1. Log in to the OpenShift Web Console.
2. Click Operators OperatorHub.
3. Scroll or type OpenShift Data Foundation into the Filter by keyword box to search for the OpenShift Data Foundation Operator.
4. Click Install.

5. Set the following options on the Install Operator page:

   1. Update Channel as stable-4.15.
   2. Installation Mode as A specific namespace on the cluster.
   3. Installed Namespace as Operator recommended namespace openshift-storage. If Namespace openshift-storage does not exist, it is created during the operator installation.

   4. Select Approval Strategy as Automatic or Manual.

      If you select Automatic updates, then the Operator Lifecycle Manager (OLM) automatically upgrades the running instance of your Operator without any intervention.

      If you selected Manual updates, then the OLM creates an update request. As a cluster administrator, you must then manually approve that update request to update the Operator to a newer version.

6. Ensure that the Enable option is selected for the Console plugin.
7. Click Install.

Procedure

1. In the OpenShift Web Console, click Operators Installed Operators to view all the installed operators.

   Ensure that the Project selected is openshift-storage.

2. Click on the OpenShift Data Foundation operator and then click Create StorageSystem.
3. In the Backing storage page, select the Create a new StorageClass using the local storage devices option.

4. Click Next.

   Important

   You are prompted to install the Local Storage Operator if it is not already installed. Click Install, and follow the procedure as described in Installing Local Storage Operator.

5. In the Create local volume set page, provide the following information:

   1. Enter a name for the LocalVolumeSet and the StorageClass.

      By default, the local volume set name appears for the storage class name. You can change the name.

   2. Choose one of the following:

      • Disks on all nodes

        Uses the available disks that match the selected filters on all the nodes.

      • Disks on selected nodes

        Uses the available disks that match the selected filters only on selected nodes.

        Important

        If the nodes selected do not match the OpenShift Data Foundation cluster requirement of an aggregated 30 CPUs and 72 GiB of RAM, a minimal cluster is deployed.

        For minimum starting node requirements, see the Resource requirements section in the Planning guide.

   3. Select SSD or NVMe to build a supported configuration. You can select HDDs for unsupported test installations.

   4. Expand the Advanced section and set the following options:

      Volume Mode	Block is selected by default.
      Device Type	Select one or more device types from the dropdown list.
      Disk Size	Set a minimum size of 100GB for the device and maximum available size of the device that needs to be included.
      Maximum Disks Limit	This indicates the maximum number of PVs that can be created on a node. If this field is left empty, then PVs are created for all the available disks on the matching nodes.

   5. Click Next.

      A pop-up to confirm the creation of LocalVolumeSet is displayed.

   6. Click Yes to continue.

6. In the Capacity and nodes page, configure the following:

   1. Available raw capacity is populated with the capacity value based on all the attached disks associated with the storage class. This takes some time to show up.

      The Selected nodes list shows the nodes based on the storage class.

   2. Select Enable arbiter checkbox if you want to use the stretch clusters. This option is available only when all the prerequisites for arbiter are fulfilled and the selected nodes are populated. For more information, see Arbiter stretch cluster requirements in Requirements for enabling stretch cluster.

      Select the arbiter zone from the dropdown list.

7. Choose a performance profile for Configure performance.

   You can also configure the performance profile after the deployment using the Configure performance option from the options menu of the StorageSystems tab.

   Important

   Before selecting a resource profile, make sure to check the current availability of resources within the cluster. Opting for a higher resource profile in a cluster with insufficient resources might lead to installation failures. For more information about resource requirements, see Resource requirement for performance profiles.

   1. Click Next.

8. Optional: In the Security and network page, configure the following based on your requirement:

   1. To enable encryption, select Enable data encryption for block and file storage.

   2. Select one of the following Encryption level:

      • Cluster-wide encryption to encrypt the entire cluster (block and file).
      • StorageClass encryption to create encrypted persistent volume (block only) using encryption enabled storage class.

   3. Optional: Select the Connect to an external key management service checkbox. This is optional for cluster-wide encryption.

      1. From the Key Management Service Provider drop-down list, either select Vault or Thales CipherTrust Manager (using KMIP). If you selected Vault, go to the next step. If you selected Thales CipherTrust Manager (using KMIP), go to step iii.

      2. Select an Authentication Method.

         Using Token authentication method

         • Enter a unique Connection Name, host Address of the Vault server ('https://<hostname or ip>'), Port number and Token.

         • Expand Advanced Settings to enter additional settings and certificate details based on your Vault configuration:

           • Enter the Key Value secret path in the Backend Path that is dedicated and unique to OpenShift Data Foundation.
           • Optional: Enter TLS Server Name and Vault Enterprise Namespace.
           • Upload the respective PEM encoded certificate file to provide the CA Certificate, Client Certificate and Client Private Key .
           • Click Save and skip to step iv.

         Using Kubernetes authentication method

         • Enter a unique Vault Connection Name, host Address of the Vault server ('https://<hostname or ip>'), Port number and Role name.

         • Expand Advanced Settings to enter additional settings and certificate details based on your Vault configuration:

           • Enter the Key Value secret path in the Backend Path that is dedicated and unique to OpenShift Data Foundation.
           • Optional: Enter TLS Server Name and Authentication Path if applicable.
           • Upload the respective PEM encoded certificate file to provide the CA Certificate, Client Certificate and Client Private Key .
           • Click Save and skip to step iv.

      3. To use Thales CipherTrust Manager (using KMIP) as the KMS provider, follow the steps below:

         1. Enter a unique Connection Name for the Key Management service within the project.

         2. In the Address and Port sections, enter the IP of Thales CipherTrust Manager and the port where the KMIP interface is enabled. For example:

            • Address: 123.34.3.2
            • Port: 5696
         3. Upload the Client Certificate, CA certificate, and Client Private Key.
         4. If StorageClass encryption is enabled, enter the Unique Identifier to be used for encryption and decryption generated above.
         5. The TLS Server field is optional and used when there is no DNS entry for the KMIP endpoint. For example, kmip_all_<port>.ciphertrustmanager.local.

   4. Network is set to Default (OVN) if you are using a single network.

      You can switch to Custom (Multus) if you are using multiple network interfaces and then choose any one of the following:

      1. Select a Public Network Interface from the dropdown.
      2. Select a Cluster Network Interface from the dropdown.
   Note

   If you are using only one additional network interface, select the single NetworkAttachementDefinition, that is, ocs-public-cluster for the Public Network Interface, and leave the Cluster Network Interface blank.

   1. Click Next.
9. In the Data Protection page, click Next.

10. In the Review and create page, review the configuration details.

    To modify any configuration settings, click Back to go back to the previous configuration page.

11. Click Create StorageSystem.

1. Create a new project.

   $ oc new-project my-shared-storage

2. Deploy the example PHP application called file-uploader.

   $ oc new-app openshift/php:latest~https://github.com/mashetty330/openshift-php-upload-demo --name=file-uploader

   Example Output:

   Found image 4f2dcc0 (9 days old) in image stream "openshift/php" under tag "7.2-ubi8" for "openshift/php:7.2-
   ubi8"
   
   Apache 2.4 with PHP 7.2
   -----------------------
   PHP 7.2 available as container is a base platform for building and running various PHP 7.2 applications and frameworks. PHP is an HTML-embedded scripting language. PHP attempts to make it easy for developers to write dynamically generated web pages. PHP also offers built-in database integration for several commercial and non-commercial database management systems, so writing a database-enabled webpage with PHP is fairly simple. The most common
   use of PHP coding is probably as a replacement for CGI scripts.
   
   Tags: builder, php, php72, php-72
   
   * A source build using source code from https://github.com/christianh814/openshift-php-upload-demo will be cr
   eated
   * The resulting image will be pushed to image stream tag "file-uploader:latest"
   * Use 'oc start-build' to trigger a new build
   
   --> Creating resources ...
       imagestream.image.openshift.io "file-uploader" created
       buildconfig.build.openshift.io "file-uploader" created
       deployment.apps "file-uploader" created
       service "file-uploader" created
   --> Success
       Build scheduled, use 'oc logs -f buildconfig/file-uploader' to track its progress.
   
       Application is not exposed. You can expose services to the outside world by executing one or more of the commands below:
        'oc expose service/file-uploader'
   
       Run 'oc status' to view your app.

3. View the build log and wait until the application is deployed.

   $ oc logs -f bc/file-uploader -n my-shared-storage

   Example Output:

   Cloning "https://github.com/christianh814/openshift-php-upload-demo" ...
   
       [...]
       Generating dockerfile with builder image image-registry.openshift-image-regis
   try.svc:5000/openshift/php@sha256:d97466f33999951739a76bce922ab17088885db610c
   0e05b593844b41d5494ea
   STEP 1: FROM image-registry.openshift-image-registry.svc:5000/openshift/php@s
   ha256:d97466f33999951739a76bce922ab17088885db610c0e05b593844b41d5494ea
   STEP 2: LABEL "io.openshift.build.commit.author"="Christian Hernandez <christ
   ian.hernandez@yahoo.com>"       "io.openshift.build.commit.date"="Sun Oct 1 1
   7:15:09 2017 -0700"       "io.openshift.build.commit.id"="288eda3dff43b02f7f7
   b6b6b6f93396ffdf34cb2"       "io.openshift.build.commit.ref"="master"       "
   io.openshift.build.commit.message"="trying to modularize"       "io.openshift
   .build.source-location"="https://github.com/christianh814/openshift-php-uploa
   d-demo"       "io.openshift.build.image"="image-registry.openshift-image-regi
   stry.svc:5000/openshift/php@sha256:d97466f33999951739a76bce922ab17088885db610
   c0e05b593844b41d5494ea"
   STEP 3: ENV OPENSHIFT_BUILD_NAME="file-uploader-1"     OPENSHIFT_BUILD_NAMESP
   ACE="my-shared-storage"     OPENSHIFT_BUILD_SOURCE="https://github.com/christ
   ianh814/openshift-php-upload-demo"     OPENSHIFT_BUILD_COMMIT="288eda3dff43b0
   2f7f7b6b6b6f93396ffdf34cb2"
   STEP 4: USER root
   STEP 5: COPY upload/src /tmp/src
   STEP 6: RUN chown -R 1001:0 /tmp/src
   STEP 7: USER 1001
   STEP 8: RUN /usr/libexec/s2i/assemble
   ---> Installing application source...
   => sourcing 20-copy-config.sh ...
   ---> 17:24:39     Processing additional arbitrary httpd configuration provide
   d by s2i ...
   => sourcing 00-documentroot.conf ...
   => sourcing 50-mpm-tuning.conf ...
   => sourcing 40-ssl-certs.sh ...
   STEP 9: CMD /usr/libexec/s2i/run
   STEP 10: COMMIT temp.builder.openshift.io/my-shared-storage/file-uploader-1:3
   b83e447
   Getting image source signatures
   
   [...]

   The command prompt returns out of the tail mode after you see Push successful.

   Note

   The new-app command deploys the application directly from the git repository and does not use the OpenShift template, hence the OpenShift route resource is not created by default. You need to create the route manually.

Scaling the application

1. Scale the application to four replicas and expose its services to make the application zone aware and available.

   $ oc expose svc/file-uploader -n my-shared-storage

   $ oc scale --replicas=4 deploy/file-uploader -n my-shared-storage

   $ oc get pods -o wide -n my-shared-storage

   You should have four file-uploader pods in a few minutes. Repeat the above command until there are 4 file-uploader pods in the Running status.

2. Create a PVC and attach it into an application.

   $ oc set volume deploy/file-uploader --add --name=my-shared-storage \
   -t pvc --claim-mode=ReadWriteMany --claim-size=10Gi \
   --claim-name=my-shared-storage --claim-class=ocs-storagecluster-cephfs \
   --mount-path=/opt/app-root/src/uploaded \
   -n my-shared-storage

   This command:

   • Creates a PVC.
   • Updates the application deployment to include a volume definition.
   • Updates the application deployment to attach a volume mount into the specified mount-path.
   • Creates a new deployment with the four application pods.

3. Check the result of adding the volume.

   $ oc get pvc -n my-shared-storage

   Example Output:

   NAME                STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS                AGE
   my-shared-storage   Bound    pvc-5402cc8a-e874-4d7e-af76-1eb05bd2e7c7   10Gi       RWX            ocs-storagecluster-cephfs   52s

   Notice the ACCESS MODE is set to RWX.

   All the four file-uploader pods are using the same RWX volume. Without this access mode, OpenShift does not attempt to attach multiple pods to the same Persistent Volume (PV) reliably. If you attempt to scale up the deployments that are using ReadWriteOnce (RWO) PV, the pods may get colocated on the same node.