Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Work with Shared Resources

builds for Red Hat OpenShift 1.2

Using Shared Resources CSI Driver Operator

Red Hat OpenShift Documentation Team

Abstract

This document provides procedural examples for using Shared Resources CSI Driver.

Chapter 1. Shared Resource CSI Driver Operator

As a cluster administrator, you can use the Shared Resource Container Storage Interface (CSI) Driver in any Kubernetes object that uses the volume type as csi to provision inline ephemeral volumes that contain the contents of Secret or ConfigMap objects. This way, pods and other Kubernetes types that expose volume mounts, and OpenShift Container Platform Builds can securely use the contents of those objects across potentially any namespace in the cluster. To accomplish this, there are currently two types of shared resources:

  • SharedSecret custom resource for Secret objects
  • SharedConfigMap custom resource for ConfigMap objects

1.1. About CSI

Storage vendors have traditionally provided storage drivers as part of Kubernetes. With the implementation of the Container Storage Interface (CSI), third-party providers can deliver storage plugins by using a standard interface without changing the core Kubernetes code. CSI Operators give Builds users storage options, such as volume snapshots, that are not possible with in-tree volume plugins.

1.2. Sharing secrets across namespaces

To share a secret across namespaces in a cluster, you create a SharedSecret custom resource (CR) instance for the Secret object.

Prerequisites

  • You have created a Secret object that you want to share in other namespaces.

  • You must have permissions to perform the following actions:

    • You subscribed to the cluster and have entitlement keys synced through the Insights Operator.
    • You created the instances of the sharedsecrets.sharedresource.openshift.io custom resource definition (CRD) at a cluster-scoped level.
    • You created the ClusterRole object for the SharedConfigMap CR.
    • You created the Role and RoleBinding objects for the Shared Resource CSI Driver.
    • You managed roles and role bindings across the namespaces in the cluster to control users.
    • You managed roles and role bindings to control whether the service account specified by a pod can mount a Container Storage Interface (CSI) volume that references the Secret or Configmap CR you want to use.
    • You have access to the namespaces that contain the secrets you want to share.

Procedure

  1. Create a SharedSecret instance to share the referenced Secret across namespaces in the cluster by using the following example configuration: 

    apiVersion: sharedresource.openshift.io/v1alpha1
kind: SharedSecret
metadata:
  name: shared-test-secret 1
spec:
  secretRef:
    name: test-secret
    namespace: <name_of_the_source_namespace>
    1
    Defines the name of the SharedSecret CR.

  2. Create a ClusterRole object that grants RBAC permission to use the referenced shared resource by using the following example configuration: 

    . apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: use-shared-test-secret 1
rules:
  - apiGroups:
      - sharedresource.openshift.io
    resources:
      - sharedsecrets
    resourceNames:
      - shared-test-secret
    verbs:
      - use
    1
    Defines the name of the ClusterRole CR.

  3. Create the Role and RoleBinding objects that grant the Shared Resources CSI driver the permission to access the SharedSecret object:

    Example Role object

    apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: share-etc-pki-entitlement 1
  namespace: openshift-config-managed
rules:
  - apiGroups:
      - ""
    resources:
      - secrets
    resourceNames:
      - etc-pki-entitlement
    verbs:
      - get
      - list
      - watch

    1
    Defines the name of the Role CR.

    Example RoleBinding object

    apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: share-etc-pki-entitlement 1
  namespace: openshift-config-managed
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: share-etc-pki-entitlement 2
subjects:
  - kind: ServiceAccount
    name: csi-driver-shared-resource 3
    namespace: openshift-builds

    1
    Defines the name of the RoleBinding CR.
    2
    Defines the name of the Role CR.
    3
    Defines the name of the ServiceAccount CR.

1.3. Using a SharedSecret instance in a pod

To access the SharedSecret custom resource (CR) from a pod, you grant Role-based access control (RBAC) permissions to a service account.

Prerequisites

  • You have created a SharedConfigMap CR instance for the config map that you want to share across namespaces in the cluster.

  • You must have permissions to perform the following actions:

    • Get the list of the SharedConfigMap CR instances by entering the oc get sharedconfigmaps command.
    • Check if a service account is allowed to use the SharedSecret CR and the service account is listed in your namespace by running the oc adm policy who-can use <sharedsecret_identifier> command.
    • Confirm that the service account of your pod is allowed to use csi volumes. If you created the pod as a user, confirm that you are allowed to use csi volumes.
Note

If you are unable to complete the last two prerequisites, the cluster administrator can establish the necessary role-based access control (RBAC) so that you can enable service accounts to use the SharedConfigMap CR.

Procedure

  1. Create the RoleBinding object associated with the role and grant the permission to your service account to use the shared resource. See the following example configuration: 

    apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: use-shared-secret 1
  namespace: app-namespace
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: use-shared-secret
subjects:
  - kind: ServiceAccount
    name: <service_account_name> 2
    1
    Defines the name of the RoleBinding CR.
    2
    Specify the service account name of your application.

  2. Mount the shared resource csi driver into a pod or any other resource that accepts csi volumes. See the following example configuration: 

    apiVersion: v1
kind: Pod
metadata:
  name: example-shared-secret
  namespace: <app_namespace> 1
spec:
  ...
  serviceAccountName: default
  volumes:
    - name: shared-secret
      csi:
        readOnly: true
        driver: csi.sharedresource.openshift.io
        volumeAttributes:
          sharedSecret: shared-test-secret 2
    1
    Replace <app_namespace> with the namespace name of your application.
    2
    Defines the value of the sharedSecret attribute that must be same as the name of the SharedSecret CR.

1.4. Sharing a config map across namespaces

To share a config map across namespaces in a cluster, you create a SharedConfigMap custom resource (CR) instance.

Prerequisites

You must have permissions to perform the following actions:

  • Create sharedconfigmaps.sharedresource.openshift.io custom resource definition (CRD) at a cluster-scoped level.
  • Create a ClusterRole object for the SharedConfigMap CR.
  • Create role and role binding for the Shared Resource Container Storage Interface (CSI) Driver.
  • Manage roles and role bindings across the namespaces in the cluster to control which users can get, list, and watch those instances.
  • Manage roles and role bindings across the namespaces in the cluster to control which service accounts in pods that mount your CSI volume can use those instances.
  • Access the namespaces that contain the secrets you want to share.

Procedure

  1. Create a SharedConfigMap CR instance for the config map that you want to share across namespaces in the cluster. See the following example configuration: 

    apiVersion: sharedresource.openshift.io/v1alpha1
kind: SharedConfigMap
metadata:
  name: share-test-config 1
spec:
  configMapRef:
    name: shared-config
    namespace: <name_of_the_source_namespace> 2
    1
    Defines the name of the SharedConfigMap CR.
    2
    Replace <name_of_the_source_namespace> with the name of your source namespace.

  2. Create a ClusterRole CR instance that grants Role-based access control (RBAC) permission to use the referenced shared resource by using the following example configuration: 

    apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: <cluster_role_name> 1
rules:
  - apiGroups:
      - sharedresource.openshift.io
    resources:
      - sharedconfigmaps
    resourceNames:
      - share-test-config 2
    verbs:
      - use
    1
    Replace <cluster_role_name> with the name of your cluster role.
    2
    Defines the name of the SharedSecret CR.

  3. Create a Role and RoleBinding object to grant the CSI driver permission to access the config map:

    Example Role object

    apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: shared-test-config
  namespace: test-share-source 1
rules:
  - apiGroups: [""]
    resources: ["configmaps"]
    resourceNames: ["shared-config"]
    verbs: ["get", "list", "watch"]

    1
    Defines the name of the source namespace.

    Example RoleBinding object

     apiVersion: rbac.authorization.k8s.io/v1
 kind: RoleBinding
 metadata:
   name: shared-test-config
   namespace: test-share-source 1
 roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: Role
   name: shared-test-config
 subjects: 2
   - kind: ServiceAccount
     name: csi-driver-shared-resource
     namespace: openshift-builds

    1
    Defines the name of the source namespace.
    2
    Defines a list of service accounts for the Shared Resource CSI driver DaemonSet. When deployed with Builds for OpenShift, the service account name is csi-driver-shared-resource, and the namespace is same where the Builds for OpenShift Operator is deployed.

1.5. Using a SharedConfigMap instance in a pod

To access a SharedConfigMap custom resource (CR) instance from a pod, you grant a given service account role-based access control (RBAC) permissions to use that SharedConfigMap CR instance.

Prerequisites

  • You have created a SharedConfigMap CR instance for the config map that you want to share across namespaces in the cluster.

  • You must have permissions to perform the following actions:

    • Get the list of the SharedConfigMap CR instances by entering the oc get sharedconfigmaps command.
    • Check if a service account is allowed to use the SharedSecret CR and the service account is listed in your namespace by running the oc adm policy who-can use <sharedsecret_identifier> command.
    • Confirm that the service account of your pod is allowed to use csi volumes. If you created the pod as a user, confirm that you are allowed to use csi volumes.
Note

If you are unable to complete the last two prerequisites, the cluster administrator can establish the necessary role-based access control (RBAC) so that you can enable service accounts to use the SharedConfigMap CR.

Procedure

  1. Create the RoleBinding object associated with the role and grant the permission to your service account to use the shared resource. See the following example configuration: 

    apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: use-shared-config
  namespace: <app_namespace> 1
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: use-shared-config
subjects:
  - kind: ServiceAccount
    name: <service_account_name> 2
    1
    Replace <app_namespace> with the namespace name of your application.
    2
    Specify the service account name of your application.

  2. Mount the shared resource csi driver into a pod or any other resource that accepts csi volumes. See the following example configuration: 

    apiVersion: v1
kind: Pod
metadata:
  name: example-shared-config
  namespace: <app_namespace> 1
spec:
  ...
  serviceAccountName: default
  volumes:
    - name: shared-config
      csi:
        readOnly: true
        driver: csi.sharedresource.openshift.io
        volumeAttributes:
          sharedConfigMap: share-test-config 2
    1
    Replace <app_namespace> with the namespace name of your application.
    2
    Defines the name of the sharedConfigMap object.

1.6. Conditions to validate volumeAttributes

In the volumeAttributes field of a single volume, you must set either a sharedSecret or a sharedConfigMap attribute to the value of a SharedSecret or a SharedConfigMap instance. Otherwise, when the volume is provisioned during pod startup, a validation check returns an error to the kubelet. Review the following conditions to provision a shared resource volume for a pod:

  • You must specify values for the sharedSecret and sharedConfigMap attributes.
  • The value of the sharedSecret and sharedConfigMap attributes must match the name of their custom resources (CRs).

1.7. Building images using the OpenShift subscription entitlements

Integration between shared resources, the Insights Operator, and builds for Red Hat OpenShift Builds makes using Red Hat subscriptions (RHEL entitlements) easier in builds for Red Hat OpenShift Builds.

In OpenShift Container Platform 4.10 and later, builds for Red Hat OpenShift can use Red Hat subscriptions (RHEL entitlements) by referencing shared resources and the simple content access feature provided by Insights Operator:

  • The simple content access feature imports your subscription credentials to a well-known Secret object.
  • The cluster administrator creates a SharedSecret custom resource (CR) with the Secret object and grants permission to projects or namespaces. The cluster administrator gives the builder service account permission to use the SharedSecret CR.
  • Builds that run within those projects or namespaces can mount a Container Storage Interface (CSI) Volume that references the SharedSecret CR instance and its entitled RHEL content.

1.8. Running builds using SharedSecret objects

You can use a SharedSecret object to securely access the RHEL entitlement keys of a cluster in builds. The SharedSecret object allows you to share and synchronize secrets across namespaces.

Prerequisites

  • You must have permissions to perform the following actions:

    • Create build configs and start builds.
    • Create SharedSecret object.
    • Discover which SharedSecret custom resource (CR) instances are available by entering the oc get sharedsecrets command and getting a non-empty list back.
    • Check if a service account is allowed to use the SharedSecret CR and the service account is listed in your namespace by running the oc adm policy who-can use <sharedsecret_identifier> command.
Note

If you are unable to complete the last two prerequisites, the cluster administrator can establish the necessary role-based access control (RBAC) so that you can grant service accounts to use the SharedSecret CR.

Procedure

  1. Create a SharedSecret object instance with the entitlement secret of a cluster by running the following command: 

    $ oc apply -f -<<EOF
apiVersion: sharedresource.openshift.io/v1alpha1
kind: SharedSecret
metadata:
  name: etc-pki-entitlement
spec:
  secretRef:
    name: etc-pki-entitlement
    namespace: openshift-config-managed
EOF

  2. Create a ClusterRole object to grant permission to access the SharedSecret object by using the following example configuration:: 

    apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: use-share-etc-pki-entitlement 1
rules:
  - apiGroups:
      - sharedresource.openshift.io
    resources:
      - sharedsecrets
    resourceNames:
      - etc-pki-entitlement
    verbs:
      - use
    1
    Defines the name of the ClusterRole CR.

  3. Create the Role and RoleBinding object that grant the Shared Resources CSI driver the permission to access the SharedSecret object:

    Example Role object

    apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: share-etc-pki-entitlement 1
  namespace: openshift-config-managed
rules:
  - apiGroups:
      - ""
    resources:
      - secrets
    resourceNames:
      - etc-pki-entitlement
    verbs:
      - get
      - list
      - watch

    1
    Defines the name of the Role CR.

    Example RoleBinding object

    apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: share-etc-pki-entitlement 1
  namespace: openshift-config-managed
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: share-etc-pki-entitlement
subjects:
  - kind: ServiceAccount
    name: csi-driver-shared-resource
    namespace: openshift-builds 2

    1
    Defines the name of the RoleBinding CR.
    2
    Defines the name of the namespace where openshift-builds is installed.

  4. Create a RoleBinding object for the builder and pipeline service accounts in the namespace where builds run: 

    apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: use-share-etc-pki-entitlement 1
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: use-share-etc-pki-entitlement
subjects:
  - kind: ServiceAccount
    name: pipeline
  - kind: ServiceAccount
    name: builder
    1
    Defines the name of the RoleBinding CR for the builder and pipeline service accounts.
    Note

    The service account that consumes the SharedSecret object is created and managed by the OpenShift controllers.

  5. Mount the SharedSecret object by using the buildah build strategy. See the following example: 

    $ oc apply -f -<<EOF
apiVersion: shipwright.io/v1beta1
kind: Build
metadata:
  name: buildah-rhel
spec:
  source:
    type: Git
      git:
      url: https://github.com/redhat-openshift-builds/samples
    contextDir: buildah-build
  strategy:
    name: buildah
    kind: ClusterBuildStrategy
  paramValues:
  - name: dockerfile
    value: DockerFile
  volumes:
  - csi:
      driver: csi.sharedresource.openshift.io
      readOnly: true 1
      volumeAttributes:
        sharedSecret:  <sharedsecret_object_name> 2
    name: etc-pki-entitlement
  output:
    image: <output_image_location> 3
EOF
1
You must set readOnly to true to mount the shared resource in the build.
2
Specify the name of the SharedSecret object to include it in the build.
3
Specify the location where you want to push the built image.

Legal Notice

Copyright © 2024 Red Hat, Inc.

OpenShift documentation is licensed under the Apache License 2.0 (https://www.apache.org/licenses/LICENSE-2.0).

Modified versions must remove all Red Hat trademarks.

Portions adapted from https://github.com/kubernetes-incubator/service-catalog/ with modifications by Red Hat.

Red Hat, Red Hat Enterprise Linux, the Red Hat logo, the Shadowman logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.

Linux® is the registered trademark of Linus Torvalds in the United States and other countries.

Java® is a registered trademark of Oracle and/or its affiliates.

XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.

MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.

Node.js® is an official trademark of Joyent. Red Hat Software Collections is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.

The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation’s permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.

All other trademarks are the property of their respective owners.

Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.