Chapter 3. Preparing the environment for virtualized control planes

Procedure

1. Create the Namespace CR for KubeVirt Redfish by creating a YAML file with content such as the following example:

   apiVersion: v1
   kind: Namespace
   metadata:
     name: kubevirt-redfish
     labels:
       app.kubernetes.io/name: kubevirt-redfish

2. Apply the resource by running the following command:

   $ oc apply -f namespace.yaml

3. Create the ServiceAccount CR by creating a YAML file with content such as the following example:

   apiVersion: v1
   kind: ServiceAccount
   metadata:
     name: kubevirt-redfish
     namespace: kubevirt-redfish
     labels:
       app.kubernetes.io/name: kubevirt-redfish
       app.kubernetes.io/component: rbac

4. Apply the resource by running the following command:

   $ oc apply -f serviceaccount.yaml

5. Create the ClusterRole CR with required permissions by creating a YAML file with content such as the following example:

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRole
   metadata:
     name: kubevirt-redfish-role
     labels:
       app.kubernetes.io/name: kubevirt-redfish
       app.kubernetes.io/component: rbac
   rules:
     - apiGroups: ["kubevirt.io"]
       resources: ["virtualmachines", "virtualmachineinstances"]
       verbs: ["get", "list", "watch", "update", "patch"]
     - apiGroups: ["kubevirt.io"]
       resources: ["virtualmachines/status", "virtualmachineinstances/status"]
       verbs: ["get", "list", "watch", "patch"]
     - apiGroups: ["kubevirt.io"]
       resources: ["virtualmachines/restart", "virtualmachines/start", "virtualmachines/stop"]
       verbs: ["create"]
     - apiGroups: ["subresources.kubevirt.io"]
       resources: ["virtualmachineinstances/pause", "virtualmachineinstances/unpause"]
       verbs: ["create", "update"]
     - apiGroups: [""]
       resources: ["pods", "services", "configmaps", "secrets"]
       verbs: ["get", "list", "watch", "create", "update", "delete"]
     - apiGroups: [""]
       resources: ["namespaces"]
       verbs: ["get", "list"]
     - apiGroups: ["cdi.kubevirt.io"]
       resources: ["datavolumes", "volumeimportsources"]
       verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
     - apiGroups: [""]
       resources: ["persistentvolumeclaims"]
       verbs: ["get", "list", "create", "update", "patch", "delete", "watch"]
     - apiGroups: ["storage.k8s.io"]
       resources: ["storageclasses"]
       verbs: ["get", "list"]

6. Apply the resource by running the following command:

   $ oc apply -f clusterrole.yaml

7. Create the ClusterRoleBinding CR by creating a YAML file with content such as the following example:

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRoleBinding
   metadata:
     name: kubevirt-redfish-binding
     labels:
       app.kubernetes.io/name: kubevirt-redfish
       app.kubernetes.io/component: rbac
   roleRef:
     apiGroup: rbac.authorization.k8s.io
     kind: ClusterRole
     name: kubevirt-redfish-role
   subjects:
     - kind: ServiceAccount
       name: kubevirt-redfish
       namespace: kubevirt-redfish

8. Apply the resource by running the following command:

   $ oc apply -f clusterrolebinding.yaml

9. Create the Secret CR containing the configuration by creating a YAML file with content such as the following example. Edit the config.yaml section to match your environment:

   apiVersion: v1
   kind: Secret
   metadata:
     name: kubevirt-redfish-secret
     namespace: kubevirt-redfish
     labels:
       app.kubernetes.io/name: kubevirt-redfish
       app.kubernetes.io/component: config
   type: Opaque
   stringData:
     config.yaml: |
       server:
         host: "0.0.0.0"
         port: 8443
         tls:
           enabled: false
       system_id_convention: "enhanced"
       chassis:
         - name: "<chassis_name>"
           namespace: "<vm_namespace>"
           service_account: "kubevirt-redfish"
           vm_selector:
             labels:
               redfish-enabled: "true"
       authentication:
         users:
           - username: "admin"
             password: "<password>"
             chassis: ["<chassis_name>"]
       datavolume:
         storage_class: "<storage_class>"
         storage_size: "3Gi"

   where:

   • system_id_convention specifies the format for Redfish system IDs. The recommended setting is enhanced to use <namespace>.<vm-name> format. The legacy setting uses <vm-name> only.
   • chassis specifies the namespaces where VMs are deployed. Replace <chassis_name> with a name for this chassis configuration and <vm_namespace> with the namespace containing your VMs. The vm_selector labels identify which VMs in the namespace are exposed through Redfish. Only VMs with matching labels are visible. You can configure multiple chassis entries to expose different subsets of VMs in the same namespace, each with different authentication users.
   • authentication specifies the username and password required to access the Redfish API. These credentials enable full management control over exposed VMs, independently of any OpenShift Container Platform privileges. Replace <password> with a secure password.
   • datavolume specifies storage for VirtualMedia operations. Replace <storage_class> with a storage class available on your cluster, such as lvms-vg1 or ocs-storagecluster-ceph-rbd-virtualization. For more information about storage options, see Storage requirements in "Prerequisites for virtualized control planes".

10. Apply the resource by running the following command:

    $ oc apply -f secret.yaml

    Warning

    The credentials defined in this Secret CR enable full management control over the VMs exposed through KubeVirt Redfish, independently of any OpenShift Container Platform privileges.

11. Create the Deployment CR by creating a YAML file with content such as the following example:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: kubevirt-redfish
      namespace: kubevirt-redfish
      labels:
        app.kubernetes.io/name: kubevirt-redfish
        app.kubernetes.io/component: server
    spec:
      replicas: 1
      selector:
        matchLabels:
          app.kubernetes.io/name: kubevirt-redfish
          app.kubernetes.io/component: server
      template:
        metadata:
          labels:
            app.kubernetes.io/name: kubevirt-redfish
            app.kubernetes.io/component: server
        spec:
          serviceAccountName: kubevirt-redfish
          securityContext:
            runAsNonRoot: true
          containers:
            - name: kubevirt-redfish
              image: registry.redhat.io/container-native-virtualization/kubevirt-redfish-rhel9:v4.22
              imagePullPolicy: Always
              ports:
                - name: http
                  containerPort: 8443
                  protocol: TCP
              env:
                - name: CONFIG_PATH
                  value: "/app/config/config.yaml"
                - name: LOG_LEVEL
                  value: "info"
              resources:
                requests:
                  memory: "512Mi"
                  cpu: "100m"
                limits:
                  memory: "2Gi"
                  cpu: "500m"
              livenessProbe:
                httpGet:
                  path: /redfish/v1/
                  port: 8443
                  scheme: HTTP
                initialDelaySeconds: 30
                periodSeconds: 10
              readinessProbe:
                httpGet:
                  path: /redfish/v1/
                  port: 8443
                  scheme: HTTP
                initialDelaySeconds: 5
                periodSeconds: 5
              securityContext:
                runAsNonRoot: true
                allowPrivilegeEscalation: false
                capabilities:
                  drop:
                    - ALL
              volumeMounts:
                - name: config-volume
                  mountPath: /app/config
                  readOnly: true
          volumes:
            - name: config-volume
              secret:
                secretName: kubevirt-redfish-secret

    where:

    • The image field specifies the KubeVirt Redfish container image.

12. Apply the resource by running the following command:

    $ oc apply -f deployment.yaml

13. Create the Service CR by creating a YAML file with content such as the following example:

    apiVersion: v1
    kind: Service
    metadata:
      name: kubevirt-redfish
      namespace: kubevirt-redfish
      labels:
        app.kubernetes.io/name: kubevirt-redfish
        app.kubernetes.io/component: server
    spec:
      type: ClusterIP
      ports:
        - name: http
          port: 8443
          targetPort: 8443
          protocol: TCP
      selector:
        app.kubernetes.io/name: kubevirt-redfish
        app.kubernetes.io/component: server

14. Apply the resource by running the following command:

    $ oc apply -f service.yaml

15. Create the Route CR to expose the Redfish API externally by creating a YAML file with content such as the following example:

    apiVersion: route.openshift.io/v1
    kind: Route
    metadata:
      name: kubevirt-redfish
      namespace: kubevirt-redfish
      labels:
        app.kubernetes.io/name: kubevirt-redfish
        app.kubernetes.io/component: server
    spec:
      port:
        targetPort: http
      to:
        kind: Service
        name: kubevirt-redfish
        weight: 100
      tls:
        termination: edge
        insecureEdgeTerminationPolicy: Redirect

16. Apply the resource by running the following command:

    $ oc apply -f route.yaml

Verification

1. Verify that the pods are running by running the following command:

   $ oc get pods -n kubevirt-redfish

   Example output

   NAME                                READY   STATUS    RESTARTS   AGE
   kubevirt-redfish-587cd94988-xthml   1/1     Running   0          2m

2. Get the route hostname by running the following command:

   $ oc get route kubevirt-redfish -n kubevirt-redfish -o jsonpath='{.spec.host}'

3. Test the Redfish endpoint by running the following command:

   $ curl -sk -u "admin:<password>" https://<route_hostname>/redfish/v1/

   A successful response returns JSON with the Redfish service root:

   {
     "@odata.id": "/redfish/v1",
     "@odata.type": "#ServiceRoot.v1_0_0.ServiceRoot",
     "Id": "RootService",
     "Name": "Root Service",
     "Systems": {
       "@odata.id": "/redfish/v1/Systems"
     }
   }

Procedure

1. Enable the RebootPolicy feature gate on the hosting cluster by running the following command:

   $ oc annotate --overwrite -n openshift-cnv hyperconverged kubevirt-hyperconverged \
       kubevirt.kubevirt.io/jsonpatch='[{"op":"add","path":"/spec/configuration/developerConfiguration/featureGates/-","value":"RebootPolicy"}]'

   Note

   The RebootPolicy feature gate enables the rebootPolicy field in VirtualMachine specifications. This configuration is required when using KubeVirt Redfish for cluster installation. The feature gate is enabled through an annotation on the HyperConverged resource, which propagates the configuration to the underlying KubeVirt CR.

2. Enable the declarativeHotplugVolumes feature gate on the hosting cluster by running the following command:

   $ oc patch hyperconverged kubevirt-hyperconverged -n openshift-cnv \
       --type merge \
       -p '{"spec": {"featureGates": {"declarativeHotplugVolumes": true}}}'

   Note

   The declarativeHotplugVolumes feature gate enables KubeVirt Redfish to dynamically attach boot media to VMs through the Redfish API. This configuration is required when using KubeVirt Redfish for cluster installation.

3. Create a VirtualMachine CR for each control plane node by creating a YAML file with content such as the following example:

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     name: master-0
     namespace: <vm_namespace>
     labels:
       redfish-enabled: "true"
   spec:
     runStrategy: Halted
     template:
       metadata:
         labels:
           redfish-enabled: "true"
       spec:
         domain:
           rebootPolicy: Terminate
           cpu:
             cores: 8
           memory:
             guest: 16Gi
           devices:
             disks:
               - name: rootdisk
                 disk:
                   bus: virtio
               - name: cloudinitdisk
                 disk:
                   bus: virtio
             interfaces:
               - name: default
                 bridge: {}
         networks:
           - name: default
             multus:
               networkName: <network_attachment_definition>
         volumes:
           - name: rootdisk
             dataVolume:
               name: master-0-disk
           - name: cloudinitdisk
             cloudInitNoCloud:
               userData: |
                 #cloud-config
                 hostname: master-0
                 user: core

   where:

   • <vm_namespace> specifies the namespace for the VMs. Must match the namespace specified in the KubeVirt Redfish chassis configuration.
   • redfish-enabled: "true" specifies the label that must match the vm_selector labels in the KubeVirt Redfish configuration so the VM is exposed through the Redfish API.
   • runStrategy: Halted specifies that VMs must be powered off initially. The installation powers them on by using the Redfish API.
   • rebootPolicy: Terminate specifies the reboot behavior required for Redfish API boot override operations. Ensures the VM terminates cleanly when boot media changes.
   • cores: 8 and guest: 16Gi specify the minimum recommended resources for control plane nodes.

   • <network_attachment_definition> specifies the name of a NetworkAttachmentDefinition configured on your hosting cluster. All control plane VMs must share the same L2 network segment. Common options include localnet, Linux bridge, or OVN Layer 2 networks.

     Important

     For production deployments, configure anti-affinity rules to ensure control plane VMs are distributed across different physical nodes. This prevents a single node failure from affecting multiple control plane VMs simultaneously. Add pod anti-affinity rules or topology spread constraints to the VM specification based on your environment requirements.

4. Apply the resource by running the following command:

   $ oc apply -f master-0.yaml