Upgrading

Procedure

1. In the OpenShift Container Platform web console, go to the RHACS Operator page.
2. In the top navigation menu, select Secured Cluster.
3. Click the instance name, for example, stackrox-secured-cluster-services.

4. Use one of the following methods to change the setting:

   • In the Form view, under Per Node Settings → Collector Settings → Collection, select CORE_BPF.
   • Click YAML to open the YAML editor and locate the spec.perNode.collector.collection attribute. If the value is KernelModule or EBPF, then change it to CORE_BPF.
5. Click Save.

Procedure

1. Delete the Operator Lifecycle Manager (OLM) subscription and cluster service version (CSV):

   Important

   If you use Kubernetes, enter kubectl instead of oc.

   1. To delete the OLM subscription, run the following command:

      $ oc -n rhacs-operator delete subscription rhacs-operator

      Copy to Clipboard Toggle word wrap

      Example output

      subscription.operators.coreos.com "rhacs-operator" deleted

      Copy to Clipboard Toggle word wrap

   2. To delete the CSV, run the following command:

      $ oc -n rhacs-operator delete csv -l operators.coreos.com/rhacs-operator.rhacs-operator

      Copy to Clipboard Toggle word wrap

      Example output

      clusterserviceversion.operators.coreos.com "rhacs-operator.v4.8.4" deleted

      Copy to Clipboard Toggle word wrap

2. Install the previous version of the RHACS Operator:

   1. In the OpenShift web console, click OperatorHub.
   2. Search for Advanced Cluster Security for Kubernetes.

   3. Select and install the previous version of the Operator.

      Note

      Installing the previous Operator version rolls back RHACS to the same version as the Operator.

Procedure

1. In the OpenShift web console, click Operators → Installed Operators.
2. From the list of projects, select rhacs-operator.

3. Locate the Advanced Cluster Security for Kubernetes Operator:

   1. Click the overflow menu → Uninstall Operator.

      The uninstall Operator dialog is displayed.

   2. Ensure that the Delete all operand instances for this operator checkbox is clear to avoid uninstallation of Red Hat Advanced Cluster Security for Kubernetes (RHACS).
   3. Click Uninstall.

4. Install the previous version of RHACS Operator:

   1. In the OpenShift web console, click OperatorHub.
   2. Search for Advanced Cluster Security for Kubernetes.

   3. Select and install the previous version of the Operator.

      Note

      Installing the previous Operator version rolls back RHACS to the same version as the Operator.

1. Check the status of the central-db pod:

   $ oc -n <namespace> get pod -l app=central-db 

   1

   Copy to Clipboard Toggle word wrap

   1

   If you use Kubernetes, enter kubectl instead of oc.

2. If the status of the pod is Pending, use the describe command to get more details:

   $ oc -n <namespace> describe po/<central-db-pod-name> 

   1

   Copy to Clipboard Toggle word wrap

   1

   If you use Kubernetes, enter kubectl instead of oc.

3. You might see the FailedScheduling warning message:

   Type     Reason            Age   From               Message
   ----     ------            ----  ----               -------
   Warning  FailedScheduling  54s   default-scheduler  0/7 nodes are available: 1 Insufficient memory, 3 node(s) had untolerated taint {node-role.kubernetes.io/master: }, 4 Insufficient cpu. preemption: 0/7 nodes are available: 3 Preemption is not helpful for scheduling, 4 No preemption victims found for incoming pod.

   Copy to Clipboard Toggle word wrap

4. This warning message suggests that the scheduled node had insufficient memory to accommodate the pod’s resource requirements. If you have a small environment, consider increasing resources on the nodes or adding a larger node that can support the database.

   Otherwise, consider decreasing the resource requirements for the central-db pod in the custom resource under central → db → resources. However, running central with fewer resources than the recommended minimum might lead to degraded performance for RHACS.

Procedure

1. Determine the roxctl architecture for the target operating system:

   $ arch="$(uname -m | sed "s/x86_64//")"; arch="${arch:+-$arch}"

   Copy to Clipboard Toggle word wrap

2. Download the roxctl CLI:

   $ curl -L -f -o roxctl "https://mirror.openshift.com/pub/rhacs/assets/4.7.6/bin/Linux/roxctl${arch}"

   Copy to Clipboard Toggle word wrap

3. Make the roxctl binary executable:

   $ chmod +x roxctl

   Copy to Clipboard Toggle word wrap

4. Place the roxctl binary in a directory that is on your PATH:

   To check your PATH, execute the following command:

   $ echo $PATH

   Copy to Clipboard Toggle word wrap

Procedure

1. Determine the roxctl architecture for the target operating system:

   $ arch="$(uname -m | sed "s/x86_64//")"; arch="${arch:+-$arch}"

   Copy to Clipboard Toggle word wrap

2. Download the roxctl CLI:

   $ curl -L -f -o roxctl "https://mirror.openshift.com/pub/rhacs/assets/4.7.6/bin/Darwin/roxctl${arch}"

   Copy to Clipboard Toggle word wrap

3. Remove all extended attributes from the binary:

   $ xattr -c roxctl

   Copy to Clipboard Toggle word wrap

4. Make the roxctl binary executable:

   $ chmod +x roxctl

   Copy to Clipboard Toggle word wrap

5. Place the roxctl binary in a directory that is on your PATH:

   To check your PATH, execute the following command:

   $ echo $PATH

   Copy to Clipboard Toggle word wrap

Procedure

1. To update the Central image, run the following command:

   $ oc -n stackrox set image deploy/central \
   central=registry.redhat.io/advanced-cluster-security/rhacs-main-rhel8:4.7.6

   Copy to Clipboard Toggle word wrap

2. To update the Central-db image, run the following command:

   $ oc -n stackrox set image deploy/central-db \
   central-db=registry.redhat.io/advanced-cluster-security/rhacs-central-db-rhel8:4.7.6 \
   init-db=registry.redhat.io/advanced-cluster-security/rhacs-central-db-rhel8:4.7.6

   Copy to Clipboard Toggle word wrap

3. To update the config controller image, run the following command:

   $ oc -n stackrox set image deploy/config-controller \
   manager=registry.redhat.io/advanced-cluster-security/rhacs-main-rhel8:4.7.6

   Copy to Clipboard Toggle word wrap

Procedure

1. If you have created custom Scanner configurations, you must apply these changes before updating the Scanner configuration file:

   1. To generate Scanner, run the following command:

      $ roxctl -e "$ROX_CENTRAL_ADDRESS" scanner generate

      Copy to Clipboard Toggle word wrap

   2. To apply the TLS secrets YAML file, run the following command:

      $ oc apply -f scanner-bundle/scanner/02-scanner-03-tls-secret.yaml

      Copy to Clipboard Toggle word wrap

   3. To apply the Scanner configuration YAML file, run the following command:

      $ oc apply -f scanner-bundle/scanner/02-scanner-04-scanner-config.yaml

      Copy to Clipboard Toggle word wrap

2. To update the Scanner image, run the following command:

   $ oc -n stackrox set image deploy/scanner \
   scanner=registry.redhat.io/advanced-cluster-security/rhacs-scanner-rhel8:4.7.6

   Copy to Clipboard Toggle word wrap

3. To update the Scanner database image, run the following command:

   $ oc -n stackrox set image deploy/scanner-db \
   db=registry.redhat.io/advanced-cluster-security/rhacs-scanner-db-rhel8:4.7.6 \
   init-db=registry.redhat.io/advanced-cluster-security/rhacs-scanner-db-rhel8:4.7.6

   Copy to Clipboard Toggle word wrap

Procedure

1. Update the Sensor image:

   $ oc -n stackrox set image deploy/sensor sensor=registry.redhat.io/advanced-cluster-security/rhacs-main-rhel8:4.7.6 

   1

   Copy to Clipboard Toggle word wrap

   1

   If you use Kubernetes, enter kubectl instead of oc.

2. Update the Compliance image:

   $ oc -n stackrox set image ds/collector compliance=registry.redhat.io/advanced-cluster-security/rhacs-main-rhel8:4.7.6 

   1

   Copy to Clipboard Toggle word wrap

   1

   If you use Kubernetes, enter kubectl instead of oc.

3. Update the Collector image:

   $ oc -n stackrox set image ds/collector collector=registry.redhat.io/advanced-cluster-security/rhacs-collector-rhel8:4.7.6 

   1

   Copy to Clipboard Toggle word wrap

   1

   If you use Kubernetes, enter kubectl instead of oc.

4. Update the admission control image:

   $ oc -n stackrox set image deploy/admission-control admission-control=registry.redhat.io/advanced-cluster-security/rhacs-main-rhel8:4.7.6

   Copy to Clipboard Toggle word wrap

Procedure

1. Patch sensor to ensure POD_NAMESPACE is set by running the following command:

   $ [[ -z "$(oc -n stackrox get deployment sensor -o yaml | grep POD_NAMESPACE)" ]] && oc -n stackrox patch deployment sensor --type=json -p '[{"op":"add","path":"/spec/template/spec/containers/0/env/-","value":{"name":"POD_NAMESPACE","valueFrom":{"fieldRef":{"fieldPath":"metadata.namespace"}}}}]'

   Copy to Clipboard Toggle word wrap

2. Patch admission-control to ensure POD_NAMESPACE is set by running the following command:

   $ [[ -z "$(oc -n stackrox get deployment admission-control -o yaml | grep POD_NAMESPACE)" ]] && oc -n stackrox patch deployment admission-control --type=json -p '[{"op":"add","path":"/spec/template/spec/containers/0/env/-","value":{"name":"POD_NAMESPACE","valueFrom":{"fieldRef":{"fieldPath":"metadata.namespace"}}}}]'

   Copy to Clipboard Toggle word wrap

Procedure

1. List all of the RHACS services that are deployed on Central and all secured clusters:

   $ oc -n stackrox describe pods | grep 'openshift.io/scc\|^Name:'

   Copy to Clipboard Toggle word wrap

   Example output

   Name:      admission-control-6f4dcc6b4c-2phwd
              openshift.io/scc: stackrox-admission-control
   #...
   Name:      central-575487bfcb-sjdx8
              openshift.io/scc: stackrox-central
   Name:      central-db-7c7885bb-6bgbd
              openshift.io/scc: stackrox-central-db
   Name:      collector-56nkr
              openshift.io/scc: stackrox-collector
   #...
   Name:      scanner-68fc55b599-f2wm6
              openshift.io/scc: stackrox-scanner
   Name:      scanner-68fc55b599-fztlh
   #...
   Name:      sensor-84545f86b7-xgdwf
              openshift.io/scc: stackrox-sensor
   #...

   Copy to Clipboard Toggle word wrap

   In this example, you can see that each pod has its own custom SCC, which is specified through the openshift.io/scc field.

2. Add the required roles and role bindings to use the Red Hat OpenShift SCCs instead of the RHACS custom SCCs.

   To add the required roles and role bindings to use the Red Hat OpenShift SCCs for the Central cluster, complete the following steps:

   1. Create a file named update-central.yaml that defines the role and role binding resources by using the following content:

      Example 3.1. Example YAML file

      apiVersion: rbac.authorization.k8s.io/v1
      kind: Role 

      1

      
      metadata:
        annotations:
           email: support@stackrox.com
           owner: stackrox
        labels:
           app.kubernetes.io/component: central
           app.kubernetes.io/instance: stackrox-central-services
           app.kubernetes.io/name: stackrox
           app.kubernetes.io/part-of: stackrox-central-services
           app.kubernetes.io/version: 4.4.0
        name: use-central-db-scc 

      2

      
        namespace: stackrox 

      3

      
      Rules: 

      4

      
      - apiGroups:
        - security.openshift.io
        resourceNames:
        - nonroot-v2
        resources:
        - securitycontextconstraints
        verbs:
        - use
      - - -
      apiVersion: rbac.authorization.k8s.io/v1
      kind: Role
      metadata:
        annotations:
           email: support@stackrox.com
           owner: stackrox
        labels:
           app.kubernetes.io/component: central
           app.kubernetes.io/instance: stackrox-central-services
           app.kubernetes.io/managed-by: Helm
           app.kubernetes.io/name: stackrox
           app.kubernetes.io/part-of: stackrox-central-services
           app.kubernetes.io/version: 4.4.0
        name: use-central-scc
        namespace: stackrox
      rules:
      - apiGroups:
        - security.openshift.io
        resourceNames:
        - nonroot-v2
        resources:
        - securitycontextconstraints
        verbs:
        - use
      - - -
      apiVersion: rbac.authorization.k8s.io/v1
      kind: Role
      metadata:
        annotations:
           email: support@stackrox.com
           owner: stackrox
        labels:
           app.kubernetes.io/component: scanner
           app.kubernetes.io/instance: stackrox-central-services
           app.kubernetes.io/name: stackrox
           app.kubernetes.io/part-of: stackrox-central-services
           app.kubernetes.io/version: 4.4.0
        name: use-scanner-scc
        namespace: stackrox
      rules:
      - apiGroups:
        - security.openshift.io
        resourceNames:
        - nonroot-v2
        resources:
        - securitycontextconstraints
        verbs:
        - use
      - - -
      apiVersion: rbac.authorization.k8s.io/v1
      kind: RoleBinding 

      5

      
      metadata:
        annotations:
           email: support@stackrox.com
           owner: stackrox
        labels:
           app.kubernetes.io/component: central
           app.kubernetes.io/instance: stackrox-central-services
           app.kubernetes.io/name: stackrox
           app.k ubernetes.io/part-of: stackrox-central-services
           app.kubernetes.io/version: 4.4.0
        name: central-db-use-scc 

      6

      
        namespace: stackrox
      roleRef: 

      7

      
        apiGroup: rbac.authorization.k8s.io
        kind: Role
        name: use-central-db-scc
      subjects: 

      8

      
      - kind: ServiceAccount
        name: central-db
        namespace: stackrox
      - - -
      apiVersion: rbac.authorization.k8s.io/v1
      kind: RoleBinding
      metadata:
        annotations:
           email: support@stackrox.com
           owner: stackrox
        labels:
           app.kubernetes.io/component: central
           app.kubernetes.io/instance: stackrox-central-services
           app.kubernetes.io/name: stackrox
           app.kubernetes.io/part-of: stackrox-central-services
           app.kubernetes.io/version: 4.4.0
        name: central-use-scc
        namespace: stackrox
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: Role
        name: use-central-scc
      subjects:
      - kind: ServiceAccount
        name: central
        namespace: stackrox
      - - -
      apiVersion: rbac.authorization.k8s.io/v1
      kind: RoleBinding
      metadata:
        annotations:
           email: support@stackrox.com
           owner: stackrox
        labels:
           app.kubernetes.io/component: scanner
           app.kubernetes.io/instance: stackrox-central-services
           app.kubernetes.io/name: stackrox
           app.kubernetes.io/part-of: stackrox-central-services
           app.kubernetes.io/version: 4.4.0
        name: scanner-use-scc
        namespace: stackrox
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: Role
        name: use-scanner-scc
      subjects:
      - kind: ServiceAccount
        name: scanner
        namespace: stackrox
      - - -

      Copy to Clipboard Toggle word wrap

      1

      The type of Kubernetes resource, in this example, Role.

      2

      The name of the role resource.

      3

      The namespace in which the role is created.

      4

      Describes the permissions granted by the role resource.

      5

      The type of Kubernetes resource, in this example, RoleBinding.

      6

      The name of the role binding resource.

      7

      Specifies the role to bind in the same namespace.

      8

      Specifies the subjects that are bound to the role.

   2. Create the role and role binding resources specified in the update-central.yaml file by running the following command:

      $ oc -n stackrox create -f ./update-central.yaml

      Copy to Clipboard Toggle word wrap

3. To add the required roles and role bindings to use the Red Hat OpenShift SCCs for all secured clusters, complete the following steps:

   1. Create a file named upgrade-scs.yaml that defines the role and role binding resources by using the following content:

      Example 3.2. Example YAML file

      apiVersion: rbac.authorization.k8s.io/v1
      kind: Role  

      1

      
      metadata:
        annotations:
           email: support@stackrox.com
           owner: stackrox
        labels:
           app.kubernetes.io/component: collector
           app.kubernetes.io/instance: stackrox-secured-cluster-services
           app.kubernetes.io/name: stackrox
           app.kubernetes.io/part-of: stackrox-secured-cluster-services
           app.kubernetes.io/version: 4.4.0
           auto-upgrade.stackrox.io/component: sensor
        name: use-privileged-scc  

      2

      
        namespace: stackrox 

      3

      
      rules:  

      4

      
      - apiGroups:
        - security.openshift.io
        resourceNames:
        - privileged
        resources:
        - securitycontextconstraints
        verbs:
        - use
      - - -
      apiVersion: rbac.authorization.k8s.io/v1
      kind: RoleBinding 

      5

      
      metadata:
        annotations:
           email: support@stackrox.com
           owner: stackrox
        labels:
           app.kubernetes.io/component: collector
           app.kubernetes.io/instance: stackrox-secured-cluster-services
           app.kubernetes.io/name: stackrox
           app.kubernetes.io/part-of: stackrox-secured-cluster-services
           app.kubernetes.io/version: 4.4.0
           auto-upgrade.stackrox.io/component: sensor
        name: collector-use-scc 

      6

      
        namespace: stackrox
      roleRef: 

      7

      
        apiGroup: rbac.authorization.k8s.io
        kind: Role
        name: use-privileged-scc
      subjects: 

      8

      
      - kind: ServiceAccount
        name: collector
        namespace: stackrox
      - - -

      Copy to Clipboard Toggle word wrap

      1

      The type of Kubernetes resource, in this example, Role.

      2

      The name of the role resource.

      3

      The namespace in which the role is created.

      4

      Describes the permissions granted by the role resource.

      5

      The type of Kubernetes resource, in this example, RoleBinding.

      6

      The name of the role binding resource.

      7

      Specifies the role to bind in the same namespace.

      8

      Specifies the subjects that are bound to the role.

   2. Create the role and role binding resources specified in the upgrade-scs.yaml file by running the following command:

      $ oc -n stackrox create -f ./update-scs.yaml

      Copy to Clipboard Toggle word wrap
      Important

      You must run this command on each secured cluster to create the role and role bindings specified in the upgrade-scs.yaml file.

4. Delete the SCCs that are specific to RHACS:

   1. To delete the SCCs that are specific to the Central cluster, run the following command:

      $ oc delete scc/stackrox-central scc/stackrox-central-db scc/stackrox-scanner

      Copy to Clipboard Toggle word wrap

   2. To delete the SCCs that are specific to all secured clusters, run the following command:

      $ oc delete scc/stackrox-admission-control scc/stackrox-collector scc/stackrox-sensor

      Copy to Clipboard Toggle word wrap
      Important

      You must run this command on each secured cluster to delete the SCCs that are specific to each secured cluster.

Procedure

1. Ensure that the bundle for the secured cluster was generated with future upgrades enabled before clicking Download YAML file and keys.
2. If possible, remove that secured cluster and generate a new bundle making sure that future upgrades are enabled.

3. If you cannot re-create the cluster, you can take these actions:

   1. Ensure that the service account sensor-upgrader exists in the same namespace as Sensor.
   2. Ensure that a ClusterRoleBinding exists (default name: <namespace>:upgrade-sensors) that grants the cluster-admin ClusterRole to the sensor-upgrader service account.

Procedure

1. Ensure that the Secured Cluster can access the registry and pull the image <image_reference:tag>.
2. Ensure that the image pull secrets are configured correctly in the secured cluster.

Procedure

1. Ensure that the upgrader has enough permissions for accessing the cluster objects. For more information, see "Upgrader is missing permissions".
2. Check the upgrader logs for more insights.