Red Hat Summit9장. Upgrading RHACS Cloud Service
9.1. Upgrading secured clusters in RHACS Cloud Service by using the Operator
Red Hat provides regular service updates for the components that it manages, including Central services. These service updates include upgrades to new versions of Red Hat Advanced Cluster Security Cloud Service.
You must regularly upgrade the version of RHACS on your secured clusters to ensure compatibility with RHACS Cloud Service.
9.1.1. Preparing to upgrade
Before you upgrade the Red Hat Advanced Cluster Security for Kubernetes (RHACS) version, complete the following steps:
-
If the cluster you are upgrading contains the
SecuredClustercustom resource (CR), change the collection method toCORE_BPF. For more information, see "Changing the collection method".
9.1.1.1. Changing the collection method
If the cluster that you are upgrading contains the SecuredCluster CR, you must ensure that the per node collection setting is set to CORE_BPF before you upgrade.
Procedure
- In the OpenShift Container Platform web console, go to the RHACS Operator page.
- In the top navigation menu, select Secured Cluster.
- Click the instance name, for example, stackrox-secured-cluster-services.
Use one of the following methods to change the setting:
-
In the Form view, under Per Node Settings
Collector Settings Collection, select CORE_BPF. -
Click YAML to open the YAML editor and locate the
spec.perNode.collector.collectionattribute. If the value isKernelModuleorEBPF, then change it toCORE_BPF.
-
In the Form view, under Per Node Settings
- Click Save.
9.1.2. Rolling back an Operator upgrade for secured clusters
To roll back an Operator upgrade, you can use either the CLI or the OpenShift Container Platform web console.
On secured clusters, rolling back Operator upgrades is needed only in rare cases, for example, if an issue exists with the secured cluster.
9.1.2.1. Rolling back an Operator upgrade by using the CLI
You can roll back the Operator version by using command-line interface (CLI) commands.
Procedure
Delete the Operator Lifecycle Manager (OLM) subscription and cluster service version (CSV):
참고If you use Kubernetes, enter
kubectlinstead ofoc.To delete the OLM subscription, run the following command:
$ oc -n rhacs-operator delete subscription rhacs-operatorExample output
subscription.operators.coreos.com "rhacs-operator" deletedTo delete the CSV, run the following command:
$ oc -n rhacs-operator delete csv -l operators.coreos.com/rhacs-operator.rhacs-operatorExample output
clusterserviceversion.operators.coreos.com "rhacs-operator.v4.8.4" deleted
- Install the latest version of the Operator on the rolled back channel.
9.1.2.2. Rolling back an Operator upgrade by using the web console
You can roll back the Operator version by using the OpenShift Container Platform web console.
Prerequisites
-
You have access to an OpenShift Container Platform cluster web console using an account with
cluster-adminpermissions.
Procedure
-
In the OpenShift web console, click Operators
Installed Operators. - From the list of projects, select rhacs-operator.
Locate the Advanced Cluster Security for Kubernetes Operator:
Click the overflow menu
Uninstall Operator. The uninstall Operator dialog is displayed.
- Ensure that the Delete all operand instances for this operator checkbox is clear to avoid uninstallation of Red Hat Advanced Cluster Security for Kubernetes (RHACS).
- Click Uninstall.
- Install the latest version of the Operator on the rolled back channel.
9.1.3. Troubleshooting Operator upgrade issues
Follow these instructions to investigate and resolve upgrade-related issues for the RHACS Operator.
9.1.3.1. Central or Secured cluster fails to deploy
When RHACS Operator has the following conditions, you must check the custom resource conditions to find the issue:
- If the Operator fails to deploy Secured Cluster
- If the Operator fails to apply CR changes to actual resources
For Secured clusters, run the following command to check the conditions:
$ oc -n rhacs-operator describe securedclusters.platform.stackrox.ioYou can identify configuration errors from the conditions output:
Example output
Conditions: Last Transition Time: 2023-04-19T10:49:57Z Status: False Type: Deployed Last Transition Time: 2023-04-19T10:49:57Z Status: True Type: Initialized Last Transition Time: 2023-04-19T10:59:10Z Message: Deployment.apps "central" is invalid: spec.template.spec.containers[0].resources.requests: Invalid value: "50": must be less than or equal to cpu limit Reason: ReconcileError Status: True Type: Irreconcilable Last Transition Time: 2023-04-19T10:49:57Z Message: No proxy configuration is desired Reason: NoProxyConfig Status: False Type: ProxyConfigFailed Last Transition Time: 2023-04-19T10:49:57Z Message: Deployment.apps "central" is invalid: spec.template.spec.containers[0].resources.requests: Invalid value: "50": must be less than or equal to cpu limit Reason: InstallError Status: True Type: ReleaseFailedAdditionally, you can view RHACS pod logs to find more information about the issue. Run the following command to view the logs:
oc -n rhacs-operator logs deploy/rhacs-operator-controller-manager manager
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}