Chapter 6. Setting up RHACS Cloud Service with Kubernetes secured clusters
6.1. Creating a RHACS Cloud instance for Kubernetes clusters Copy linkLink copied to clipboard!
Access Red Hat Advanced Cluster Security Cloud Service (RHACS Cloud Service) by selecting an instance in the Red Hat Hybrid Cloud Console. An ACS instance contains the RHACS Cloud Service management interface and services that Red Hat configures and manages for you. The management interface connects to your secured clusters, which contain the services that scan and collect information about vulnerabilities. One instance can connect to and monitor many clusters.
6.1.1. Creating an instance in the console Copy linkLink copied to clipboard!
In the Red Hat Hybrid Cloud Console, create an ACS instance to connect to your secured clusters.
Procedure
To create an ACS instance:
- Log in to the Red Hat Hybrid Cloud Console.
-
From the navigation menu, select Advanced Cluster Security
ACS Instances. Select Create ACS instance and enter information into the displayed fields or select the appropriate option from the drop-down list:
- Name: Enter the name of your ACS instance. An ACS instance contains the RHACS Central component, also referred to as "Central", which includes the RHACS Cloud Service management interface and services that are configured and managed by Red Hat. You manage your secured clusters that communicate with Central. You can connect many secured clusters to one instance.
- Cloud provider: The cloud provider where Central is located. Select AWS.
Cloud region: The region for your cloud provider where Central is located. Select one of the following regions:
- US-East, N. Virginia
- Europe, Ireland
- Availability zones: Use the default value (Multi).
- Click Create instance.
6.1.2. Next steps Copy linkLink copied to clipboard!
-
On each Kubernetes cluster you want to secure, install secured cluster resources by using Helm charts or the
roxctlCLI.
6.2. Generating an init bundle for Kubernetes secured clusters Copy linkLink copied to clipboard!
Before you install the SecuredCluster resource on a cluster, you must create an init bundle. The cluster that has SecuredCluster installed and configured then uses this bundle to authenticate with the ACS Console. You can create an init bundle by using either the RHACS portal or the roxctl CLI. You then apply the init bundle by using it to create resources.
6.2.1. Generating an init bundle by using the RHACS portal Copy linkLink copied to clipboard!
You can create an init bundle containing secrets by using the RHACS portal, also called the ACS Console.
You must have the Admin user role to create an init bundle.
Procedure
-
On the RHACS portal, navigate to Platform Configuration
Integrations. - Navigate to the Authentication Tokens section and click on Cluster Init Bundle.
- Click Generate bundle.
Enter a name for the cluster init bundle and click Generate.
- If you are installing using Helm charts, click Download Helm Values File to download the generated bundle.
- If you are installing using the Operator, click Download Kubernetes Secret File to download the generated bundle.
Store this bundle securely because it contains secrets. You can use the same bundle to create multiple secured clusters.
Next steps
- Apply the init bundle by creating a resource on the secured cluster.
- Install secured cluster services on each cluster.
6.2.2. Generating an init bundle by using the roxctl CLI Copy linkLink copied to clipboard!
You can create an init bundle with secrets by using the roxctl CLI.
You must have the Admin user role to create init bundles.
Prerequisites
You have configured the ROX_API_TOKEN and the ROX_CENTRAL_ADDRESS environment variables.
Set the
ROX_API_TOKENand theROX_CENTRAL_ADDRESSenvironment variables:$ export ROX_API_TOKEN=<api_token>$ export ROX_CENTRAL_ADDRESS=<address>:<port_number>
Procedure
Run the following command to generate a cluster init bundle containing secrets:
For Helm installations:
$ roxctl -e "$ROX_CENTRAL_ADDRESS" \ central init-bundles generate <cluster_init_bundle_name> \ --output cluster_init_bundle.yamlFor Operator installations:
$ roxctl -e "$ROX_CENTRAL_ADDRESS" \ central init-bundles generate <cluster_init_bundle_name> \ --output-secrets cluster_init_bundle.yamlImportantEnsure that you store this bundle securely because it contains secrets. You can use the same bundle to set up multiple secured clusters.
Next Step
- Use the Red Hat OpenShift CLI to create resources using the init bundle.
6.3. Applying an init bundle for Kubernetes secured clusters Copy linkLink copied to clipboard!
Apply the init bundle by using it to create resources.
6.3.1. Creating resources by using the init bundle Copy linkLink copied to clipboard!
Before you install secured clusters, you must use the init bundle to create the required resources on the cluster that will allow the services on the secured clusters to communicate with RHACS Cloud Service.
If you are installing by using Helm charts, do not perform this step. Complete the installation by using Helm; See "Installing RHACS on secured clusters by using Helm charts" in the additional resources section.
Prerequisites
- You must have generated an init bundle containing secrets.
Procedure
To create resources, perform one of the following steps:
- In the OpenShift Container Platform web console, in the top menu, click + to open the Import YAML page. You can drag the init bundle file or copy and paste its contents into the editor, and then click Create.
Using the Red Hat OpenShift CLI, run the following command to create the resources:
$ oc create -f <init_bundle>.yaml \1 -n <stackrox>2 Using the
kubectlCLI, run the following commands to create the resources:$ kubectl create namespace stackrox1 $ kubectl create -f <init_bundle>.yaml \2 -n <stackrox>3
Next Step
- Install RHACS secured cluster services in all clusters that you want to monitor.
6.4. Installing secured cluster services from RHACS Cloud Service on Kubernetes clusters Copy linkLink copied to clipboard!
You can install RHACS Cloud Service on your secured clusters by using one of the following methods:
- By using Helm charts
-
By using the
roxctlCLI (do not use this method unless you have a specific installation need that requires using it)
6.4.1. Installing RHACS Cloud Service on secured clusters by using Helm charts Copy linkLink copied to clipboard!
You can install RHACS on secured clusters by using Helm charts with no customization, by using Helm charts with the default values, or by using Helm charts with customizations of configuration parameters.
First, ensure that you add the Helm chart repository.
6.4.1.1. Adding the Helm chart repository Copy linkLink copied to clipboard!
Procedure
Add the RHACS charts repository.
$ helm repo add rhacs https://mirror.openshift.com/pub/rhacs/charts/
The Helm repository for Red Hat Advanced Cluster Security for Kubernetes includes Helm charts for installing different components, including:
Secured Cluster Services Helm chart (
secured-cluster-services) for installing the per-cluster and per-node components (Sensor, Admission Controller, Collector, and Scanner-slim).NoteDeploy the per-cluster components into each cluster that you want to monitor and deploy the per-node components in all nodes that you want to monitor.
Verification
Run the following command to verify the added chart repository:
$ helm search repo -l rhacs/
6.4.1.2. Installing RHACS Cloud Service on secured clusters by using Helm charts without customizations Copy linkLink copied to clipboard!
6.4.1.2.1. Installing the secured-cluster-services Helm chart without customization Copy linkLink copied to clipboard!
Use the following instructions to install the secured-cluster-services Helm chart to deploy the per-cluster and per-node components (Sensor, Admission controller, Collector, and Scanner-slim).
To install Collector on systems that have Unified Extensible Firmware Interface (UEFI) and that have Secure Boot enabled, you must use eBPF probes because kernel modules are unsigned, and the UEFI firmware cannot load unsigned packages. Collector identifies Secure Boot status at the start and switches to eBPF probes if required.
Prerequisites
- You must have generated an RHACS init bundle for your cluster.
-
You must have access to the Red Hat Container Registry and a pull secret for authentication. For information about downloading images from
registry.redhat.io, see Red Hat Container Registry Authentication. -
You must have the Central API Endpoint, including the address and the port number. You can view this information by choosing Advanced Cluster Security
ACS Instances from the cloud console navigation menu, then clicking the ACS instance you created.
6.4.1.3. Configuring the secured-cluster-services Helm chart with customizations Copy linkLink copied to clipboard!
Procedure
This section describes Helm chart configuration parameters that you can use with the helm install and helm upgrade commands. You can specify these parameters by using the --set option or by creating YAML configuration files.
Create the following files for configuring the Helm chart for installing Red Hat Advanced Cluster Security for Kubernetes:
-
Public configuration file
values-public.yaml: Use this file to save all non-sensitive configuration options. -
Private configuration file
values-private.yaml: Use this file to save all sensitive configuration options. Ensure that you store this file securely.
While using the secured-cluster-services Helm chart, do not modify the values.yaml file that is part of the chart.
6.4.1.3.1. Configuration parameters Copy linkLink copied to clipboard!
| Parameter | Description |
|---|---|
|
| Name of your cluster. |
|
|
Address, including port number, of the Central endpoint. If you are using a non-gRPC capable load balancer, use the WebSocket protocol by prefixing the endpoint address with |
|
| Address of the Sensor endpoint including port number. |
|
| Image pull policy for the Sensor container. |
|
| The internal service-to-service TLS certificate that Sensor uses. |
|
| The internal service-to-service TLS certificate key that Sensor uses. |
|
| The memory request for the Sensor container. Use this parameter to override the default value. |
|
| The CPU request for the Sensor container. Use this parameter to override the default value. |
|
| The memory limit for the Sensor container. Use this parameter to override the default value. |
|
| The CPU limit for the Sensor container. Use this parameter to override the default value. |
|
|
Specify a node selector label as |
|
| If the node selector selects tainted nodes, use this parameter to specify a taint toleration key, value, and effect for Sensor. This parameter is mainly used for infrastructure nodes. |
|
|
The name of the |
|
| The name of the Collector image. |
|
| Address of the registry you are using for the main image. |
|
| Address of the registry you are using for the Collector image. |
|
|
Image pull policy for |
|
| Image pull policy for the Collector images. |
|
|
Tag of |
|
|
Tag of |
|
|
Either |
|
| Image pull policy for the Collector container. |
|
| Image pull policy for the Compliance container. |
|
|
If you specify |
|
| The memory request for the Collector container. Use this parameter to override the default value. |
|
| The CPU request for the Collector container. Use this parameter to override the default value. |
|
| The memory limit for the Collector container. Use this parameter to override the default value. |
|
| The CPU limit for the Collector container. Use this parameter to override the default value. |
|
| The memory request for the Compliance container. Use this parameter to override the default value. |
|
| The CPU request for the Compliance container. Use this parameter to override the default value. |
|
| The memory limit for the Compliance container. Use this parameter to override the default value. |
|
| The CPU limit for the Compliance container. Use this parameter to override the default value. |
|
| The internal service-to-service TLS certificate that Collector uses. |
|
| The internal service-to-service TLS certificate key that Collector uses. |
|
|
This setting controls whether Kubernetes is configured to contact Red Hat Advanced Cluster Security for Kubernetes with |
|
|
When you set this parameter as |
|
|
This setting controls whether the cluster is configured to contact Red Hat Advanced Cluster Security for Kubernetes with |
|
| This setting controls whether Red Hat Advanced Cluster Security for Kubernetes evaluates policies; if it is disabled, all AdmissionReview requests are automatically accepted. |
|
|
This setting controls the behavior of the admission control service. You must specify |
|
|
If you set this option to |
|
|
Set it to |
|
| The maximum time, in seconds, Red Hat Advanced Cluster Security for Kubernetes should wait while evaluating admission review requests. Use this to set request timeouts when you enable image scanning. If the image scan runs longer than the specified time, Red Hat Advanced Cluster Security for Kubernetes accepts the request. |
|
| The memory request for the Admission Control container. Use this parameter to override the default value. |
|
| The CPU request for the Admission Control container. Use this parameter to override the default value. |
|
| The memory limit for the Admission Control container. Use this parameter to override the default value. |
|
| The CPU limit for the Admission Control container. Use this parameter to override the default value. |
|
|
Specify a node selector label as |
|
| If the node selector selects tainted nodes, use this parameter to specify a taint toleration key, value, and effect for Admission Control. This parameter is mainly used for infrastructure nodes. |
|
| The internal service-to-service TLS certificate that Admission Control uses. |
|
| The internal service-to-service TLS certificate key that Admission Control uses. |
|
|
Use this parameter to override the default |
|
|
If you specify |
|
|
Specify |
|
|
Specify |
|
|
Specify |
|
| Resource specification for Sensor. |
|
| Resource specification for Admission controller. |
|
| Resource specification for Collector. |
|
| Resource specification for Collector’s Compliance container. |
|
|
If you set this option to |
|
|
If you set this option to |
|
|
If you set this option to |
|
| If the node selector selects tainted nodes, use this parameter to specify a taint toleration key, value, and effect for Scanner DB. |
|
| Resource specification for Collector’s Compliance container. |
|
| Setting this parameter allows you to modify the scanner log level. Use this option only for troubleshooting purposes. |
|
|
If you set this option to |
|
| The minimum number of replicas for autoscaling. Defaults to 2. |
|
| The maximum number of replicas for autoscaling. Defaults to 5. |
|
|
Specify a node selector label as |
|
| If the node selector selects tainted nodes, use this parameter to specify a taint toleration key, value, and effect for Scanner. |
|
|
Specify a node selector label as |
|
| If the node selector selects tainted nodes, use this parameter to specify a taint toleration key, value, and effect for Scanner DB. |
|
| The memory request for the Scanner container. Use this parameter to override the default value. |
|
| The CPU request for the Scanner container. Use this parameter to override the default value. |
|
| The memory limit for the Scanner container. Use this parameter to override the default value. |
|
| The CPU limit for the Scanner container. Use this parameter to override the default value. |
|
| The memory request for the Scanner DB container. Use this parameter to override the default value. |
|
| The CPU request for the Scanner DB container. Use this parameter to override the default value. |
|
| The memory limit for the Scanner DB container. Use this parameter to override the default value. |
|
| The CPU limit for the Scanner DB container. Use this parameter to override the default value. |
|
|
If you set this option to |
6.4.1.3.1.1. Environment variables Copy linkLink copied to clipboard!
You can specify environment variables for Sensor and Admission controller in the following format:
customize:
envVars:
ENV_VAR1: "value1"
ENV_VAR2: "value2"
The customize setting allows you to specify custom Kubernetes metadata (labels and annotations) for all objects created by this Helm chart and additional pod labels, pod annotations, and container environment variables for workloads.
The configuration is hierarchical, in the sense that metadata defined at a more generic scope (for example, for all objects) can be overridden by metadata defined at a narrower scope (for example, only for the Sensor deployment).
6.4.1.3.2. Installing the secured-cluster-services Helm chart Copy linkLink copied to clipboard!
After you configure the values-public.yaml and values-private.yaml files, install the secured-cluster-services Helm chart to deploy the per-cluster and per-node components (Sensor, Admission controller, Collector, and Scanner-slim).
To install Collector on systems that have Unified Extensible Firmware Interface (UEFI) and that have Secure Boot enabled, you must use eBPF probes because kernel modules are unsigned, and the UEFI firmware cannot load unsigned packages. Collector identifies Secure Boot status at the start and switches to eBPF probes if required.
Prerequisites
- You must have generated an RHACS init bundle for your cluster.
-
You must have access to the Red Hat Container Registry and a pull secret for authentication. For information about downloading images from
registry.redhat.io, see Red Hat Container Registry Authentication. -
You must have the Central API Endpoint, including the address and the port number. You can view this information by choosing Advanced Cluster Security
ACS Instances from the cloud console navigation menu, then clicking the ACS instance you created.
Procedure
Run the following command:
$ helm install -n stackrox \ --create-namespace stackrox-secured-cluster-services rhacs/secured-cluster-services \ -f <name_of_cluster_init_bundle.yaml> \ -f <path_to_values_public.yaml> -f <path_to_values_private.yaml> \1 --set imagePullSecrets.username=<username> \2 --set imagePullSecrets.password=<password>3
To deploy secured-cluster-services Helm chart by using a continuous integration (CI) system, pass the init bundle YAML file as an environment variable to the helm install command:
$ helm install ... -f <(echo "$INIT_BUNDLE_YAML_SECRET")
- 1
- If you are using base64 encoded variables, use the
helm install … -f <(echo "$INIT_BUNDLE_YAML_SECRET" | base64 --decode)command instead.
6.4.1.4. Changing configuration options after deploying the secured-cluster-services Helm chart Copy linkLink copied to clipboard!
You can make changes to any configuration options after you have deployed the secured-cluster-services Helm chart.
Procedure
-
Update the
values-public.yamlandvalues-private.yamlconfiguration files with new values. Run the
helm upgradecommand and specify the configuration files using the-foption:$ helm upgrade -n stackrox \ stackrox-secured-cluster-services rhacs/secured-cluster-services \ --reuse-values \1 -f <path_to_values_public.yaml> \ -f <path_to_values_private.yaml>- 1
- You must specify the
--reuse-valuesparameter, otherwise the Helm upgrade command resets all previously configured settings.
NoteYou can also specify configuration values using the
--setor--set-fileparameters. However, these options are not saved, and it requires you to manually specify all the options again whenever you make changes.
6.4.2. Installing RHACS on secured clusters by using the roxctl CLI Copy linkLink copied to clipboard!
To install RHACS on secured clusters by using the CLI, perform the following steps:
-
Install the
roxctlCLI. - Install Sensor.
6.4.2.1. Installing the roxctl CLI Copy linkLink copied to clipboard!
You must first download the binary. You can install roxctl on Linux, Windows, or macOS.
6.4.2.1.1. Installing the roxctl CLI on Linux Copy linkLink copied to clipboard!
You can install the roxctl CLI binary on Linux by using the following procedure.
Procedure
Download the latest version of the
roxctlCLI:$ curl -O https://mirror.openshift.com/pub/rhacs/assets/4.2.5/bin/Linux/roxctlMake the
roxctlbinary executable:$ chmod +x roxctlPlace the
roxctlbinary in a directory that is on yourPATH:To check your
PATH, execute the following command:$ echo $PATH
Verification
Verify the
roxctlversion you have installed:$ roxctl version
6.4.2.1.2. Installing the roxctl CLI on macOS Copy linkLink copied to clipboard!
You can install the roxctl CLI binary on macOS by using the following procedure.
Procedure
Download the latest version of the
roxctlCLI:$ curl -O https://mirror.openshift.com/pub/rhacs/assets/4.2.5/bin/Darwin/roxctlRemove all extended attributes from the binary:
$ xattr -c roxctlMake the
roxctlbinary executable:$ chmod +x roxctlPlace the
roxctlbinary in a directory that is on yourPATH:To check your
PATH, execute the following command:$ echo $PATH
Verification
Verify the
roxctlversion you have installed:$ roxctl version
6.4.2.1.3. Installing the roxctl CLI on Windows Copy linkLink copied to clipboard!
You can install the roxctl CLI binary on Windows by using the following procedure.
Procedure
Download the latest version of the
roxctlCLI:$ curl -O https://mirror.openshift.com/pub/rhacs/assets/4.2.5/bin/Windows/roxctl.exe
Verification
Verify the
roxctlversion you have installed:$ roxctl version
6.4.2.2. Installing Sensor Copy linkLink copied to clipboard!
To monitor a cluster, you must deploy Sensor. You must deploy Sensor into each cluster that you want to monitor. The following steps describe adding Sensor by using the RHACS portal.
Prerequisites
- You must have already installed Central services, or you can access Central services by selecting your ACS instance on Red Hat Advanced Cluster Security Cloud Service (RHACS Cloud Service).
Procedure
-
On your secured cluster, in the RHACS portal, navigate to Platform Configuration
Clusters. - Select + New Cluster.
- Specify a name for the cluster.
Provide appropriate values for the fields based on where you are deploying the Sensor.
-
Enter the Central API Endpoint, including the address and the port number. You can view this information again in the Red Hat Hybrid Cloud Console by choosing Advanced Cluster Security
ACS Instances, and then clicking the ACS instance you created.
-
Enter the Central API Endpoint, including the address and the port number. You can view this information again in the Red Hat Hybrid Cloud Console by choosing Advanced Cluster Security
- Click Next to continue with the Sensor setup.
Click Download YAML File and Keys to download the cluster bundle (zip archive).
ImportantThe cluster bundle zip archive includes unique configurations and keys for each cluster. Do not reuse the same files in another cluster.
From a system that has access to the monitored cluster, unzip and run the
sensorscript from the cluster bundle:$ unzip -d sensor sensor-<cluster_name>.zip$ ./sensor/sensor.shIf you get a warning that you do not have the required permissions to deploy Sensor, follow the on-screen instructions, or contact your cluster administrator for assistance.
After Sensor is deployed, it contacts Central and provides cluster information.
Verification
Return to the RHACS portal and check if the deployment is successful. If successful, when viewing your list of clusters in Platform Configuration
Clusters, the cluster status displays a green checkmark and a Healthy status. If you do not see a green checkmark, use the following command to check for problems: On Kubernetes, enter the following command:
$ kubectl get pod -n stackrox -w
- Click Finish to close the window.
After installation, Sensor starts reporting security information to RHACS and the RHACS portal dashboard begins showing deployments, images, and policy violations from the cluster on which you have installed the Sensor.
6.5. Verifying installation of secured clusters Copy linkLink copied to clipboard!
After installing RHACS Cloud Service, you can perform some steps to verify that the installation was successful.
To verify installation, access your ACS Console from the Red Hat Hybrid Cloud Console. The Dashboard displays the number of clusters that RHACS Cloud Service is monitoring, along with information about nodes, deployments, images, and violations.
If no data appears in the ACS Console:
-
Ensure that at least one secured cluster is connected to your RHACS Cloud Service instance. For more information, see instructions for installing by using Helm charts or by using the
roxctlCLI. - Examine your Sensor pod logs to ensure that the connection to your RHACS Cloud Service instance is successful.
-
Examine the values in the
SecuredClusterAPI in the Operator on your local cluster to ensure that the Central API Endpoint has been entered correctly. This value should be the same value as shown in the ACS instance details in the Red Hat Hybrid Cloud Console.