Chapter 6. Setting up RHACS Cloud Service with Kubernetes secured clusters
6.1. Creating a RHACS Cloud instance for Kubernetes clusters
Access Red Hat Advanced Cluster Security Cloud Service (RHACS Cloud Service) by selecting an instance in the Red Hat Hybrid Cloud Console. An ACS instance contains the RHACS Cloud Service management interface and services that Red Hat configures and manages for you. The management interface connects to your secured clusters, which contain the services that scan and collect information about vulnerabilities. One instance can connect to and monitor many clusters.
6.1.1. Creating an instance in the console
In the Red Hat Hybrid Cloud Console, create an ACS instance to connect to your secured clusters.
Procedure
To create an ACS instance:
- Log in to the Red Hat Hybrid Cloud Console.
-
From the navigation menu, select Advanced Cluster Security
ACS Instances. Select Create ACS instance and enter information into the displayed fields or select the appropriate option from the drop-down list:
- Name: Enter the name of your ACS instance. An ACS instance contains the RHACS Central component, also referred to as "Central", which includes the RHACS Cloud Service management interface and services that are configured and managed by Red Hat. You manage your secured clusters that communicate with Central. You can connect many secured clusters to one instance.
- Cloud provider: The cloud provider where Central is located. Select AWS.
Cloud region: The region for your cloud provider where Central is located. Select one of the following regions:
- US-East, N. Virginia
- Europe, Ireland
- Availability zones: Use the default value (Multi).
- Click Create instance.
6.1.2. Next steps
-
On each Kubernetes cluster you want to secure, install secured cluster resources by using Helm charts or the
roxctl
CLI.
6.2. Generating an init bundle for Kubernetes secured clusters
Before you install the SecuredCluster
resource on a cluster, you must create an init bundle. The cluster that has SecuredCluster
installed and configured then uses this bundle to authenticate with the ACS Console. You can create an init bundle by using either the RHACS portal or the roxctl
CLI. You then apply the init bundle by using it to create resources.
6.2.1. Generating an init bundle by using the RHACS portal
You can create an init bundle containing secrets by using the RHACS portal, also called the ACS Console.
You must have the Admin
user role to create an init bundle.
Procedure
-
On the RHACS portal, navigate to Platform Configuration
Integrations. - Navigate to the Authentication Tokens section and click on Cluster Init Bundle.
- Click Generate bundle.
Enter a name for the cluster init bundle and click Generate.
- If you are installing using Helm charts, click Download Helm Values File to download the generated bundle.
- If you are installing using the Operator, click Download Kubernetes Secret File to download the generated bundle.
Store this bundle securely because it contains secrets. You can use the same bundle to create multiple secured clusters.
Next steps
- Apply the init bundle by creating a resource on the secured cluster.
- Install secured cluster services on each cluster.
6.2.2. Generating an init bundle by using the roxctl CLI
You can create an init bundle with secrets by using the roxctl
CLI.
You must have the Admin
user role to create init bundles.
Prerequisites
You have configured the ROX_API_TOKEN
and the ROX_CENTRAL_ADDRESS
environment variables.
Set the
ROX_API_TOKEN
and theROX_CENTRAL_ADDRESS
environment variables:$ export ROX_API_TOKEN=<api_token>
$ export ROX_CENTRAL_ADDRESS=<address>:<port_number>
Procedure
Run the following command to generate a cluster init bundle containing secrets:
For Helm installations:
$ roxctl -e "$ROX_CENTRAL_ADDRESS" \ central init-bundles generate <cluster_init_bundle_name> \ --output cluster_init_bundle.yaml
For Operator installations:
$ roxctl -e "$ROX_CENTRAL_ADDRESS" \ central init-bundles generate <cluster_init_bundle_name> \ --output-secrets cluster_init_bundle.yaml
ImportantEnsure that you store this bundle securely because it contains secrets. You can use the same bundle to set up multiple secured clusters.
Next Step
- Use the Red Hat OpenShift CLI to create resources using the init bundle.
6.3. Applying an init bundle for Kubernetes secured clusters
Apply the init bundle by using it to create resources.
6.3.1. Creating resources by using the init bundle
Before you install secured clusters, you must use the init bundle to create the required resources on the cluster that will allow the services on the secured clusters to communicate with RHACS Cloud Service.
If you are installing by using Helm charts, do not perform this step. Complete the installation by using Helm; See "Installing RHACS on secured clusters by using Helm charts" in the additional resources section.
Prerequisites
- You must have generated an init bundle containing secrets.
Procedure
To create resources, perform one of the following steps:
- In the OpenShift Container Platform web console, in the top menu, click + to open the Import YAML page. You can drag the init bundle file or copy and paste its contents into the editor, and then click Create.
Using the Red Hat OpenShift CLI, run the following command to create the resources:
$ oc create -f <init_bundle>.yaml \ 1 -n <stackrox> 2
Using the
kubectl
CLI, run the following commands to create the resources:$ kubectl create namespace stackrox 1 $ kubectl create -f <init_bundle>.yaml \ 2 -n <stackrox> 3
Next Step
- Install RHACS secured cluster services in all clusters that you want to monitor.
6.4. Installing secured cluster services from RHACS Cloud Service on Kubernetes clusters
You can install RHACS Cloud Service on your secured clusters by using one of the following methods:
- By using Helm charts
-
By using the
roxctl
CLI (do not use this method unless you have a specific installation need that requires using it)
6.4.1. Installing RHACS Cloud Service on secured clusters by using Helm charts
You can install RHACS on secured clusters by using Helm charts with no customization, by using Helm charts with the default values, or by using Helm charts with customizations of configuration parameters.
First, ensure that you add the Helm chart repository.
6.4.1.1. Adding the Helm chart repository
Procedure
Add the RHACS charts repository.
$ helm repo add rhacs https://mirror.openshift.com/pub/rhacs/charts/
The Helm repository for Red Hat Advanced Cluster Security for Kubernetes includes Helm charts for installing different components, including:
Secured Cluster Services Helm chart (
secured-cluster-services
) for installing the per-cluster and per-node components (Sensor, Admission Controller, Collector, and Scanner-slim).NoteDeploy the per-cluster components into each cluster that you want to monitor and deploy the per-node components in all nodes that you want to monitor.
Verification
Run the following command to verify the added chart repository:
$ helm search repo -l rhacs/
6.4.1.2. Installing RHACS Cloud Service on secured clusters by using Helm charts without customizations
6.4.1.2.1. Installing the secured-cluster-services Helm chart without customization
Use the following instructions to install the secured-cluster-services
Helm chart to deploy the per-cluster and per-node components (Sensor, Admission controller, Collector, and Scanner-slim).
To install Collector on systems that have Unified Extensible Firmware Interface (UEFI) and that have Secure Boot enabled, you must use eBPF probes because kernel modules are unsigned, and the UEFI firmware cannot load unsigned packages. Collector identifies Secure Boot status at the start and switches to eBPF probes if required.
Prerequisites
- You must have generated an RHACS init bundle for your cluster.
-
You must have access to the Red Hat Container Registry and a pull secret for authentication. For information about downloading images from
registry.redhat.io
, see Red Hat Container Registry Authentication. -
You must have the Central API Endpoint, including the address and the port number. You can view this information by choosing Advanced Cluster Security
ACS Instances from the cloud console navigation menu, then clicking the ACS instance you created.
6.4.1.3. Configuring the secured-cluster-services Helm chart with customizations
Procedure
This section describes Helm chart configuration parameters that you can use with the helm install
and helm upgrade
commands. You can specify these parameters by using the --set
option or by creating YAML configuration files.
Create the following files for configuring the Helm chart for installing Red Hat Advanced Cluster Security for Kubernetes:
-
Public configuration file
values-public.yaml
: Use this file to save all non-sensitive configuration options. -
Private configuration file
values-private.yaml
: Use this file to save all sensitive configuration options. Ensure that you store this file securely.
While using the secured-cluster-services
Helm chart, do not modify the values.yaml
file that is part of the chart.
6.4.1.3.1. Configuration parameters
Parameter | Description |
---|---|
| Name of your cluster. |
|
Address, including port number, of the Central endpoint. If you are using a non-gRPC capable load balancer, use the WebSocket protocol by prefixing the endpoint address with |
| Address of the Sensor endpoint including port number. |
| Image pull policy for the Sensor container. |
| The internal service-to-service TLS certificate that Sensor uses. |
| The internal service-to-service TLS certificate key that Sensor uses. |
| The memory request for the Sensor container. Use this parameter to override the default value. |
| The CPU request for the Sensor container. Use this parameter to override the default value. |
| The memory limit for the Sensor container. Use this parameter to override the default value. |
| The CPU limit for the Sensor container. Use this parameter to override the default value. |
|
Specify a node selector label as |
| If the node selector selects tainted nodes, use this parameter to specify a taint toleration key, value, and effect for Sensor. This parameter is mainly used for infrastructure nodes. |
|
The name of the |
| The name of the Collector image. |
| Address of the registry you are using for the main image. |
| Address of the registry you are using for the Collector image. |
|
Image pull policy for |
| Image pull policy for the Collector images. |
|
Tag of |
|
Tag of |
|
Either |
| Image pull policy for the Collector container. |
| Image pull policy for the Compliance container. |
|
If you specify |
| The memory request for the Collector container. Use this parameter to override the default value. |
| The CPU request for the Collector container. Use this parameter to override the default value. |
| The memory limit for the Collector container. Use this parameter to override the default value. |
| The CPU limit for the Collector container. Use this parameter to override the default value. |
| The memory request for the Compliance container. Use this parameter to override the default value. |
| The CPU request for the Compliance container. Use this parameter to override the default value. |
| The memory limit for the Compliance container. Use this parameter to override the default value. |
| The CPU limit for the Compliance container. Use this parameter to override the default value. |
| The internal service-to-service TLS certificate that Collector uses. |
| The internal service-to-service TLS certificate key that Collector uses. |
|
This setting controls whether Kubernetes is configured to contact Red Hat Advanced Cluster Security for Kubernetes with |
|
When you set this parameter as |
|
This setting controls whether the cluster is configured to contact Red Hat Advanced Cluster Security for Kubernetes with |
| This setting controls whether Red Hat Advanced Cluster Security for Kubernetes evaluates policies; if it is disabled, all AdmissionReview requests are automatically accepted. |
|
This setting controls the behavior of the admission control service. You must specify |
|
If you set this option to |
|
Set it to |
| The maximum time, in seconds, Red Hat Advanced Cluster Security for Kubernetes should wait while evaluating admission review requests. Use this to set request timeouts when you enable image scanning. If the image scan runs longer than the specified time, Red Hat Advanced Cluster Security for Kubernetes accepts the request. |
| The memory request for the Admission Control container. Use this parameter to override the default value. |
| The CPU request for the Admission Control container. Use this parameter to override the default value. |
| The memory limit for the Admission Control container. Use this parameter to override the default value. |
| The CPU limit for the Admission Control container. Use this parameter to override the default value. |
|
Specify a node selector label as |
| If the node selector selects tainted nodes, use this parameter to specify a taint toleration key, value, and effect for Admission Control. This parameter is mainly used for infrastructure nodes. |
| The internal service-to-service TLS certificate that Admission Control uses. |
| The internal service-to-service TLS certificate key that Admission Control uses. |
|
Use this parameter to override the default |
|
If you specify |
|
Specify |
|
Specify |
|
Specify |
| Resource specification for Sensor. |
| Resource specification for Admission controller. |
| Resource specification for Collector. |
| Resource specification for Collector’s Compliance container. |
|
If you set this option to |
|
If you set this option to |
|
If you set this option to |
| If the node selector selects tainted nodes, use this parameter to specify a taint toleration key, value, and effect for Scanner DB. |
| Resource specification for Collector’s Compliance container. |
| Setting this parameter allows you to modify the scanner log level. Use this option only for troubleshooting purposes. |
|
If you set this option to |
| The minimum number of replicas for autoscaling. Defaults to 2. |
| The maximum number of replicas for autoscaling. Defaults to 5. |
|
Specify a node selector label as |
| If the node selector selects tainted nodes, use this parameter to specify a taint toleration key, value, and effect for Scanner. |
|
Specify a node selector label as |
| If the node selector selects tainted nodes, use this parameter to specify a taint toleration key, value, and effect for Scanner DB. |
| The memory request for the Scanner container. Use this parameter to override the default value. |
| The CPU request for the Scanner container. Use this parameter to override the default value. |
| The memory limit for the Scanner container. Use this parameter to override the default value. |
| The CPU limit for the Scanner container. Use this parameter to override the default value. |
| The memory request for the Scanner DB container. Use this parameter to override the default value. |
| The CPU request for the Scanner DB container. Use this parameter to override the default value. |
| The memory limit for the Scanner DB container. Use this parameter to override the default value. |
| The CPU limit for the Scanner DB container. Use this parameter to override the default value. |
|
If you set this option to |
6.4.1.3.1.1. Environment variables
You can specify environment variables for Sensor and Admission controller in the following format:
customize: envVars: ENV_VAR1: "value1" ENV_VAR2: "value2"
The customize
setting allows you to specify custom Kubernetes metadata (labels and annotations) for all objects created by this Helm chart and additional pod labels, pod annotations, and container environment variables for workloads.
The configuration is hierarchical, in the sense that metadata defined at a more generic scope (for example, for all objects) can be overridden by metadata defined at a narrower scope (for example, only for the Sensor deployment).
6.4.1.3.2. Installing the secured-cluster-services Helm chart
After you configure the values-public.yaml
and values-private.yaml
files, install the secured-cluster-services
Helm chart to deploy the per-cluster and per-node components (Sensor, Admission controller, Collector, and Scanner-slim).
To install Collector on systems that have Unified Extensible Firmware Interface (UEFI) and that have Secure Boot enabled, you must use eBPF probes because kernel modules are unsigned, and the UEFI firmware cannot load unsigned packages. Collector identifies Secure Boot status at the start and switches to eBPF probes if required.
Prerequisites
- You must have generated an RHACS init bundle for your cluster.
-
You must have access to the Red Hat Container Registry and a pull secret for authentication. For information about downloading images from
registry.redhat.io
, see Red Hat Container Registry Authentication. -
You must have the Central API Endpoint, including the address and the port number. You can view this information by choosing Advanced Cluster Security
ACS Instances from the cloud console navigation menu, then clicking the ACS instance you created.
Procedure
Run the following command:
$ helm install -n stackrox \ --create-namespace stackrox-secured-cluster-services rhacs/secured-cluster-services \ -f <name_of_cluster_init_bundle.yaml> \ -f <path_to_values_public.yaml> -f <path_to_values_private.yaml> \ 1 --set imagePullSecrets.username=<username> \ 2 --set imagePullSecrets.password=<password> 3
To deploy secured-cluster-services
Helm chart by using a continuous integration (CI) system, pass the init bundle YAML file as an environment variable to the helm install
command:
$ helm install ... -f <(echo "$INIT_BUNDLE_YAML_SECRET") 1
- 1
- If you are using base64 encoded variables, use the
helm install … -f <(echo "$INIT_BUNDLE_YAML_SECRET" | base64 --decode)
command instead.
6.4.1.4. Changing configuration options after deploying the secured-cluster-services Helm chart
You can make changes to any configuration options after you have deployed the secured-cluster-services
Helm chart.
Procedure
-
Update the
values-public.yaml
andvalues-private.yaml
configuration files with new values. Run the
helm upgrade
command and specify the configuration files using the-f
option:$ helm upgrade -n stackrox \ stackrox-secured-cluster-services rhacs/secured-cluster-services \ --reuse-values \ 1 -f <path_to_values_public.yaml> \ -f <path_to_values_private.yaml>
- 1
- You must specify the
--reuse-values
parameter, otherwise the Helm upgrade command resets all previously configured settings.
NoteYou can also specify configuration values using the
--set
or--set-file
parameters. However, these options are not saved, and it requires you to manually specify all the options again whenever you make changes.
6.4.2. Installing RHACS on secured clusters by using the roxctl CLI
To install RHACS on secured clusters by using the CLI, perform the following steps:
-
Install the
roxctl
CLI. - Install Sensor.
6.4.2.1. Installing the roxctl CLI
You must first download the binary. You can install roxctl
on Linux, Windows, or macOS.
6.4.2.1.1. Installing the roxctl CLI on Linux
You can install the roxctl
CLI binary on Linux by using the following procedure.
Procedure
Download the latest version of the
roxctl
CLI:$ curl -O https://mirror.openshift.com/pub/rhacs/assets/4.2.5/bin/Linux/roxctl
Make the
roxctl
binary executable:$ chmod +x roxctl
Place the
roxctl
binary in a directory that is on yourPATH
:To check your
PATH
, execute the following command:$ echo $PATH
Verification
Verify the
roxctl
version you have installed:$ roxctl version
6.4.2.1.2. Installing the roxctl CLI on macOS
You can install the roxctl
CLI binary on macOS by using the following procedure.
Procedure
Download the latest version of the
roxctl
CLI:$ curl -O https://mirror.openshift.com/pub/rhacs/assets/4.2.5/bin/Darwin/roxctl
Remove all extended attributes from the binary:
$ xattr -c roxctl
Make the
roxctl
binary executable:$ chmod +x roxctl
Place the
roxctl
binary in a directory that is on yourPATH
:To check your
PATH
, execute the following command:$ echo $PATH
Verification
Verify the
roxctl
version you have installed:$ roxctl version
6.4.2.1.3. Installing the roxctl CLI on Windows
You can install the roxctl
CLI binary on Windows by using the following procedure.
Procedure
Download the latest version of the
roxctl
CLI:$ curl -O https://mirror.openshift.com/pub/rhacs/assets/4.2.5/bin/Windows/roxctl.exe
Verification
Verify the
roxctl
version you have installed:$ roxctl version
6.4.2.2. Installing Sensor
To monitor a cluster, you must deploy Sensor. You must deploy Sensor into each cluster that you want to monitor. The following steps describe adding Sensor by using the RHACS portal.
Prerequisites
- You must have already installed Central services, or you can access Central services by selecting your ACS instance on Red Hat Advanced Cluster Security Cloud Service (RHACS Cloud Service).
Procedure
-
On your secured cluster, in the RHACS portal, navigate to Platform Configuration
Clusters. - Select + New Cluster.
- Specify a name for the cluster.
Provide appropriate values for the fields based on where you are deploying the Sensor.
-
Enter the Central API Endpoint, including the address and the port number. You can view this information again in the Red Hat Hybrid Cloud Console by choosing Advanced Cluster Security
ACS Instances, and then clicking the ACS instance you created.
-
Enter the Central API Endpoint, including the address and the port number. You can view this information again in the Red Hat Hybrid Cloud Console by choosing Advanced Cluster Security
- Click Next to continue with the Sensor setup.
Click Download YAML File and Keys to download the cluster bundle (zip archive).
ImportantThe cluster bundle zip archive includes unique configurations and keys for each cluster. Do not reuse the same files in another cluster.
From a system that has access to the monitored cluster, unzip and run the
sensor
script from the cluster bundle:$ unzip -d sensor sensor-<cluster_name>.zip
$ ./sensor/sensor.sh
If you get a warning that you do not have the required permissions to deploy Sensor, follow the on-screen instructions, or contact your cluster administrator for assistance.
After Sensor is deployed, it contacts Central and provides cluster information.
Verification
Return to the RHACS portal and check if the deployment is successful. If successful, when viewing your list of clusters in Platform Configuration
Clusters, the cluster status displays a green checkmark and a Healthy status. If you do not see a green checkmark, use the following command to check for problems: On Kubernetes, enter the following command:
$ kubectl get pod -n stackrox -w
- Click Finish to close the window.
After installation, Sensor starts reporting security information to RHACS and the RHACS portal dashboard begins showing deployments, images, and policy violations from the cluster on which you have installed the Sensor.
6.5. Verifying installation of secured clusters
After installing RHACS Cloud Service, you can perform some steps to verify that the installation was successful.
To verify installation, access your ACS Console from the Red Hat Hybrid Cloud Console. The Dashboard displays the number of clusters that RHACS Cloud Service is monitoring, along with information about nodes, deployments, images, and violations.
If no data appears in the ACS Console:
-
Ensure that at least one secured cluster is connected to your RHACS Cloud Service instance. For more information, see instructions for installing by using Helm charts or by using the
roxctl
CLI. - Examine your Sensor pod logs to ensure that the connection to your RHACS Cloud Service instance is successful.
-
Examine the values in the
SecuredCluster
API in the Operator on your local cluster to ensure that the Central API Endpoint has been entered correctly. This value should be the same value as shown in the ACS instance details in the Red Hat Hybrid Cloud Console.