Chapter 8. Setting up RHACS Cloud Service with Kubernetes secured clusters
8.1. Creating an RHACS Cloud Service instance for Kubernetes clusters
Access Red Hat Advanced Cluster Security Cloud Service (RHACS Cloud Service) by selecting an instance in the Red Hat Hybrid Cloud Console. An ACS instance contains the RHACS Cloud Service management interface and services that Red Hat configures and manages for you. The management interface connects to your secured clusters, which contain the services that scan and collect information about vulnerabilities. One instance can connect to and monitor many clusters.
8.1.1. Creating an instance in the console
In the Red Hat Hybrid Cloud Console, create an ACS instance to connect to your secured clusters.
Procedure
To create an ACS instance:
- Log in to the Red Hat Hybrid Cloud Console.
-
From the navigation menu, select Advanced Cluster Security
ACS Instances. Select Create ACS instance and enter information into the displayed fields or select the appropriate option from the drop-down list:
- Name: Enter the name of your ACS instance. An ACS instance contains the RHACS Central component, also referred to as "Central", which includes the RHACS Cloud Service management interface and services that are configured and managed by Red Hat. You manage your secured clusters that communicate with Central. You can connect many secured clusters to one instance.
- Cloud provider: The cloud provider where Central is located. Select AWS.
Cloud region: The region for your cloud provider where Central is located. Select one of the following regions:
- US-East, N. Virginia
- Europe, Ireland
- Availability zones: Use the default value (Multi).
- Click Create instance.
8.1.2. Next steps
-
On each Kubernetes cluster you want to secure, install secured cluster resources by using Helm charts or the
roxctl
CLI.
8.2. Generating an init bundle for Kubernetes secured clusters
Before you install the SecuredCluster
resource on a cluster, you must create an init bundle. The cluster that has SecuredCluster
installed and configured then uses this bundle to authenticate with the ACS Console. You can create an init bundle by using either the RHACS portal or the roxctl
CLI. You then apply the init bundle by using it to create resources.
8.2.1. Generating an init bundle by using the RHACS portal
You can create an init bundle containing secrets by using the RHACS portal.
You must have the Admin
user role to create an init bundle.
Procedure
- Find the address of the RHACS portal as described in "Verifying Central installation using the Operator method".
- Log in to the RHACS portal.
-
If you do not have secured clusters, the Platform Configuration
Clusters page appears. - Click Create init bundle.
- Enter a name for the cluster init bundle.
- Select your platform.
- Select the installation method you will use for your secured clusters: Operator or Helm chart.
Click Download to generate and download the init bundle, which is created in the form of a YAML file. You can use one init bundle and its corresponding YAML file for all secured clusters if you are using the same installation method.
ImportantStore this bundle securely because it contains secrets.
- Apply the init bundle by using it to create resources on the secured cluster.
- Install secured cluster services on each cluster.
8.2.2. Generating an init bundle by using the roxctl CLI
You can create an init bundle with secrets by using the roxctl
CLI.
You must have the Admin
user role to create init bundles.
Prerequisites
You have configured the
ROX_API_TOKEN
and theROX_CENTRAL_ADDRESS
environment variables:Set the
ROX_API_TOKEN
by running the following command:$ export ROX_API_TOKEN=<api_token>
Set the
ROX_CENTRAL_ADDRESS
environment variable by running the following command:$ export ROX_CENTRAL_ADDRESS=<address>:<port_number>
In RHACS Cloud Service, when using roxctl
commands that require the Central address, use the Central instance address as displayed in the Instance Details section of the Red Hat Hybrid Cloud Console. For example, use acs-ABCD12345.acs.rhcloud.com
instead of acs-data-ABCD12345.acs.rhcloud.com
.
Procedure
To generate a cluster init bundle containing secrets for Helm installations, run the following command:
$ roxctl -e "$ROX_CENTRAL_ADDRESS" \ central init-bundles generate --output \ <cluster_init_bundle_name> cluster_init_bundle.yaml
To generate a cluster init bundle containing secrets for Operator installations, run the following command:
$ roxctl -e "$ROX_CENTRAL_ADDRESS" \ central init-bundles generate --output-secrets \ <cluster_init_bundle_name> cluster_init_bundle.yaml
ImportantEnsure that you store this bundle securely because it contains secrets. You can use the same bundle to set up multiple secured clusters.
8.2.3. Next steps
8.3. Applying an init bundle for Kubernetes secured clusters
Apply the init bundle by using it to create resources.
8.3.1. Applying the init bundle on the secured cluster
Before you configure a secured cluster, you must apply the init bundle by using it to create the required resources on the secured cluster. Applying the init bundle allows the services on the secured cluster to communicate with RHACS Cloud Service.
If you are installing by using Helm charts, do not perform this step. Complete the installation by using Helm; See "Installing RHACS on secured clusters by using Helm charts" in the additional resources section.
Prerequisites
- You must have generated an init bundle containing secrets.
-
You must have created the
stackrox
project, or namespace, on the cluster where secured cluster services will be installed. Usingstackrox
for the project is not required, but ensures that vulnerabilities for RHACS processes are not reported when scanning your clusters.
Procedure
To create resources, perform only one of the following steps:
-
Create resources using the OpenShift Container Platform web console: In the OpenShift Container Platform web console, make sure that you are in the
stackrox
namespace. In the top menu, click + to open the Import YAML page. You can drag the init bundle file or copy and paste its contents into the editor, and then click Create. When the command is complete, the display shows that thecollector-tls
,sensor-tls
, and admission-control-tls` resources were created. Create resources using the Red Hat OpenShift CLI: Using the Red Hat OpenShift CLI, run the following command to create the resources:
$ oc create -f <init_bundle>.yaml \1 -n <stackrox> 2
Using the
kubectl
CLI, run the following commands to create the resources:$ kubectl create namespace stackrox 1 $ kubectl create -f <init_bundle>.yaml \2 -n <stackrox> 3
Verification
Restart Sensor to pick up the new certificates.
For more information about how to restart Sensor, see "Restarting the Sensor container" in the "Additional resources" section.
8.3.2. Next steps
- Install RHACS secured cluster services in all clusters that you want to monitor.
8.3.3. Additional resources
8.4. Installing secured cluster services from RHACS Cloud Service on Kubernetes clusters
You can install RHACS Cloud Service on your secured clusters by using one of the following methods:
- By using Helm charts
-
By using the
roxctl
CLI (do not use this method unless you have a specific installation need that requires using it)
8.4.1. Installing RHACS Cloud Service on secured clusters by using Helm charts
You can install RHACS on secured clusters by using Helm charts with no customization, by using Helm charts with the default values, or by using Helm charts with customizations of configuration parameters.
First, ensure that you add the Helm chart repository.
8.4.1.1. Adding the Helm chart repository
Procedure
Add the RHACS charts repository.
$ helm repo add rhacs https://mirror.openshift.com/pub/rhacs/charts/
The Helm repository for Red Hat Advanced Cluster Security for Kubernetes includes Helm charts for installing different components, including:
Secured Cluster Services Helm chart (
secured-cluster-services
) for installing the per-cluster and per-node components (Sensor, Admission Controller, Collector, and Scanner-slim).NoteDeploy the per-cluster components into each cluster that you want to monitor and deploy the per-node components in all nodes that you want to monitor.
Verification
Run the following command to verify the added chart repository:
$ helm search repo -l rhacs/
8.4.1.2. Installing RHACS Cloud Service on secured clusters by using Helm charts without customizations
8.4.1.2.1. Installing the secured-cluster-services Helm chart without customization
Use the following instructions to install the secured-cluster-services
Helm chart to deploy the per-cluster and per-node components (Sensor, Admission controller, Collector, and Scanner-slim).
Prerequisites
- You must have generated an RHACS init bundle for your cluster.
-
You must have access to the Red Hat Container Registry and a pull secret for authentication. For information about downloading images from
registry.redhat.io
, see Red Hat Container Registry Authentication. -
You must have the Central API Endpoint address. You can view this information by choosing Advanced Cluster Security
ACS Instances from the Red Hat Hybrid Cloud Console navigation menu, then clicking the ACS instance you created.
8.4.1.3. Configuring the secured-cluster-services Helm chart with customizations
Procedure
This section describes Helm chart configuration parameters that you can use with the helm install
and helm upgrade
commands. You can specify these parameters by using the --set
option or by creating YAML configuration files.
Create the following files for configuring the Helm chart for installing Red Hat Advanced Cluster Security for Kubernetes:
-
Public configuration file
values-public.yaml
: Use this file to save all non-sensitive configuration options. -
Private configuration file
values-private.yaml
: Use this file to save all sensitive configuration options. Ensure that you store this file securely.
While using the secured-cluster-services
Helm chart, do not modify the values.yaml
file that is part of the chart.
8.4.1.3.1. Configuration parameters
Parameter | Description |
---|---|
| Name of your cluster. |
|
Address of the Central endpoint. If you are using a non-gRPC capable load balancer, use the WebSocket protocol by prefixing the endpoint address with |
| Address of the Sensor endpoint including port number. |
| Image pull policy for the Sensor container. |
| The internal service-to-service TLS certificate that Sensor uses. |
| The internal service-to-service TLS certificate key that Sensor uses. |
| The memory request for the Sensor container. Use this parameter to override the default value. |
| The CPU request for the Sensor container. Use this parameter to override the default value. |
| The memory limit for the Sensor container. Use this parameter to override the default value. |
| The CPU limit for the Sensor container. Use this parameter to override the default value. |
|
Specify a node selector label as |
| If the node selector selects tainted nodes, use this parameter to specify a taint toleration key, value, and effect for Sensor. This parameter is mainly used for infrastructure nodes. |
|
The name of the |
| The name of the Collector image. |
| The address of the registry you are using for the main image. |
| The address of the registry you are using for the Collector image. |
| The address of the registry you are using for the Scanner image. |
| The address of the registry you are using for the Scanner DB image. |
| The address of the registry you are using for the Scanner V4 image. |
| The address of the registry you are using for the Scanner V4 DB image. |
|
Image pull policy for |
| Image pull policy for the Collector images. |
|
Tag of |
|
Tag of |
|
Either |
| Image pull policy for the Collector container. |
| Image pull policy for the Compliance container. |
|
If you specify |
| The memory request for the Collector container. Use this parameter to override the default value. |
| The CPU request for the Collector container. Use this parameter to override the default value. |
| The memory limit for the Collector container. Use this parameter to override the default value. |
| The CPU limit for the Collector container. Use this parameter to override the default value. |
| The memory request for the Compliance container. Use this parameter to override the default value. |
| The CPU request for the Compliance container. Use this parameter to override the default value. |
| The memory limit for the Compliance container. Use this parameter to override the default value. |
| The CPU limit for the Compliance container. Use this parameter to override the default value. |
| The internal service-to-service TLS certificate that Collector uses. |
| The internal service-to-service TLS certificate key that Collector uses. |
|
This setting controls whether Kubernetes is configured to contact Red Hat Advanced Cluster Security for Kubernetes with |
|
When you set this parameter as |
|
This setting controls whether the cluster is configured to contact Red Hat Advanced Cluster Security for Kubernetes with |
| This setting controls whether Red Hat Advanced Cluster Security for Kubernetes evaluates policies; if it is disabled, all AdmissionReview requests are automatically accepted. |
|
This setting controls the behavior of the admission control service. You must specify |
|
If you set this option to |
|
Set it to |
|
Use this parameter to specify the maximum number of seconds RHACS must wait for an admission review before marking it as fail open. If the admission webhook does not receive information that it is requesting before the end of the timeout period, it fails, but in fail open status, it still allows the operation to succeed. For example, the admission controller would allow a deployment to be created even if a scan had timed out and RHACS could not determine if the deployment violated a policy. Beginning in release 4.5, Red Hat reduced the default timeout setting for the RHACS admission controller webhooks from 20 seconds to 10 seconds, resulting in an effective timeout of 12 seconds within the |
| The memory request for the Admission Control container. Use this parameter to override the default value. |
| The CPU request for the Admission Control container. Use this parameter to override the default value. |
| The memory limit for the Admission Control container. Use this parameter to override the default value. |
| The CPU limit for the Admission Control container. Use this parameter to override the default value. |
|
Specify a node selector label as |
| If the node selector selects tainted nodes, use this parameter to specify a taint toleration key, value, and effect for Admission Control. This parameter is mainly used for infrastructure nodes. |
| The internal service-to-service TLS certificate that Admission Control uses. |
| The internal service-to-service TLS certificate key that Admission Control uses. |
|
Use this parameter to override the default |
|
If you specify |
|
Specify |
|
Specify |
|
Deprecated. Specify |
| Resource specification for Sensor. |
| Resource specification for Admission controller. |
| Resource specification for Collector. |
| Resource specification for Collector’s Compliance container. |
|
If you set this option to |
|
If you set this option to |
|
If you set this option to |
| If the node selector selects tainted nodes, use this parameter to specify a taint toleration key, value, and effect for Scanner DB. |
| Resource specification for Collector’s Compliance container. |
| Setting this parameter allows you to modify the scanner log level. Use this option only for troubleshooting purposes. |
|
If you set this option to |
| The minimum number of replicas for autoscaling. Defaults to 2. |
| The maximum number of replicas for autoscaling. Defaults to 5. |
|
Specify a node selector label as |
| If the node selector selects tainted nodes, use this parameter to specify a taint toleration key, value, and effect for Scanner. |
|
Specify a node selector label as |
| If the node selector selects tainted nodes, use this parameter to specify a taint toleration key, value, and effect for Scanner DB. |
| The memory request for the Scanner container. Use this parameter to override the default value. |
| The CPU request for the Scanner container. Use this parameter to override the default value. |
| The memory limit for the Scanner container. Use this parameter to override the default value. |
| The CPU limit for the Scanner container. Use this parameter to override the default value. |
| The memory request for the Scanner DB container. Use this parameter to override the default value. |
| The CPU request for the Scanner DB container. Use this parameter to override the default value. |
| The memory limit for the Scanner DB container. Use this parameter to override the default value. |
| The CPU limit for the Scanner DB container. Use this parameter to override the default value. |
|
If you set this option to |
|
To provide security at the network level, RHACS creates default Warning Disabling creation of default network policies can break communication between RHACS components. If you disable creation of default policies, you must create your own network policies to allow this communication. |
8.4.1.3.1.1. Environment variables
You can specify environment variables for Sensor and Admission controller in the following format:
customize: envVars: ENV_VAR1: "value1" ENV_VAR2: "value2"
The customize
setting allows you to specify custom Kubernetes metadata (labels and annotations) for all objects created by this Helm chart and additional pod labels, pod annotations, and container environment variables for workloads.
The configuration is hierarchical, in the sense that metadata defined at a more generic scope (for example, for all objects) can be overridden by metadata defined at a narrower scope (for example, only for the Sensor deployment).
8.4.1.3.2. Installing the secured-cluster-services Helm chart with customizations
After you configure the values-public.yaml
and values-private.yaml
files, install the secured-cluster-services
Helm chart to deploy the following per-cluster and per-node components:
- Sensor
- Admission controller
- Collector
- Scanner: optional for secured clusters when the StackRox Scanner is installed
- Scanner DB: optional for secured clusters when the StackRox Scanner is installed
- Scanner V4 Indexer and Scanner V4 DB: optional for secured clusters when Scanner V4 is installed
Prerequisites
- You must have generated an RHACS init bundle for your cluster.
-
You must have access to the Red Hat Container Registry and a pull secret for authentication. For information about downloading images from
registry.redhat.io
, see Red Hat Container Registry Authentication. -
You must have the Central API Endpoint address. You can view this information by choosing Advanced Cluster Security
ACS Instances from the Red Hat Hybrid Cloud Console navigation menu, then clicking the RHACS instance you created.
Procedure
Run the following command:
$ helm install -n stackrox \ --create-namespace stackrox-secured-cluster-services rhacs/secured-cluster-services \ -f <name_of_cluster_init_bundle.yaml> \ -f <path_to_values_public.yaml> -f <path_to_values_private.yaml> \1 --set imagePullSecrets.username=<username> \2 --set imagePullSecrets.password=<password> 3
To deploy secured-cluster-services
Helm chart by using a continuous integration (CI) system, pass the init bundle YAML file as an environment variable to the helm install
command:
$ helm install ... -f <(echo "$INIT_BUNDLE_YAML_SECRET") 1
- 1
- If you are using base64 encoded variables, use the
helm install … -f <(echo "$INIT_BUNDLE_YAML_SECRET" | base64 --decode)
command instead.
8.4.1.4. Changing configuration options after deploying the secured-cluster-services Helm chart
You can make changes to any configuration options after you have deployed the secured-cluster-services
Helm chart.
When using the helm upgrade
command to make changes, the following guidelines and requirements apply:
-
You can also specify configuration values using the
--set
or--set-file
parameters. However, these options are not saved, and you must manually specify all the options again whenever you make changes. Some changes, such as enabling a new component like Scanner V4, require new certificates to be issued for the component. Therefore, you must provide a CA when making these changes.
-
If the CA was generated by the Helm chart during the initial installation, you must retrieve these automatically generated values from the cluster and provide them to the
helm upgrade
command. The post-installation notes of thecentral-services
Helm chart include a command for retrieving the automatically generated values. -
If the CA was generated outside of the Helm chart and provided during the installation of the
central-services
chart, then you must perform that action again when using thehelm upgrade
command, for example, by using the--reuse-values
flag with thehelm upgrade
command.
-
If the CA was generated by the Helm chart during the initial installation, you must retrieve these automatically generated values from the cluster and provide them to the
Procedure
-
Update the
values-public.yaml
andvalues-private.yaml
configuration files with new values. Run the
helm upgrade
command and specify the configuration files using the-f
option:$ helm upgrade -n stackrox \ stackrox-secured-cluster-services rhacs/secured-cluster-services \ --reuse-values \1 -f <path_to_values_public.yaml> \ -f <path_to_values_private.yaml>
- 1
- If you have modified values that are not included in the
values_public.yaml
andvalues_private.yaml
files, include the--reuse-values
parameter.
8.4.2. Installing RHACS on secured clusters by using the roxctl CLI
To install RHACS on secured clusters by using the CLI, perform the following steps:
-
Install the
roxctl
CLI. - Install Sensor.
8.4.2.1. Installing the roxctl CLI
You must first download the binary. You can install roxctl
on Linux, Windows, or macOS.
8.4.2.1.1. Installing the roxctl CLI on Linux
You can install the roxctl
CLI binary on Linux by using the following procedure.
roxctl
CLI for Linux is available for amd64
, arm64
, ppc64le
, and s390x
architectures.
Procedure
Determine the
roxctl
architecture for the target operating system:$ arch="$(uname -m | sed "s/x86_64//")"; arch="${arch:+-$arch}"
Download the
roxctl
CLI:$ curl -L -f -o roxctl "https://mirror.openshift.com/pub/rhacs/assets/4.6.2/bin/Linux/roxctl${arch}"
Make the
roxctl
binary executable:$ chmod +x roxctl
Place the
roxctl
binary in a directory that is on yourPATH
:To check your
PATH
, execute the following command:$ echo $PATH
Verification
Verify the
roxctl
version you have installed:$ roxctl version
8.4.2.1.2. Installing the roxctl CLI on macOS
You can install the roxctl
CLI binary on macOS by using the following procedure.
roxctl
CLI for macOS is available for amd64
and arm64
architectures.
Procedure
Determine the
roxctl
architecture for the target operating system:$ arch="$(uname -m | sed "s/x86_64//")"; arch="${arch:+-$arch}"
Download the
roxctl
CLI:$ curl -L -f -o roxctl "https://mirror.openshift.com/pub/rhacs/assets/4.6.2/bin/Darwin/roxctl${arch}"
Remove all extended attributes from the binary:
$ xattr -c roxctl
Make the
roxctl
binary executable:$ chmod +x roxctl
Place the
roxctl
binary in a directory that is on yourPATH
:To check your
PATH
, execute the following command:$ echo $PATH
Verification
Verify the
roxctl
version you have installed:$ roxctl version
8.4.2.1.3. Installing the roxctl CLI on Windows
You can install the roxctl
CLI binary on Windows by using the following procedure.
roxctl
CLI for Windows is available for the amd64
architecture.
Procedure
Download the
roxctl
CLI:$ curl -f -O https://mirror.openshift.com/pub/rhacs/assets/4.6.2/bin/Windows/roxctl.exe
Verification
Verify the
roxctl
version you have installed:$ roxctl version
8.4.2.2. Installing Sensor
To monitor a cluster, you must deploy Sensor. You must deploy Sensor into each cluster that you want to monitor. This installation method is also called the manifest installation method.
To perform an installation by using the manifest installation method, follow only one of the following procedures:
- Use the RHACS web portal to download the cluster bundle, and then extract and run the sensor script.
-
Use the
roxctl
CLI to generate the required sensor configuration for your OpenShift Container Platform cluster and associate it with your Central instance.
Prerequisites
- You must have already installed Central services, or you can access Central services by selecting your ACS instance on Red Hat Advanced Cluster Security Cloud Service (RHACS Cloud Service).
8.4.2.2.1. Manifest installation method by using the web portal
Procedure
-
On your secured cluster, in the RHACS portal, go to Platform Configuration
Clusters. -
Select Secure a cluster
Legacy installation method. - Specify a name for the cluster.
Provide appropriate values for the fields based on where you are deploying the Sensor.
-
Enter the Central API Endpoint address. You can view this information by choosing Advanced Cluster Security
ACS Instances from the Red Hat Hybrid Cloud Console navigation menu, then clicking the RHACS instance you created.
-
Enter the Central API Endpoint address. You can view this information by choosing Advanced Cluster Security
- Click Next to continue with the Sensor setup.
Click Download YAML File and Keys to download the cluster bundle (zip archive).
ImportantThe cluster bundle zip archive includes unique configurations and keys for each cluster. Do not reuse the same files in another cluster.
From a system that has access to the monitored cluster, extract and run the
sensor
script from the cluster bundle:$ unzip -d sensor sensor-<cluster_name>.zip
$ ./sensor/sensor.sh
If you get a warning that you do not have the required permissions to deploy Sensor, follow the on-screen instructions, or contact your cluster administrator for help.
After Sensor is deployed, it contacts Central and provides cluster information.
8.4.2.2.2. Manifest installation by using the roxctl CLI
Procedure
Generate the required sensor configuration for your OpenShift Container Platform cluster and associate it with your Central instance by running the following command:
$ roxctl sensor generate openshift --openshift-version <ocp_version> --name <cluster_name> --central "$ROX_ENDPOINT" 1
- 1
- For the
--openshift-version
option, specify the major OpenShift Container Platform version number for your cluster. For example, specify3
for OpenShift Container Platform version3.x
and specify4
for OpenShift Container Platform version4.x
.
From a system that has access to the monitored cluster, extract and run the
sensor
script from the cluster bundle:$ unzip -d sensor sensor-<cluster_name>.zip
$ ./sensor/sensor.sh
If you get a warning that you do not have the required permissions to deploy Sensor, follow the on-screen instructions, or contact your cluster administrator for help.
After Sensor is deployed, it contacts Central and provides cluster information.
Verification
Return to the RHACS portal and check if the deployment is successful. If successful, when viewing your list of clusters in Platform Configuration
Clusters, the cluster status displays a green checkmark and a Healthy status. If you do not see a green checkmark, use the following command to check for problems: On Kubernetes, enter the following command:
$ kubectl get pod -n stackrox -w
- Click Finish to close the window.
After installation, Sensor starts reporting security information to RHACS and the RHACS portal dashboard begins showing deployments, images, and policy violations from the cluster on which you have installed the Sensor.
8.5. Verifying installation of secured clusters
After installing RHACS Cloud Service, you can perform some steps to verify that the installation was successful.
To verify installation, access your ACS Console from the Red Hat Hybrid Cloud Console. The Dashboard displays the number of clusters that RHACS Cloud Service is monitoring, along with information about nodes, deployments, images, and violations.
If no data appears in the ACS Console:
-
Ensure that at least one secured cluster is connected to your RHACS Cloud Service instance. For more information, see instructions for installing by using Helm charts or by using the
roxctl
CLI. - Examine your Sensor pod logs to ensure that the connection to your RHACS Cloud Service instance is successful.
-
Examine the values in the
SecuredCluster
API in the Operator on your local cluster to ensure that the Central API Endpoint has been entered correctly. This value should be the same value as shown in the ACS instance details in the Red Hat Hybrid Cloud Console.