Chapter 1. Red Hat Advanced Cluster Security for Kubernetes architecture
Discover Red Hat Advanced Cluster Security for Kubernetes architecture and concepts.
1.1. Red Hat Advanced Cluster Security for Kubernetes architecture overview
Red Hat Advanced Cluster Security for Kubernetes (RHACS) uses a distributed architecture that supports high-scale deployments and is optimized to minimize the impact on the underlying OpenShift Container Platform or Kubernetes nodes.
RHACS architecture
The following graphic shows the architecture with the StackRox Scanner and Scanner V4 components. Installation of Scanner V4 is optional, but provides additional benefits.
You install RHACS as a set of containers in your OpenShift Container Platform or Kubernetes cluster. RHACS includes the following services:
- Central services you install on one cluster
- Secured cluster services you install on each cluster you want to secure by RHACS
In addition to these primary services, RHACS also interacts with other external components to enhance your clusters' security.
Installation differences
When you install RHACS on OpenShift Container Platform by using the Operator, RHACS installs a lightweight version of Scanner on every secured cluster. The lightweight Scanner enables the scanning of images in the integrated OpenShift image registry. When you install RHACS on OpenShift Container Platform or Kubernetes by using the Helm install method with the default values, the lightweight version of Scanner is not installed. To install the lightweight Scanner on the secured cluster by using Helm, you must set the scanner.disable=false parameter. You cannot install the lightweight Scanner by using the roxctl installation method.
Additional resources
1.2. Central services
You install Central services on a single cluster. These services include the following components:
- Central: Central is the RHACS application management interface and services. It handles API interactions and user interface (RHACS Portal) access. You can use the same Central instance to secure multiple OpenShift Container Platform or Kubernetes clusters.
- Central DB: Central DB is the database for RHACS and handles all data persistence. It is currently based on PostgreSQL 13.
Scanner V4: Beginning with version 4.4, RHACS contains the Scanner V4 vulnerability scanner for scanning container images. Scanner V4 is built on ClairCore, which also powers the Clair scanner. Scanner V4 supports scanning of language and OS-specific image components. For version 4.4, you must use this scanner in conjunction with the StackRox Scanner to provide node and platform scanning capabilities until Scanner V4 support those capabilities. Scanner V4 contains the Indexer, Matcher, and DB components.
- Scanner V4 Indexer: The Scanner V4 Indexer performs image indexing, previously known as image analysis. Given an image and registry credentials, the Indexer pulls the image from the registry. It finds the base operating system, if it exists, and looks for packages. It stores and outputs an index report, which contains the findings for the given image.
- Scanner V4 Matcher: The Scanner V4 Matcher performs vulnerability matching. If the Central services Scanner V4 Indexer indexed the image, then the Matcher fetches the index report from the Indexer and matches the report with the vulnerabilities stored in the Scanner V4 database. If a Secured Cluster services Scanner V4 Indexer performed the indexing, then the Matcher uses the index report that was sent from that Indexer, and then matches against vulnerabilities. The Matcher also fetches vulnerability data and updates the Scanner V4 database with the latest vulnerability data. The Scanner V4 Matcher outputs a vulnerability report, which contains the final results of an image.
- Scanner V4 DB: This database stores information for Scanner V4, including all vulnerability data and index reports. A persistent volume claim (PVC) is required for Scanner V4 DB on the cluster where Central is installed.
- StackRox Scanner: The StackRox Scanner is the default scanner in RHACS. Version 4.4 adds a new scanner, Scanner V4. The StackRox Scanner originates from a fork of the Clair v2 open source scanner. You must continue using this scanner for RHCOS node scanning and platform scanning.
- Scanner-DB: This database contains data for the StackRox Scanner.
RHACS scanners analyze each image layer to determine the base operating system and identify programming language packages and packages that were installed by the operating system package manager. They match the findings against known vulnerabilities from various vulnerability sources. In addition, the StackRox Scanner identifies vulnerabilities in the node’s operating system and platform. These capabilities are planned for Scanner V4 in a future release.
1.2.1. Vulnerability data sources
Sources for vulnerabilities depend on the scanner that is used in your system. RHACS contains two scanners: StackRox Scanner and Scanner V4. StackRox Scanner is the default scanner and is deprecated beginning with release 4.6. Scanner V4 was introduced in release 4.4 and is the recommended image scanner.
1.2.1.1. StackRox Scanner sources
StackRox Scanner uses the following vulnerability sources:
- Red Hat OVAL v2
- Alpine Security Database
- Data tracked in Amazon Linux Security Center
- Debian Security Tracker
- Ubuntu CVE Tracker
NVD: This is used for various purposes such as filling in information gaps when vendors do not provide information. For example, Alpine does not provide a description, CVSS score, severity, or published date.
NoteThis product uses the NVD API but is not endorsed or certified by the NVD.
- Linux manual entries and NVD manual entries: The upstream StackRox project maintains a set of vulnerabilities that might not be discovered due to data formatting from other sources or absence of data.
- repository-to-cpe.json: Maps RPM repositories to their related CPEs, which is required for matching vulnerabilities for RHEL-based images.
1.2.1.2. Scanner V4 sources
Scanner V4 uses the following vulnerability sources:
- Red Hat VEX
Used with release 4.6 and later. This source provides vulnerability data in Vulnerability Exploitability eXchange(VEX) format. RHACS takes advantage of VEX benefits to significantly decrease the time needed for the initial loading of vulnerability data, and the space needed to store vulnerability data.
RHACS might list a different number of vulnerabilities when you are scanning with a RHACS version that uses OVAL, such as RHACS version 4.5, and a version that uses VEX, such as version 4.6. For example, RHACS no longer displays vulnerabilities with a status of "under investigation," while these vulnerabilities were included with previous versions that used OVAL data.
For more information about Red Hat security data, including information about the use of OVAL, Common Security Advisory Framework Version 2.0 (CSAF), and VEX, see The future of Red Hat security data.
- Red Hat CVE Map
- This is used in addition with VEX data for images which appear in the Red Hat Container Catalog.
- OSV
This is used for language-related vulnerabilities, such as Go, Java, JavaScript, Python, and Ruby. This source might provide vulnerability IDs other than CVE IDs for vulnerabilities, such as a GitHub Security Advisory (GHSA) ID.
NoteRHACS uses the OSV database available at OSV.dev under Apache License 2.0.
- NVD
This is used for various purposes such as filling in information gaps when vendors do not provide information. For example, Alpine does not provide a description, CVSS score, severity, or published date.
NoteThis product uses the NVD API but is not endorsed or certified by the NVD.
- Additional vulnerability sources
- Alpine Security Database
- Data tracked in Amazon Linux Security Center
- Debian Security Tracker
- Oracle OVAL
- Photon OVAL
- SUSE OVAL
- Ubuntu OVAL
- StackRox: The upstream StackRox project maintains a set of vulnerabilities that might not be discovered due to data formatting from other sources or absence of data.
- Scanner V4 Indexer sources
Scanner V4 indexer uses the following files to index Red Hat containers:
- repository-to-cpe.json: Maps RPM repositories to their related CPEs, which is required for matching vulnerabilities for RHEL-based images.
- container-name-repos-map.json: This matches container names to their respective repositories.
1.3. Secured cluster services
You install the secured cluster services on each cluster that you want to secure by using the Red Hat Advanced Cluster Security. Secured cluster services include the following components:
- Sensor: Sensor is the service responsible for analyzing and monitoring the cluster. Sensor listens to the OpenShift Container Platform or Kubernetes API and Collector events to report the current state of the cluster. Sensor also triggers deploy-time and runtime violations based on RHACS policies. In addition, Sensor is responsible for all cluster interactions, such as applying network policies, initiating reprocessing of RHACS policies, and interacting with the Admission controller.
- Admission controller: The Admission controller prevents users from creating workloads that violate security policies in RHACS.
- Collector: Collector analyzes and monitors container activity on cluster nodes. It collects container runtime and network activity information and sends the collected data to Sensor.
- StackRox Scanner: In Kubernetes, the secured cluster services include Scanner-slim as an optional component. However, on OpenShift Container Platform, RHACS installs a Scanner-slim version on each secured cluster to scan images in the OpenShift Container Platform integrated registry and optionally other registries.
- Scanner-DB: This database contains data for the StackRox Scanner.
Scanner V4: Scanner V4 components are installed on the secured cluster if enabled.
- Scanner V4 Indexer: The Scanner V4 Indexer performs image indexing, previously known as image analysis. Given an image and registry credentials, the Indexer pulls the image from the registry. It finds the base operating system, if it exists, and looks for packages. It stores and outputs an index report, which contains the findings for the given image.
Scanner V4 DB: This component is installed if Scanner V4 is enabled. This database stores information for Scanner V4, including index reports. For best performance, configure a persistent volume claim (PVC) for Scanner V4 DB.
NoteWhen secured cluster services are installed on the same cluster as Central services and installed in the same namespace, secured cluster services do not deploy Scanner V4 components. Instead, it is assumed that Central services already include a deployment of Scanner V4.
1.4. External components
Red Hat Advanced Cluster Security for Kubernetes (RHACS) interacts with the following external components:
- Third-party systems: You can integrate RHACS with other systems such as CI/CD pipelines, event management (SIEM) systems, logging, email, and more.
-
roxctl:roxctlis a command-line interface (CLI) for running commands on RHACS. - Image registries: You can integrate RHACS with various image registries and use RHACS to scan and view images. RHACS automatically configures registry integrations for active images by using the image pull secrets discovered in secured clusters. However, for scanning inactive images, you must manually configure registry integrations.
-
definitions.stackrox.io: RHACS aggregates the data from various vulnerability feeds at thedefinitions.stackrox.ioendpoint and passes this information to Central. The feeds include general, National Vulnerability Database (NVD) data, and distribution-specific data, such as Alpine, Debian, and Ubuntu. -
collector-modules.stackrox.io: Central reaches out tocollector-modules.stackrox.ioto obtain supported kernel modules and passes on these modules to Collector.
1.5. Interaction between the services
This section explains how RHACS services interact with each other.
| Component | Direction | Component | Description |
|---|---|---|---|
| Central | ⮂ | Scanner V4 Indexer | Central requests the Indexer to download and index (analyze) given images. This process results in an index report. Scanner V4 Indexer requests mapping files from Central that assist the indexing process. |
| Central | ⮂ | Scanner V4 Matcher | Central requests that the Scanner V4 Matcher match given images to known vulnerabilities. This process results in the final scan result: a vulnerability report. Scanner V4 Matcher requests the latest vulnerabilities from Central. |
| Sensor | ⮂ | Scanner V4 Indexer |
|
| Scanner V4 Indexer | → | Image Registries | The Indexer pulls image metadata from registries to determine the layers of the image, and downloads each previously unindexed layer. |
| Scanner V4 Matcher | → | Scanner V4 Indexer | Scanner V4 Matcher requests the results of the image indexing, the index report, from the Indexer. It then uses the report to determine relevant vulnerabilities. This interaction occurs only when the image is indexed in the Central cluster. This interaction does not occur when Scanner V4 is matching vulnerabilities for images indexed in secured clusters. |
| Scanner V4 Indexer | → | Scanner V4 DB | The Indexer stores data related to the indexing results to ensure that image layers are only downloaded and indexed once. This prevents unnecessary network traffic and other resource utilization. |
| Scanner V4 Matcher | → | Scanner V4 DB | Scanner V4 Matcher stores all of its vulnerability data in the database and periodically updates this data. Scanner V4 indexer also queries this data as part of the vulnerability matching process. |
| Sensor | ⮂ | Central | There is bidirectional communication between Central and Sensor. Sensor polls Central periodically to download updates for the sensor bundle configuration. It also sends events for the observed activity for the secured cluster and observed policy violations. Central communicates with Sensor to force reprocessing of all deployments against enabled policies. |
| Collector | ⮂ | Sensor | Collector communicates with Sensor and sends all of the events to the respective Sensor for the cluster. On supported OpenShift Container Platform clusters, Collector analyzes the software packages installed on the nodes and sends them to Sensor so that Scanner can later scan them for vulnerabilities. Collector also requests missing drivers from Sensor. Sensor requests compliance scan results from Collector. Additionally, Sensor receives external Classless Inter-Domain Routing information from Central and pushes it to Collector. |
| Admission controller | ⮂ | Sensor | Sensors send the list of security policies to enforce to Admission controller. Admission controller sends security policy violation alerts to Sensor. Admission controller can also request image scans from Sensor when required. |
| Admission controller | ➞ | Central | It is not common; however, Admission controller can communicate with Central directly if the Central endpoint is known and Sensor is unavailable. |
| Component | Direction | Interacts with | Description |
|---|---|---|---|
| Central | ⮂ | Scanner | There is bidirectional communication between Central and Scanner. Central requests image scans from Scanner, and Scanner requests updates to its CVE database from Central. |
| Central | ➞ |
|
Central connects to the |
| Central | ➞ |
|
Central downloads supported kernel modules from |
| Central | ➞ | Image registries | Central queries the image registries to get image metadata. For example, to show Dockerfile instructions in the RHACS portal. |
| Scanner | ➞ | Image registries | Scanner pulls images from the image registry to identify vulnerabilities. |
| Sensor | ⮂ | Central | There is bidirectional communication between Central and Sensor. Sensor polls Central periodically for downloading updates for the sensor bundle configuration. It also sends events for the observed activity for the secured cluster and observed policy violations. Central communicates with Sensor to force reprocessing of all deployments against enabled policies. |
| Sensor | ⮂ | Scanner |
Sensor can communicate with the lightweight Scanner installed in the secured cluster. This connection allows Sensor to access registries directly from the secured cluster in scenarios where Central might be unable to access them. Scanner requests updated data from Sensor, Sensor forwards these requests to Central, and Central downloads the requested data from |
| Collector | ⮂ | Sensor | Collector communicates with Sensor and sends all of the events to the respective Sensor for the cluster. On supported OpenShift Container Platform clusters, Collector analyzes the software packages installed on the nodes and sends them to Sensor so that Scanner can later scan them for vulnerabilities. Collector also requests missing drivers from Sensor. Sensor requests compliance scan results from Collector. Additionally, Sensor receives external Classless Inter-Domain Routing information from Central and pushes it to Collector. |
| Admission controller | ⮂ | Sensor | Sensors send the list of security policies to enforce to Admission controller. Admission controller sends security policy violation alerts to Sensor. Admission controller can also request image scans from Sensor when required. |
| Admission controller | ➞ | Central | It is not common; however, Admission controller can communicate with Central directly if the Central endpoint is known and Sensor is unavailable. |
1.6. RHACS connection protocols and default ports
Components of RHACS use various default ports and connection protocols. Depending on your system and firewall configuration, you might need to configure your firewall to allow traffic on certain ports.
The following table provides default ports and protocols for some connections within RHACS and between RHACS and external components. This is helpful for configuring your firewall to allow inbound and outbound cluster traffic.
However, you might need more detailed information in some scenarios. For example, if your firewall is integrated in the cluster router, you might need to specify ports for connections that happen within one cluster but might be on different IP networks. In this scenario, you can use the RHACS network policy YAML files in your OpenShift Container Platform and Kubernetes clusters to determine connections and ports that you might need to configure.
| Component or external entity | Connection type | Port | Additional information |
|---|---|---|---|
| Central and Scanner V4 Indexer | gRPC | 8443 | |
| Central and Sensor on secured cluster |
| 443 | Sensor and Central primarily communicate over a bidirectional gRPC stream, initiated by Sensor to Central’s port 443. |
| Central and user (CLI) |
| 443 |
For more information about the |
| Central and vulnerability feeds | HTTPS | 443 |
Connects to |
| Collector to Sensor | gRPC | 443 | This is a bidirectional gRPC connection initiated by Collector to Sensor’s port 443. |
| Collector (Compliance) to Sensor | gRPC | 8444 | If node scanning is enabled on OpenShift Container Platform release 4, this connection is initiated by Sensor to compliance running in the Collector pod. |
| Scanner to Scanner-DB | TCP | 5432 | |
| Scanner V4 Indexer to Central | HTTPS | 443 | |
| Scanner V4 Indexer and Matcher to Scanner V4 DB | TCP | 5432 | |
| Sensor and Admission Controller | gRPC | 443 | This is a bidirectional gRPC stream, initiated by Admission Controller to Sensor’s port 443. This occurs in delegated scanning scenarios or in OpenShift Container Platform secured clusters. |
