Chapter 9. Integrating Spring Boot with Kubernetes


The Spring Cloud Kubernetes plugin currently enables you to integrate the following features of Spring Boot and Kubernetes:

9.1. Spring Boot externalized configuration

In Spring Boot, externalized configuration is the mechanism that enables you to inject configuration values from external sources into Java code. In your Java code, injection is typically enabled by annotating with the @Value annotation (to inject into a single field) or the @ConfigurationProperties annotation (to inject into multiple properties on a Java bean class).

The configuration data can come from a wide variety of different sources (or property sources). In particular, configuration properties are often set in a project’s application.properties file (or application.yaml file, if you prefer).

9.1.1. Kubernetes ConfigMap

A Kubernetes ConfigMap is a mechanism that can provide configuration data to a deployed application. A ConfigMap object is typically defined in a YAML file, which is then uploaded to the Kubernetes cluster, making the configuration data available to deployed applications.

9.1.2. Kubernetes Secrets

A Kubernetes Secrets is a mechanism for providing sensitive data (such as passwords, certificates, and so on) to deployed applications.

9.1.3. Spring Cloud Kubernetes plugin

The Spring Cloud Kubernetes plug-in implements the integration between Kubernetes and Spring Boot. In principle, you could access the configuration data from a ConfigMap using the Kubernetes API. It is much more convenient, however, to integrate Kubernetes ConfigMap directly with the Spring Boot externalized configuration mechanism, so that Kubernetes ConfigMaps behave as an alternative property source for Spring Boot configuration. This is essentially what the Spring Cloud Kubernetes plug-in provides.

9.1.4. Enabling Spring Boot with Kubernetes integration

You can enable Kubernetes integration by adding it as a Maven dependency to pom.xml file.

Procedure

  1. Enable the Kubernetes integration by adding the following Maven dependency to the pom.xml file of your Spring Boot Maven project.

    <project ...>
      ...
      <dependencies>
        ...
        <dependency>
          <groupId>io.fabric8</groupId>
          <artifactId>spring-cloud-kubernetes-core</artifactId>
        </dependency>
        ...
      </dependencies>
      ...
    </project>
  2. To complete the integration,

    • Add some annotations to your Java source code
    • Create a Kubernetes ConfigMap object
    • Modify the OpenShift service account permissions to allow your application to read the ConfigMap object.

Additional resources

9.2. Running tutorial for ConfigMap Property Source

The following tutorial allows you to experiment with setting Kubernetes Secrets and ConfigMaps. Enable the Spring Cloud Kubernetes plug-in as explained in the Enabling Spring Boot with Kubernetes Integration to integrate Kubernetes configuration objects with Spring Boot Externalized Configuration.

9.2.1. Running Spring Boot Camel Config quickstart

The following tutorial is based on the spring-boot-camel-config-archetype Maven archetype, which enables you to set up Kubernetes Secrets and ConfigMaps.

Procedure

  1. Open a new shell prompt and enter the following Maven command to create a simple Camel Spring Boot project.

    mvn org.apache.maven.plugins:maven-archetype-plugin:2.4:generate \
      -DarchetypeCatalog=https://maven.repository.redhat.com/ga/io/fabric8/archetypes/archetypes-catalog/2.2.0.fuse-750020-redhat-00002/archetypes-catalog-2.2.0.fuse-750020-redhat-00002-archetype-catalog.xml \
      -DarchetypeGroupId=org.jboss.fuse.fis.archetypes \
      -DarchetypeArtifactId=spring-boot-camel-config-archetype \
      -DarchetypeVersion=2.2.0.fuse-750020-redhat-00002

    The archetype plug-in switches to interactive mode to prompt you for the remaining fields:

    Define value for property 'groupId': : org.example.fis
    Define value for property 'artifactId': : fuse75-configmap
    Define value for property 'version':  1.0-SNAPSHOT: :
    Define value for property 'package':  org.example.fis: :
    Confirm properties configuration:
    groupId: org.example.fis
    artifactId: fuse75-configmap
    version: 1.0-SNAPSHOT
    package: org.example.fis
    Y: : Y

    When prompted, enter org.example.fis for the groupId value and fuse75-configmap for the artifactId value. Accept the defaults for the remaining fields.

  2. Log in to OpenShift and switch to the OpenShift project where you will deploy your application. For example, to log in as the developer user and deploy to the openshift project, enter the following commands:

    oc login -u developer -p developer
    oc project openshift
  3. At the command line, change to the directory of the new fuse75-configmap project and create the Secret object for this application.

    cd fuse75-configmap
    oc create -f sample-secret.yml
    Note

    It is necessary to create the Secret object before you deploy the application, otherwise the deployed container enters a wait state until the Secret becomes available. If you subsequently create the Secret, the container will come out of the wait state. For more information on how to set up Secret Object, see Setting up Secret.

  4. Build and deploy the quickstart application. From the top level of the fuse75-configmap project, enter:

    mvn fabric8:deploy -Popenshift
  5. View the application log as follows.

    1. Navigate to the OpenShift web console in your browser (https://OPENSHIFT_IP_ADDR, replace OPENSHIFT_IP_ADDR with the IP address of the cluster) and log in to the console with your credentials (for example, with username developer and password, developer).
    2. In the left hand side panel, expand Home. Click Status to view the Project Status page. All the existing applications in the selected namespace (for example, openshift) are displayed.
    3. Click fuse75-configmap to view the Overview information page for the quickstart.
    4. In the left hand side panel, expand Workloads.
    5. Click Pods and then click fuse75-configmap-xxxx. The pod details for the application are displayed.
    6. Click on the Logs tab to view the application logs.
  6. The default recipient list, which is configured in src/main/resources/application.properties, sends the generated messages to two dummy endpoints: direct:async-queue and direct:file. This causes messages like the following to be written to the application log:

    5:44:57.376 [Camel (camel) thread #0 - timer://order] INFO  generate-order-route - Generating message message-44, sending to the recipient list
    15:44:57.378 [Camel (camel) thread #0 - timer://order] INFO  target-route-queue - ----> message-44 pushed to an async queue (simulation)
    15:44:57.379 [Camel (camel) thread #0 - timer://order] INFO  target-route-queue - ----> Using username 'myuser' for the async queue
    15:44:57.380 [Camel (camel) thread #0 - timer://order] INFO  target-route--file - ----> message-44 written to a file
  7. Before you can update the configuration of the fuse75-configmap application using a ConfigMap object, you must give the fuse75-configmap application permission to view data from the OpenShift ApiServer. Enter the following command to give the view permission to the fuse75-configmap application’s service account:

    oc policy add-role-to-user view system:serviceaccount:openshift:qs-camel-config
    Note

    A service account is specified using the syntax system:serviceaccount:PROJECT_NAME:SERVICE_ACCOUNT_NAME. The fis-config deployment descriptor defines the SERVICE_ACCOUNT_NAME to be qs-camel-config.

  8. To see the live reload feature in action, create a ConfigMap object as follows:

    oc create -f sample-configmap.yml

    The new ConfigMap overrides the recipient list of the Camel route in the running application, configuring it to send the generated messages to three dummy endpoints: direct:async-queue, direct:file, and direct:mail. For more information about ConfigMap object, see Setting up ConfigMap. This causes messages like the following to be written to the application log:

    16:25:24.121 [Camel (camel) thread #0 - timer://order] INFO  generate-order-route - Generating message message-9, sending to the recipient list
    16:25:24.124 [Camel (camel) thread #0 - timer://order] INFO  target-route-queue - ----> message-9 pushed to an async queue (simulation)
    16:25:24.125 [Camel (camel) thread #0 - timer://order] INFO  target-route-queue - ----> Using username 'myuser' for the async queue
    16:25:24.125 [Camel (camel) thread #0 - timer://order] INFO  target-route--file - ----> message-9 written to a file (simulation)
    16:25:24.126 [Camel (camel) thread #0 - timer://order] INFO  target-route--mail - ----> message-9 sent via mail

9.2.2. Configuration properties bean

A configuration properties bean is a regular Java bean that can receive configuration settings by injection. It provides the basic interface between your Java code and the external configuration mechanisms.

Externalized Configuration and Bean Registry

Following image shows how Spring Boot Externalized Configuration works in the spring-boot-camel-config quickstart.

kube spring boot 01

The configuration mechanism has the following main parts:

Property Sources
Provides property settings for injection into configuration. The default property source is the application.properties file for the application, and this can optionally be overridden by a ConfigMap object or a Secret object.
Configuration Properties bean
Receives configuraton updates from the property sources. A configuration properties bean is a Java bean decorated by the @Configuration and @ConfigurationProperties annotations.
Spring bean registry
With the requisite annotations, a configuration properties bean is registered in the Spring bean registry.
Integration with Camel bean registry
The Camel bean registry is automatically integrated with the Spring bean registry, so that registered Spring beans can be referenced in your Camel routes.

QuickstartConfiguration class

The configuration properties bean for the fuse75-configmap project is defined as the QuickstartConfiguration Java class (under the src/main/java/org/example/fis/ directory), as follows:

package org.example.fis;

import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Configuration;

@Configuration  1
@ConfigurationProperties(prefix = "quickstart")  2
public class QuickstartConfiguration {

    /**
     * A comma-separated list of routes to use as recipients for messages.
     */
    private String recipients;  3

    /**
     * The username to use when connecting to the async queue (simulation)
     */
    private String queueUsername;  4

    /**
     * The password to use when connecting to the async queue (simulation)
     */
    private String queuePassword;  5

    // Setters and Getters for Bean properties
    // NOT SHOWN
    ...
}
1
The @Configuration annotation causes the QuickstartConfiguration class to be instantiated and registered in Spring as the bean with ID, quickstartConfiguration. This automatically makes the bean accessible from Camel. For example, the target-route-queue route is able to access the queueUserName property using the Camel syntax ${bean:quickstartConfiguration?method=getQueueUsername}.
2
The @ConfigurationProperties annotation defines a prefix, quickstart, that must be used when defining property values in a property source. For example, a properties file would reference the recipients property as quickstart.recipients.
3
The recipient property is injectable from property sources.
4
The queueUsername property is injectable from property sources.
5
The queuePassword property is injectable from property sources.

9.2.3. Setting up Secret

The Kubernetes Secret in this quickstart is set up in the standard way, apart from one additional required step: the Spring Cloud Kubernetes plug-in must be configured with the mount paths of the Secrets, so that it can read the Secrets at run time. To set up the Secret:

  1. Create a Sample Secret Object
  2. Configure volume mount for the Secret
  3. Configure spring-cloud-kubernetes to read Secret properties

Sample Secret object

The quickstart project provides a sample Secret, sample-secret.yml, as follows. Property values in Secret objects are always base64 encoded (use the base64 command-line utility). When the Secret is mounted in a pod’s filesystem, the values are automatically decoded back into plain text.

sample-secret.yml file

apiVersion: v1
kind: Secret
metadata: 1
  name: camel-config
type: Opaque
data:
  # The username is 'myuser'
  quickstart.queue-username: bXl1c2VyCg== 2
  quickstart.queue-password: MWYyZDFlMmU2N2Rm 3

1
metadata.name: Identifies the Secret. Other parts of the OpenShift system use this identifier to reference the Secret.
2
quickstart.queue-username: Is meant to be injected into the queueUsername property of the quickstartConfiguration bean. The value must be base64 encoded.
3
quickstart.queue-password: Is meant to be injected into the queuePassword property of the quickstartConfiguration bean. The value must be base64 encoded.
Note

Kubernetes does not allow you to define property names in CamelCase (it requires property names to be all lowercase). To work around this limitation, use the hyphenated form queue-username, which Spring Boot matches with queueUsername. This takes advantage of Spring Boot’s relaxed binding rules for externalized configuration.

Configure volume mount for the Secret

The application must be configured to load the Secret at run time, by configuring the Secret as a volume mount. After the application starts, the Secret properties then become available at the specified location in the filesystem. The deployment.yml file for the application is located under src/main/fabric8/ directory, which defines the volume mount for the Secret.

deployment.yml file

spec:
  template:
    spec:
      serviceAccountName: "qs-camel-config"
      volumes: 1
        - name: "camel-config"
          secret:
            # The secret must be created before deploying this application
            secretName: "camel-config"
      containers:
        -
          volumeMounts: 2
            - name: "camel-config"
              readOnly: true
              # Mount the secret where spring-cloud-kubernetes is configured to read it
              # see src/main/resources/bootstrap.yml
              mountPath: "/etc/secrets/camel-config"
          resources:
#            requests:
#              cpu: "0.2"
#              memory: 256Mi
#            limits:
#              cpu: "1.0"
#              memory: 256Mi
             env:
              - name: SPRING_APPLICATION_JSON
               value: '{"server":{"undertow":{"io-threads":1, "worker-threads":2 }}}'

1
In the volumes section, the deployment declares a new volume named camel-config, which references the Secret named camel-config.
2
In the volumeMounts section, the deployment declares a new volume mount, which references the camel-config volume and specifies that the Secret volume should be mounted to the path /etc/secrets/camel-config in the pod’s filesystem.

Configuring spring-cloud-kubernetes to read Secret properties

To integrate secrets with Spring Boot externalized configuration, the Spring Cloud Kubernetes plug-in must be configured with the secret’s mount path. Spring Cloud Kubernetes reads the secrets from the specified location and makes them available to Spring Boot as property sources. The Spring Cloud Kubernetes plug-in is configured by settings in the bootstrap.yml file, located under src/main/resources in the quickstart project.

bootstrap.yml file

# Startup configuration of Spring-cloud-kubernetes
spring:
  application:
    name: camel-config
  cloud:
    kubernetes:
      reload:
        # Enable live reload on ConfigMap change (disabled for Secrets by default)
        enabled: true
      secrets:
        paths: /etc/secrets/camel-config

The spring.cloud.kubernetes.secrets.paths property specifies the list of paths of secrets volume mounts in the pod.

Note

A bootstrap.properties file (or bootstrap.yml file) behaves similarly to an application.properties file, but it is loaded at an earlier phase of application start-up. It is more reliable to set the properties relating to the Spring Cloud Kubernetes plug-in in the bootstrap.properties file.

9.2.4. Setting up ConfigMap

In addition to creating a ConfigMap object and setting the view permission appropriately, the integration with Spring Cloud Kubernetes requires you to match the ConfigMap’s metadata.name with the value of the spring.application.name property configured in the project’s bootstrap.yml file. To set up the ConfigMap:

  • Create Sample ConfigMap Object
  • Set up the view permission
  • Configure the Spring Cloud Kubernetes plug-in

Sample ConfigMap object

The quickstart project provides a sample ConfigMap, sample-configmap.yml.

kind: ConfigMap
apiVersion: v1
metadata: 1
  # Must match the 'spring.application.name' property of the application
  name: camel-config
data:
  application.properties: | 2
    # Override the configuration properties here
    quickstart.recipients=direct:async-queue,direct:file,direct:mail 3
1
metadata.name: Identifies the ConfigMap. Other parts of the OpenShift system use this identifier to reference the ConfigMap.
2
data.application.properties: This section lists property settings that can override settings from the original application.properties file that was deployed with the application.
3
quickstart.recipients: Is meant to be injected into the recipients property of the quickstartConfiguration bean.

Setting the view permission

As shown in the deployment.yml file for the Secret, the serviceAccountName is set to qs-camel-config in the project’s deployment.yml file. Hence, you need to enter the following command to enable the view permission on the quickstart application (assuming that it deploys into the test project namespace):

oc policy add-role-to-user view system:serviceaccount:test:qs-camel-config

Configuring the Spring Cloud Kubernetes plug-in

The Spring Cloud Kubernetes plug-in is configured by the following settings in the bootstrap.yml file.

spring.application.name
This value must match the metadata.name of the ConfigMap object (for example, as defined in sample-configmap.yml in the quickstart project). It defaults to application.
spring.cloud.kubernetes.reload.enabled
Setting this to true enables dynamic reloading of ConfigMap objects.

For more details about the supported properties, see PropertySource Reload Configuration Properties.

9.3. Using ConfigMap PropertySource

Kubernetes has the notion of ConfigMap for passing configuration to the application. The Spring cloud Kubernetes plug-in provides integration with ConfigMap to make config maps accessible by Spring Boot.

The ConfigMap PropertySource when enabled will look up Kubernetes for a ConfigMap named after the application (see spring.application.name). If the map is found it will read its data and do the following:

9.3.1. Applying individual properties

Let’s assume that we have a Spring Boot application named demo that uses properties to read its thread pool configuration.

  • pool.size.core
  • pool.size.max

This can be externalized to config map in YAML format:

kind: ConfigMap
apiVersion: v1
metadata:
  name: demo
data:
  pool.size.core: 1
  pool.size.max: 16

9.3.2. Applying application.yaml ConfigMap property

Individual properties work fine for most cases but sometimes we find YAML is more convenient. In this case we use a single property named application.yaml and embed our YAML inside it:

kind: ConfigMap
apiVersion: v1
metadata:
  name: demo
data:
  application.yaml: |-
    pool:
      size:
        core: 1
        max:16

9.3.3. Applying application.properties ConfigMap property

You can also define the ConfigMap properties in the style of a Spring Boot application.properties file. In this case we use a single property named application.properties and list the property settings inside it:

kind: ConfigMap
apiVersion: v1
metadata:
  name: demo
data:
  application.properties: |-
    pool.size.core: 1
    pool.size.max: 16

9.3.4. Deploying a ConfigMap

To deploy a ConfigMap and make it accessible to a Spring Boot application, perform the following steps.

Procedure

  1. In your Spring Boot application, use the externalized configuration mechanism to access the ConfigMap property source. For example, by annotating a Java bean with the @Configuration annotation, it becomes possible for the bean’s property values to be injected by a ConfigMap.
  2. In your project’s bootstrap.properties file (or bootstrap.yaml file), set the spring.application.name property to match the name of the ConfigMap.
  3. Enable the view permission on the service account that is associated with your application (by default, this would be the service account called default). For example, to add the view permission to the default service account:

    oc policy add-role-to-user view system:serviceaccount:$(oc project -q):default -n $(oc project -q)

9.4. Using Secrets PropertySource

Kubernetes has the notion of Secrets for storing sensitive data such as password, OAuth tokens, etc. The Spring cloud Kubernetes plug-in provides integration with Secrets to make secrets accessible by Spring Boot.

The Secrets property source when enabled will look up Kubernetes for Secrets from the following sources. If the secrets are found, their data is made available to the application.

  1. Reading recursively from secrets mounts
  2. Named after the application (see spring.application.name)
  3. Matching some labels

Please note that, by default, consuming Secrets via API (points 2 and 3 above) is not enabled.

9.4.1. Example of setting Secrets

Let’s assume that we have a Spring Boot application named demo that uses properties to read its ActiveMQ and PostreSQL configuration.

amq.username
amq.password
pg.username
pg.password

These secrets can be externalized to Secrets in YAML format:

ActiveMQ Secrets
apiVersion: v1
kind: Secret
metadata:
  name: activemq-secrets
  labels:
    broker: activemq
type: Opaque
data:
  amq.username: bXl1c2VyCg==
  amq.password: MWYyZDFlMmU2N2Rm
PostreSQL Secrets
apiVersion: v1
kind: Secret
metadata:
  name: postgres-secrets
  labels:
    db: postgres
type: Opaque
data:
  pg.username: dXNlcgo=
  pg.password: cGdhZG1pbgo=

9.4.2. Consuming the Secrets

You can select the Secrets to consume in a number of ways:

  • By listing the directories where the secrets are mapped:

    -Dspring.cloud.kubernetes.secrets.paths=/etc/secrets/activemq,etc/secrets/postgres

    If you have all the secrets mapped to a common root, you can set them like this:

    -Dspring.cloud.kubernetes.secrets.paths=/etc/secrets
  • By setting a named secret:

    -Dspring.cloud.kubernetes.secrets.name=postgres-secrets
  • By defining a list of labels:

    -Dspring.cloud.kubernetes.secrets.labels.broker=activemq
    -Dspring.cloud.kubernetes.secrets.labels.db=postgres

9.4.3. Configuration properties for Secrets PropertySource

You can use the following properties to configure the Secrets property source:

spring.cloud.kubernetes.secrets.enabled
Enable the Secrets property source. Type is Boolean and default is true.
spring.cloud.kubernetes.secrets.name
Sets the name of the secret to look up. Type is String and default is ${spring.application.name}.
spring.cloud.kubernetes.secrets.labels
Sets the labels used to lookup secrets. This property behaves as defined by Map-based binding. Type is java.util.Map and default is null.
spring.cloud.kubernetes.secrets.paths
Sets the paths where secrets are mounted. This property behaves as defined by Collection-based binding. Type is java.util.List and default is null.
spring.cloud.kubernetes.secrets.enableApi
Enable/disable consuming secrets via APIs. Type is Boolean and default is false.
Note

Access to secrets via API may be restricted for security reasons — the preferred way is to mount a secret to the POD.

9.5. Using PropertySource Reload

Some applications may need to detect changes on external property sources and update their internal status to reflect the new configuration. The reload feature of Spring Cloud Kubernetes is able to trigger an application reload when a related ConfigMap or Secret change.

9.5.1. Enabling PropertySource Reload

The PropertySource reload feature of Spring Cloud Kubernetes is disabled by default.

Procedure

  1. Navigate to src/main/resources directory of the quickstart project and open the bootstrap.yml file.
  2. Change the configuration property spring.cloud.kubernetes.reload.enabled=true.

9.5.2. Levels of PropertySource Reload

The following levels of reload are supported for property spring.cloud.kubernetes.reload.strategy:

refresh

(default) only configuration beans annotated with @ConfigurationProperties or @RefreshScope are reloaded. This reload level leverages the refresh feature of Spring Cloud Context.

Note

The PropertySource reload feature can only be used for simple properties (that is, not collections) when the reload strategy is set to refresh. Properties backed by collections must not be changed at runtime.

restart_context
the whole Spring ApplicationContext is gracefully restarted. Beans are recreated with the new configuration.
shutdown
the Spring ApplicationContext is shut down to activate a restart of the container. When using this level, make sure that the lifecycle of all non-daemon threads is bound to the ApplicationContext and that a replication controller or replica set is configured to restart the pod.

9.5.3. Example of PropertySource Reload

The following example explains what happens when the reload feature is enabled.

Procedure

  1. Assume that the reload feature is enabled with default settings (refresh mode). The following bean will be refreshed when the config map changes:

    @Configuration
    @ConfigurationProperties(prefix = "bean")
    public class MyConfig {
    
        private String message = "a message that can be changed live";
    
        // getter and setters
    
    }
  2. To see the changes that are happening, create another bean that prints the message periodically as shown below.

    @Component
    public class MyBean {
    
        @Autowired
        private MyConfig config;
    
        @Scheduled(fixedDelay = 5000)
        public void hello() {
            System.out.println("The message is: " + config.getMessage());
        }
    }
  3. You can change the message printed by the application by using a ConfigMap as shown below.

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: reload-example
    data:
      application.properties: |-
        bean.message=Hello World!

    Any change to the property named bean.message in the Config Map associated with the pod will be reflected in the output of the program.

9.5.4. PropertySource Reload operating modes

The reload feature supports two operating modes:

event
(default) watches for changes in ConfigMaps or secrets using the Kubernetes API (web socket). Any event will produce a re-check on the configuration and a reload in case of changes. The view role on the service account is required in order to listen for config map changes. A higher level role (eg. edit) is required for secrets (secrets are not monitored by default).
polling
re-creates the configuration periodically from config maps and secrets to see if it has changed. The polling period can be configured using the property spring.cloud.kubernetes.reload.period and defaults to 15 seconds. It requires the same role as the monitored property source. This means, for example, that using polling on file mounted secret sources does not require particular privileges.

9.5.5. PropertySource Reload configuration properties

The following properties can be used to configure the reloading feature:

spring.cloud.kubernetes.reload.enabled
Enables monitoring of property sources and configuration reload. Type is Boolean and default is false.
spring.cloud.kubernetes.reload.monitoring-config-maps
Allow monitoring changes in config maps. Type is Boolean and default is true.
spring.cloud.kubernetes.reload.monitoring-secrets
Allow monitoring changes in secrets. Type is Boolean and default is false.
spring.cloud.kubernetes.reload.strategy
The strategy to use when firing a reload (refresh, restart_context, shutdown). Type is Enum and default is refresh.
spring.cloud.kubernetes.reload.mode
Specifies how to listen for changes in property sources (event, polling). Type is Enum and default is event.
spring.cloud.kubernetes.reload.period
The period in milliseconds for verifying changes when using the polling strategy. Type is Long and default is 15000.

Note the following points:

  • The spring.cloud.kubernetes.reload.* properties should not be used in ConfigMaps or Secrets. Changing such properties at run time may lead to unexpected results;
  • Deleting a property or the whole config map does not restore the original state of the beans when using the refresh level.
Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.