Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 3. Configuring Java applications

To enable Cryostat to gather, store, and analyze Java Flight Recorder (JFR) data about target applications that run on Java Virtual Machines (JVMs), you must configure the applications so that Cryostat can detect and connect to them.

You can configure the applications in any of the following ways:

Cryostat agent

You can use the Cryostat agent component for detection and connectivity, which is implemented as a Java Instrumentation Agent and acts as a plug-in for applications that run on the JVM.

The Cryostat agent provides an HTTP API that the Cryostat server can use as an alternative to an application’s JMX port. By attaching a properly configured Cryostat agent to the workload applications that you deploy, you can use the full Cryostat feature set without any need for the target applications to expose a JMX port.

Note

Before Red Hat build of Cryostat 2.4, the Cryostat agent provided a read-only HTTP API that supported a limited set of JFR operations only.

The Cryostat agent’s HTTP API can offer the following benefits compared to a JMX port:

  • Greater security due to the reduced API surface area
  • Deployment flexibility due to the Cryostat agent’s dual role as a Cryostat discovery plug-in

For more information, see Working with the Cryostat agent.

Remote Java Management Extensions (JMX) connections

You can configure your target applications to allow Java Management Extensions (JMX) connections. This type of configuration uses an OpenShift Service for detection and JMX for connectivity.

JMX is a standard feature on a JVM with which you can monitor and manage target applications that run on the JVM. For Cryostat to use JMX, you must enable and configure JMX when you start the JVM, because Cryostat requires the target applications to expose a JMX port.

Cryostat communicates with the target applications over this JMX port to start and stop JFR recordings and to pull JFR data over the network, enabling Cryostat to store and analyze this JFR data. Remote monitoring requires security to ensure that unauthorized persons cannot access application. Cryostat prompts you to enter your credentials before Cryostat can access any of the application’s JFR recordings.

For more information, see Configuring applications by using JMX connections.

Cryostat agent and JMX hybrid

You can configure your target applications to use a hybrid approach where you use both the Cryostat agent and JMX. With this approach, you use the Cryostat agent to detect the target applications and JMX to expose the JFR data to Cryostat, which allows for more flexibility.

For example, you can use the agent to detect the applications without needing to depend on specific port numbers and also use the JMX connections to start and stop JFR flight recordings on demand.

If the Cryostat agent detects that JMX is also configured on your application, the agent publishes itself to the Cryostat server with both agent HTTP API definitions and JMX URL definitions. In this situation, you can use whichever configuration option you prefer.

For more information, see Configuring applications by using the Cryostat agent and JMX connections.

3.1. Working with the Cryostat agent

The Cryostat agent is implemented as a Java Instrumentation Agent that acts as a plug-in for applications that run on the JVM. The Cryostat agent provides an HTTP API that offers greater deployment flexibility than a JMX port due to the agent’s dual role as a discovery plug-in. You can configure your target applications to use the agent’s HTTP API for both detection and connectivity with Cryostat.

3.1.1. Cryostat agent features

Red Hat build of Cryostat offers different distributions of the Cryostat agent and supports different ways of deploying and using the agent. Consider the following information to help you determine the type of Cryostat agent deployment and use cases that best suit your needs.

3.1.1.1. Agent JAR file distributions

Red Hat build of Cryostat 3.0 distributes two different variations of the Cryostat agent’s JAR file:

  • An all-in-one "shaded" JAR file that is self-contained and includes the agent code and all of its dependencies

    This "shaded" JAR file provides the most convenient form of Cryostat agent to include in your existing applications, because you need to include only one additional agent JAR file. This is a common distribution pattern for similar agents and tools. Before Cryostat 2.4, the “shaded” JAR file was the only available distribution of the Cryostat agent. The “shaded” JAR file name is in the form cryostat-agent-0.4.X.redhat-xxxxx-shaded.jar, where 0.4.X and xxxxx represent the agent version and build number, respectively.

  • A standard JAR file that contains the agent code without any dependencies

    This type of JAR file is useful if you know that dependency conflicts exist between the agent and your workload applications. If you intend to apply your own strategies to provide the correct versions of each dependency to satisfy both the agent and your applications’ requirements, you can use the standalone JAR file. The standard JAR file name is in the form cryostat-agent-0.4.X.redhat-xxxxx.jar, where where 0.4.X and xxxxx represent the agent version and build number, respectively.

Important

If you want to use the Cryostat agent, the all-in-one “shaded” JAR file is the preferred way to deploy the agent. Do not use the standard JAR file unless you intend to apply your own strategies to resolve any dependency conflicts between the agent and your workload applications.

The agent JAR files are available for download from the Red Hat Maven Repository.

3.1.1.2. Static or dynamic attachment to the JVM

Depending on your requirements, the Cryostat agent can use a static or dynamic approach for attaching to the JVM.

Static attachment to the JVM

You can enable your workload application’s JVM to load and initialize the Cryostat agent at JVM startup. This static attachment approach requires that you configure the application to pass the -javaagent JVM flag with the path to the Cryostat agent‘s JAR file (for example, ‑javaagent:/deployment/app/lib/cryostat-agent-shaded.jar). Once the Cryostat agent’s basic initialization is complete, your workload application’s normal startup process begins as usual.

Depending on your workload application, the static attachment approach might require that you set one or more environment variables or add an argument to the argLine parameter. However, in some cases, you might need to reconfigure, rebuild, and redeploy your application. Static attachment also requires that you restart the application JVM, which can cause application downtime.

Note

Before Cryostat 3.0, static attachment through the -javaagent JVM flag was the only way to enable the Cryostat agent to attach and run on a target application.

For more information, see Configuring applications by using the Cryostat agent.

Dynamic attachment to the JVM

From Cryostat 3.0 onward, the Cryostat agent can attach dynamically to an application JVM that is already running without requiring an application restart. This dynamic attachment feature has the following requirements:

  • You must ensure that the agent’s JAR file is copied to the JVM’s file system (for example, by using the oc cp command).
  • You must be able to run the agent as a separate process on the same host or within the same application (for example, by using the oc exec command).

The dynamic attachment feature supports ad hoc one-time profiling or troubleshooting workflows where you might not need the agent to be attached every time the JVM starts. Dynamic attachment also suits situations where you cannot or do not want to reconfigure your application for the sole purpose of attaching the agent. Because the agent can attach to a running JVM without requiring an application restart, this also means there is no application downtime.

For more information, see Launching the Cryostat agent as a standalone process for dynamic attachment to the JVM.

3.1.1.3. Options for including the agent’s JAR file into your workload application

Depending on your requirements, you can include the Cryostat agent’s JAR file into the workload application in different ways:

  • For dynamic one-time attachment to a running JVM, you can copy the agent’s JAR file to the JVM’s file system by using the oc cp command.

  • For static attachment to a JVM at startup, you can use any of the following options:

    • The simplest option is to add the JAR file to your application’s dependencies in the pom.xml or build.gradle file. Your build tool (Maven or Gradle) downloads the JAR file for inclusion in your application build output.
    • You can use a Maven plug-in, such as the maven-dependency-plugin, to provide more fine-grained control over the download and inclusion of the JAR file in the application build output.
    • You can create a PersistentVolume storage volume that contains the JAR file. Then reconfigure your application’s Deployment/DeploymentConfig to mount the PersistentVolume and use -javaagent:/path/to/persistentvolume/cryostat-agent-shaded.jar. The exact way to accomplish this task depends on the type of PersistentVolume provider you enabled in your OpenShift cluster.

Once the Cryostat agent is successfully added to your application container and loaded, your application’s stdout and console logs start displaying log messages from the Cryostat agent.

3.1.1.4. Agent configuration properties

You can specify configuration properties for the Cryostat agent in either of two ways:

  • Use JVM system property flags on the application (for example, -Dcryostat.agent.api.writes-enabled=true).
  • Use environment variables by making all letters upper case and replacing any punctuation with underscores (for example, CRYOSTAT_AGENT_API_WRITES_ENABLED=true).

Required agent properties

You must configure the following properties to enable the Cryostat agent to operate successfully:

cryostat.agent.baseuri

This specifies the URL location of the Cryostat server back end that the Cryostat agent advertises itself to (that is, the internal OpenShift Service object) such as https://my-cryostat.my-namespace.svc.cluster.local.

cryostat.agent.callback

This specifies the URL location of the Cryostat agent instance or application itself. Cryostat uses this URL to perform health checks and request data from the agent. You can use the OpenShift/Kubernetes Downward API to determine this dynamically by setting this property to a value such as https://$(POD_IP):9977. For more information, see the Kubernetes Downward API documentation on status.podIP.

cryostat.agent.authorization

This specifies a header value that the Cryostat agent can include in API requests to the Cryostat server (for example, Bearer abcd1234). This is set to an empty string by default.

Note

This property takes precedence over cryostat.agent.authorization.type and cryostat.agent.authorization.value. If you specify a value for this property, the agent uses this value in the header of API requests. Otherwise, the agent uses the values that you specify for cryostat.agent.authorization.type and cryostat.agent.authorization.value.

cryostat.agent.authorization.type

This specifies an authorization type that is used in conjunction with the cryostat.agent.authorization.value property. The authorization type performs a mapping of the cryostat.agent.authorization.value to produce a header that the Cryostat agent can include in API requests to the Cryostat server.

Note

The agent ignores this property if you specify a value for the cryostat.agent.authorization property.

Valid authorization types are as follows:

  • basic

    This encodes the authorization value by using Base64 to produce a Basic base64(value) header.

  • bearer

    This directly embeds the authorization value into a Bearer value header.

  • kubernetes

    This reads the authorization value as a file location to produce a Bearer fileAsString(value) header.

  • none

    This produces no header.

  • auto

    This first attempts the same behavior as the kubernetes option. Otherwise, if unsuccessful, this reverts to none to produce no header.

This property is set to auto by default.

cryostat.agent.authorization.value

This specifies an authorization value that is used in conjunction with the cryostat.agent.authorization.type property. The authorization value is mapped by the authorization type to produce a header that the Cryostat agent can include in API requests to the Cryostat server.

Note

The agent ignores this property if you specify a value for the cryostat.agent.authorization property.

Consider the following guidelines:

  • If the authorization type is basic, specify an authorization value that matches the unencoded basic credentials (for example, user:pass).
  • If the authorization type is bearer, specify an authorization value that matches the token that needs to be presented.
  • If the authorization type is kubernetes, specify an authorization value that matches the file-system path to the service account token secret file
  • If the authorization type is none, any value that you specify is ignored.

By default, this property is set to /var/run/secrets/kubernetes.io/serviceaccount/token, which is the standard location for Kubernetes service account token secret files.

Optional agent properties

Depending on your setup requirements, you can also configure the following agent properties:

cryostat.agent.api.writes-enabled

This indicates whether the Cryostat agent allows write operations. This is set to false by default. If you want the Cryostat agent to accept requests to start, stop, or delete JFR flight recordings, you must set this property to true.

Note

Even if this property is set to false, the agent can still fulfill requests to list flight recordings or download individual recording files.

cryostat.agent.webserver.port

This specifies the HTTP port number that the agent uses to bind its HTTP API (by default, 9977). If this conflicts with an existing port that your application or another tooling agent uses, you must specify a different port number.

cryostat.agent.app.name

This specifies a label for identifying which application this Cryostat agent instance is attached to (by default, cryostat-agent). You could use the Downward API metadata.name or metadata.labels[‘app’] fields for this. For more information, see the Kubernetes Downward API documentation Kubernetes Downward API documentation.

The preceding list describes only a subset of the available configuration properties for the Cryostat agent. For a full list of agent properties, see Agent Properties.

3.1.2. Configuring applications by using the Cryostat agent

You can use the Cryostat agent, implemented as a Java Instrumentation Agent, to configure target applications, so that Cryostat can detect the applications, collect data, and send the data to Cryostat for analysis. You can also optionally enable the Cryostat agent to accept requests from the Cryostat server to start, stop, and delete JFR recordings.

Note

This procedure is relevant if you want to use the static attachment feature, which allows your workload application’s JVM to load and initialize the Cryostat agent at JVM startup. If you want the Cryostat agent to attach to a running JVM on a one-time basis, see Launching the Cryostat agent as a standalone process for dynamic attachment to the JVM instead.

Important

As described in Agent JAR file distributions, Red Hat provides two different variations of the Cryostat agent either as an all-in-one “shaded” JAR file or as a standard JAR file. The all-in-one “shaded” JAR file is the preferred way to deploy the Cryostat agent. Do not use the standard JAR file unless you intend to apply your own strategies to resolve any dependency conflicts between the agent and your workload applications.

The following procedure describes how to install the "shaded" JAR file distribution of the Cryostat 3.0 agent. The latest “shaded” JAR file is available for download from the Red Hat Maven repository. The “shaded” JAR file name is in the form cryostat-agent-0.4.X.redhat-xxxxx-shaded.jar, where 0.4.X and xxxxx represent the agent version and build number, respectively. For information about the latest version and build number of the Cryostat agent, refer to the Red Hat Maven repository.

As described in Options for including the agent’s JAR file into your workload application, the Cryostat 3.0 agent supports different options for including the agent’s JAR file into your workload applications. The following procedure describes how to add the "shaded" JAR file to your application’s dependencies in the pom.xml or build.gradle file.

Prerequisites

  • Logged in to your Cryostat web console.
  • Installed JDK version 11 or later.

Procedure

  1. Install the Cryostat agent. Choose one of the following options, depending on your application build:

    • Using Maven:

      Update the application pom.xml file with the Cryostat agent JAR file information.

      Example pom.xml

      <project>
  ...
  <repositories>
    <repository>
      <id>redhat-maven-repository</id>
      <url>https://maven.repository.redhat.com/earlyaccess/all/</url>
    </repository>
  </repositories>
  ...
  <build>
    <plugins>
      <plugin>
        <artifactId>maven-dependency-plugin</artifactId>
        <version>3.3.0</version>
        <executions>
          <execution>
            <phase>prepare-package</phase>
            <goals>
              <goal>copy</goal>
            </goals>
            <configuration>
              <artifactItems>
                <artifactItem>
                  <groupId>io.cryostat</groupId>
                  <artifactId>cryostat-agent</artifactId>
                  <version>0.4.X.redhat-xxxxx</version>
                  <classifier>shaded</classifier>
                </artifactItem>
              </artifactItems>
              <stripVersion>true</stripVersion>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
    ...
  </build>
  ...
</project>
      Copy to Clipboard

      Note

      In the preceding example, replace 0.4.X.redhat-xxxxx with the latest version and build number of the Cryostat agent. For information about the latest version and build number of the Cryostat agent, refer to the Red Hat Maven repository.

      The next time you build your application, the Cryostat agent JAR file is available at target/dependency/cryostat-agent-shaded.jar.

    • Using Gradle:

      Update the build.gradle file.

      Example build.gradle file

      repositories {
	…
maven {
    	url "https://maven.repository.redhat.com/earlyaccess/all/"
    	credentials {
      		username "myusername"
        	password "mytoken"
    	}
	}
}
      Copy to Clipboard

      How you package the agent JAR file into the application depends on the Gradle plug-ins that you use for the build. For example, if you are using the Jib plug-in, update the build.gradle file as follows:

      Example build.gradle file

      plugins {
	id 'java'
	id 'application'
	id 'com.google.cloud.tools.jib' version '3.3.1'
	id 'com.ryandens.javaagent-jib' version '0.5.0'
}
…
dependencies {
	…
	javaagent 'io.cryostat:cryostat-agent:0.4.X.redhat-xxxxx:shaded'
      Copy to Clipboard

      Note

      In the preceding example, replace 0.4.X.redhat-xxxxx with the latest version and build number of the Cryostat agent. For information about the latest version and build number of the Cryostat agent, refer to the Red Hat Maven repository.

  2. Update the Docker file. The following example uses the JAVA_OPTS environment variable to pass the relevant JVM information.

    Example

    ...
COPY target/dependency/cryostat-agent-shaded.jar /deployments/app/
...
ENV JAVA_OPTS="-javaagent:/deployments/app/cryostat-agent-shaded.jar"
    Copy to Clipboard

  3. Rebuild the container image that is specific to your application. 

    docker build -t docker.io/myorg/myapp:latest -f src/main/docker/Dockerfile
    Copy to Clipboard

  4. To supply the JVM system properties or environment variables that you need to configure the Cryostat agent, push the updated image and then modify your application deployment.

    Example

    apiVersion: apps/v1
kind: Deployment
...
spec:
  ...
  template:
    ...
    spec:
      containers:
        - name: sample-app
          image: docker.io/myorg/myapp:latest
          env:
            - name: CRYOSTAT_AGENT_APP_NAME
              value: "myapp"
              # Replace this with the Kubernetes DNS record
              # for the Cryostat Service
            - name: CRYOSTAT_AGENT_BASEURI
              value: "http://cryostat.mynamespace.mycluster.svc:4180"
            - name: POD_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.podIP
            - name: CRYOSTAT_AGENT_CALLBACK
              value: "http://$(POD_IP):9977"
    1 
    
              # Replace "abcd1234" with a plain-text authentication token
            - name: CRYOSTAT_AGENT_AUTHORIZATION
    2 
    
              value: "Bearer abcd1234"
            - name: CRYOSTAT_AGENT_API_WRITES_ENABLED
    3 
    
              value: true
          ports:
            - containerPort: 9977
              protocol: TCP
          resources: {}
      restartPolicy: Always
status: {}
    Copy to Clipboard

    • <1>: Port number 9977 is the default HTTP port that the agent exposes for the internal web server that services Cryostat requests. You can change this port number if it conflicts with your target application into which the agent is installed.
    • <2>: The CRYOSTAT_AGENT_AUTHORIZATION value shows the credentials that the agent includes in the API requests to Cryostat to advertise its own presence or to push JFR data. You can also create a Kubernetes Service Account for this purpose and replace abcd1234 with the plain-text authentication token associated with the service account.
    • <3>: The CRYOSTAT_AGENT_API_WRITES_ENABLED variable is set to false by default. If you want the Cryostat agent to accept requests from the Cryostat server to start, stop, or delete JFR flight recordings, you must set this variable to true.

3.1.2.1. Configuring the Cryostat agent to trust the Cryostat server

When you use the Cryostat agent with a Cryostat instance that has cert-manager integration enabled, the Cryostat agent communicates with Cryostat over a secure HTTPS connection. In this situation, the Cryostat Operator uses cert-manager to generate a self-signed certificate authority (CA) certificate, which is stored within a secret. You must configure the Cryostat agent to trust this CA certificate.

Procedure

  1. Create a Cryostat CR that defines the namespaces for your Cryostat instance and your target applications.

    For example, if you want to create a Cryostat CR in the cryostat namespace that is configured to connect to target applications in the apps namespace, enter the following details: 

    apiVersion: operator.cryostat.io/v1beta2
kind: Cryostat
metadata:
  name: cryostat-sample
  namespace: cryostat
spec:
  enableCertManager: true
  targetNamespaces:
  - apps
    Copy to Clipboard

  2. After Cryostat becomes available, to obtain the CA certificate secret in your target application’s namespace, enter the following commands: 

    $ oc project apps
$ oc get secret "cryostat-ca-$(echo -n 'cryostat/cryostat-sample' | sha256sum | cut -d ' ' -f 1)"
    Copy to Clipboard

    In the preceding example, replace apps with the namespace of your target application, and replace cryostat/cryostat-sample with the namespace and name of your Cryostat instance. Also, ensure that the namespace and name of your Cryostat instance are separated by a forward slash (/).

    The preceding command produces output similar to the following:

    Obtaining a CA certificate secret

    As shown in the preceding example, the secret name contains a hash suffix to prevent conflicts with other Cryostat instances on your cluster.

  3. Configure the Cryostat agent to trust Cryostat’s CA certificate by creating an init container to import the certificate into a truststore.

    Note

    This step assumes that you have a target application named my-app in the apps namespace that has the Cryostat agent installed and is otherwise properly configured.

    In the following examples, replace any occurrences of my-app and apps with the name and namespace of your target application, as appropriate.

    1. Create a volume for the Cryostat CA secret in your deployment by using the secret name that you obtained in Step 2.

      For example: 

      $ oc set volumes deploy/my-app --add --name=cryostat-ca --secret-name=cryostat-ca-30268177e44252b3f9b7d9bf3a6db48f3a1cd3656700a6830952afc4456c0048
      Copy to Clipboard

      The preceding command produces output similar to the following: 

      deployment.apps/my-app volume updated
      Copy to Clipboard

    2. Create an emptyDir volume to share the truststore between the init container and your application container:

      For example: 

      $ oc set volumes deploy/my-app --add --name=truststore -m /var/run/secrets/io.cryostat/truststore
      Copy to Clipboard

      The preceding command produces output similar to the following: 

      deployment.apps/my-app volume updated
      Copy to Clipboard

    3. Add an init container to the deployment YAML file for your target application.

      For example: 

      initContainers:
  - name: pem-to-truststore
    image: registry.access.redhat.com/ubi8/openjdk-11-runtime:latest
    command:
      - /bin/bash
    args:
      - -c
      - >-
        keytool -import -file
        /var/run/secrets/io.cryostat/cryostat-ca/tls.crt
        -keystore
        /var/run/secrets/io.cryostat/truststore/truststore.jks
        -trustcacerts -noprompt
        -storepass <my password>
    volumeMounts:
      - mountPath: /var/run/secrets/io.cryostat/cryostat-ca
        name: cryostat-ca
      - mountPath: /var/run/secrets/io.cryostat/truststore
        name: truststore
      Copy to Clipboard

      In the preceding example, replace <my password> with the password that you want to use.

    4. Define the javax.ssl.trustStore Java system property in your application.

      For example: 

      env:
  - name: JAVA_OPTS_APPEND
    value: |-
      # …
      -Djavax.net.ssl.trustStore=/var/run/secrets/io.cryostat/truststore/truststore.jks
      -Djavax.net.ssl.trustStorePassword=<my password>
      Copy to Clipboard

      In the preceding example, replace <my password> with the password that you specified in the previous step.

      Note

      How you set Java system properties depends on how your application is built and which base image you used. The preceding example assumes that you are using an application that was built with the OpenJDK UBI images.

3.1.3. Launching the Cryostat agent as a standalone process for dynamic attachment to the JVM

If you want the Cryostat agent to attach dynamically to an application JVM that is already running, you can launch the agent as a standalone Java process.

Note

This procedure is only relevant if you want to use the dynamic attachment feature, which allows the Cryostat agent to attach to a running JVM on an ad hoc one-time basis. If you want your workload application’s JVM to load and initialize the Cryostat agent at JVM startup, see Configuring applications by using the Cryostat agent.

Prerequisites

  • Copied the agent’s JAR file to the JVM’s file system by using the oc cp command.

    Note

    The latest “shaded” JAR file is available for download from the Red Hat Maven repository. The “shaded” JAR file name is in the form cryostat-agent-0.4.X.redhat-xxxxx-shaded.jar, where 0.4.X and xxxxx represent the agent version and build number, respectively.

Procedure

  • Enter the following command: 

    $ java -jar target/<agent_jar_file> <pid>
    Copy to Clipboard

    In the preceding command, replace <agent_jar_file> with the agent’s JAR file name and replace <pid> with the process ID (PID) of the JVM that you want the agent to attach to.

    For example: 

    $ java -jar target/cryostat-agent-0.4.1.redhat-00001-shaded.jar 1234
    Copy to Clipboard
    Note

    The preceding example is shown for illustration only and might not reflect the latest available version and build of the “shaded” JAR file.

When you run the preceding command, the agent process uses its Attach providers to look up the specified PID. If the specified PID is found, the agent process attaches to this PID and attempts to load the agent’s JAR file into this JVM, which then bootstraps into the normal agent launch process.

Agent launch behavior based on the PID value

Consider the following guidelines for agent launch behavior depending on the PID value:

  • If you specify an invalid PID, the agent cannot launch successfully.
  • If you specify a wildcard asterisk (*) as the PID value, the agent’s JAR file attempts to bootstrap into every JVM that it finds.
  • If you specify 0 as the PID value or if you do not specify any PID value, the agent checks if exactly one candidate JVM is available. If only one JVM is available, the agent attempts to bootstrap into this JVM. If multiple JVMs or no JVMs are available, the agent cannot launch successfully.
Late-binding configuration options

When launching the Cryostat agent as a standalone process, you can also specify additional late-binding configuration options to the agent launcher by using command-line options.

For example: 

$ java -jar target/cryostat-agent-0.4.1.redhat-00001-shaded.jar \
-Dcryostat.agent.baseuri=<cryostat_server_url> \
-Dcryostat.agent.callback=<cryostat_agent_url> \
-Dcryostat.agent.authorization=Bearer <authorization_value> \
--smartTrigger=[ProcessCpuLoad>0.2]~profile \
@/deployment/app/moreAgentArgs \
1234
Copy to Clipboard

For information about the available options and their behavior, run the java -jar target/<filename>.jar -h help command, where <filename> represents the name of the “shaded” agent JAR file that you copied.

As shown in the preceding example, when launching the Cryostat agent as a standalone process, ensure that you specify the -D flags for any late-binding configuration options after the -jar <path_to_jarfile> argument. The position of the -D flags is important here:

  • If you specify the -D flag for a system property before the -jar <path_to_jarfile> argument, this property is set on the standalone launcher JVM process. This could lead to unexpected issues when launching the agent as a standalone process.
  • If you specify the -D flags for system properties after the -jar <path_to_jarfile> argument, these properties are passed as arguments to the agent launcher. When the launcher bootstraps the agent into the host JVM, the agent applies these properties onto the JVM at agent startup.

As shown in the preceding example, ensure that you specify any required agent properties such as cryostat.agent.baseuri, cryostat.agent.callback, and cryostat.agent.authorization as configuration options to the agent launcher. For more information about configuration properties, see Agent configuration properties.

3.2. Configuring applications by using JMX connections

For Cryostat to detect and communicate with your target Java applications, you can configure the applications to allow remote Java Management Extensions (JMX) connections.

Prerequisites

  • Logged in to your Cryostat web console.
  • Created a Cryostat instance in your project.

Procedure

  1. To enable remote JMX connections, complete the following steps:

    1. On your application, define the following Java system property: 

      -Dcom.sun.management.jmxremote.port=<port_num>
      Copy to Clipboard
      Note

      To add the -Dcom.sun.management.jmxremote.port=<port_num> property without having to rebuild the target application, you can set the JAVA_OPTS_APPEND environment variable on the application. JAVA_OPTS_APPEND is an environment variable that is used by Red Hat Universal Base Images (UBI) only.

      If you use Red Hat UBI to build application images, set the JAVA_OPTS_APPEND variable at build time in the application Docker file, or at runtime by running the following command: 

      oc set env deployment <name> JAVA_OPTS_APPEND="..."
      Copy to Clipboard

      If you do not use Red Hat UBI to build application images, refer to the documentation for your base image for information about how to add the Java system properties at build time or runtime.

    2. Specify that the application listens for remote JMX connections by allowing traffic to the application. Use an Red Hat OpenShift Service and specify the following values for its remote JMX port:

      Example service.yaml

      apiVersion: v1
kind: Service
...
spec:
  ports:
    - name: "jfr-jmx"
      port: 9091
      targetPort: 9091
...
      Copy to Clipboard

  2. Secure the remote JMX connections:

    1. Enable and configure authentication and SSL/TLS for the remote JMX connections in your application: 

      -Dcom.sun.management.jmxremote.port=<port_num>

 # enable JMX authentication
-Dcom.sun.management.jmxremote.authenticate=true

# define users for JMX auth
-Dcom.sun.management.jmxremote.password.file=</path/to/jmxremote.password>

# set permissions for JMX users
-Dcom.sun.management.jmxremote.access.file=</path/to/jmxremote.access>

 # enable JMX SSL
 -Dcom.sun.management.jmxremote.ssl=true

# enable JMX registry SSL
-Dcom.sun.management.jmxremote.registry.ssl=true

# set your SSL keystore
-Djavax.net.ssl.keyStore=</path/to/keystore>

# set your SSL keystore password
-Djavax.net.ssl.keyStorePassword=<password>
      Copy to Clipboard

    2. Configure Cryostat to trust the application TLS certificate. Create a secret for the application in the same namespace as your Cryostat application and configure Cryostat to refer to the secret. To create a secret for the certificate, run the following command: 

      oc create secret generic myapp-cert --from-file=tls.crt=/path/to/cert.pem
      Copy to Clipboard
      Note

      The certificate must be in a .pem file format.

    3. When you create your Cryostat instance, add the secret to the list of trusted TLS certificates. For more information, see Configuring TLS certificates.

    4. To allow your applications to verify that Cryostat is connecting to them by a means other than password authentication, enable TLS client authentication: 

      -Dcom.sun.management.jmxremote.ssl.need.client.auth=true
-Djavax.net.ssl.trustStore=</path/to/truststore>
-Djavax.net.ssl.trustStorePassword=<password>
      Copy to Clipboard
      Note

      TLS client authentication requires the cert-manager operator for Red Hat OpenShift.

    5. If you use TLS client authentication for remote JMX connections, the application truststore must contain a Cryostat certificate. The Cryostat operator cert-manager integration creates a self-signed certificate for the Cryostat deployment. This certificate is located in the <cryostat>-tls secret, where <cryostat> is the name of the Cryostat instance you created.

      Note

      The cert-manager Operator also places a Java keystore truststore in the secret.

      To mount this truststore in your application deployment, run the following command, replacing "<myapp>" with the name of your application deployment and "<cryostat>" with the name of your Cryostat instance: 

      oc set volumes deploy <myapp> --add --name=truststore \
  --secret-name=<cryostat>-tls --sub-path=truststore.p12 \
  --mount-path=/var/run/secrets/<myapp>/truststore.p12
      Copy to Clipboard

    6. The Cryostat operator generates the truststore password, which you can find in the <cryostat>-keystore secret. To mount this as an environment variable in your application deployment, run the following command: 

      oc set env deploy <myapp> --from='secret/<cryostat>-keystore'
      Copy to Clipboard

    7. Configure the Java arguments for the container. Run the following commands: 

      -Dcom.sun.management.jmxremote.ssl.need.client.auth=true
-Djavax.net.ssl.trustStore=/var/run/secrets/<myapp>/truststore.p12
-Djavax.net.ssl.trustStorePassword="$(KEYSTORE_PASS)"
      Copy to Clipboard
      Warning

      If you deployed Cryostat and your applications in a testing environment, you might want to configure the target applications without any JMX or TLS authentication. You can do so by using the following set of Java system properties, however, this configuration is not secure and not recommended. 

      -Dcom.sun.management.jmxremote.port=<port_num>
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
      Copy to Clipboard

3.3. Configuring applications by using the Cryostat agent and JMX connections

You can configure target applications that run on Java Virtual Machines (JVM) to use a combination of the Cryostat agent and Java Management Extensions (JMX) connections to detect and communicate with the target applications.

You use the Cryostat agent to detect and communicate with the target application and use JMX to expose the Java Flight Recorder (JFR) data.

You must configure the Cryostat agent to communicate with Cryostat about itself and that the agent is reachable through JMX rather than through HTTP.

Prerequisites

  • Logged in to your Cryostat web console.
  • Created a Cryostat instance in your project.

Procedure

  1. Install the Cryostat agent. For application builds that use Maven, update the application pom.xml file with the Cryostat agent JAR file information.

    Example pom.xml file

    <project>
  ...
  <repositories>
    <repository>
      <id>redhat-maven-repository</id>
      <url>https://maven.repository.redhat.com/earlyaccess/all/</url>
    </repository>
  </repositories>
  ...
  <build>
    <plugins>
      <plugin>
        <artifactId>maven-dependency-plugin</artifactId>
        <version>3.3.0</version>
        <executions>
          <execution>
            <phase>prepare-package</phase>
            <goals>
              <goal>copy</goal>
            </goals>
            <configuration>
              <artifactItems>
                <artifactItem>
                  <groupId>io.cryostat</groupId>
                  <artifactId>cryostat-agent</artifactId>
                  <version>0.4.X.redhat-xxxxx</version>
                  <classifier>shaded</classifier>
                </artifactItem>
              </artifactItems>
              <stripVersion>true</stripVersion>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
    ...
  </build>
  ...
</project>
    Copy to Clipboard

    Note

    In the preceding example, replace 0.4.X.redhat-xxxxx with the latest version and build number of the Cryostat agent. For information about the latest version and build number of the Cryostat agent, refer to the Red Hat Maven repository.

  2. Modify your application deployment:

    Example

    apiVersion: apps/v1
kind: Deployment
...
spec:
  ...
  template:
    ...
    spec:
      containers:
        - name: sample-app
          image: docker.io/myorg/myapp:latest
          env:
            - name: CRYOSTAT_AGENT_APP_NAME
              value: "myapp"
              # Replace this with the Kubernetes DNS record
              # for the Cryostat Service
            - name: CRYOSTAT_AGENT_BASEURI
              value: "http://cryostat.mynamespace.mycluster.svc:4180"
            - name: POD_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.podIP
            - name: CRYOSTAT_AGENT_CALLBACK
              value: "http://$(POD_IP):9977"
            - name: CRYOSTAT_AGENT_AUTHORIZATION
              # Replace "abcd1234" with a plain-text authentication token
              value: "Bearer abcd1234"
              # This environment variable is key to the Cryostat agent
              # and JMX "hybrid" setup.
              # Set the Cryostat agent to register itself with Cryostat
              # as reachable through JMX, rather than reachable through HTTP.
            - name: CRYOSTAT_AGENT_REGISTRATION_PREFER_JMX
              value: "true"
              # Configure the application to load the agent JAR file and
              # to enable JMX, so that the Cryostat agent can register
              # itself as reachable through JMX.
              # To configure authentication and SSL/TLS for the JMX
              # connections, see <1>.
            - name: JAVA_OPTS
              value: >-
                -javaagent:/deployments/app/cryostat-agent-shaded.jar
                -Dcom.sun.management.jmxremote.port=9091
    1 
    
          ports:
            - containerPort: 9977
              protocol: TCP
          resources: {}
      restartPolicy: Always
status: {}
    Copy to Clipboard

    <1>: To configure authentication and SSL/TLS for the JMX connections and view further configuration options, see Configuring applications by using JMX connections.

  3. To enable Cryostat to detect the target application and connect to the Cryostat agent, configure an application Service:

    Example

    apiVersion: v1
kind: Service
...
spec:
  ports:
    - name: "jfr-jmx"
      port: 9091
      targetPort: 9091
    - name: "cryostat-agent"
      port: 9977
      targetPort: 9977
...
    Copy to Clipboard

Back to top
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2025 Red Hat