Configuring advanced Cryostat configurations

You can use the Pluggable Discovery API endpoint, /api/v2.2/discovery, to register an external plug-in with Cryostat and then provide information about discoverable application targets to Cryostat. A plug-in can either unregister itself after successfully registering with Cryostat or continually push updates to Cryostat about a target application.

The Pluggable Discovery API provides a more flexible way to integrate Cryostat into your deployment schema that the Red Hat OpenShift service account mechanism.

Consider an example where you need to write a plug-in program that creates a static map of application IP addresses to port numbers. The plug-in can use the Pluggable Discovery API to transport this information to Cryostat, so that Cryostat can better connect with the target application.

The Pluggable Discovery API depends on the following handlers to manage requests sent from the /api/v2.2/discovery endpoint and Cryostat:

Before you can use the API’s endpoint to interact with Cryostat, you must ensure that the client, which is the plug-in’s source, meets the following prerequisites:

If you need to register your own discovery plug-in program with Cryostat, you can disable the Cryostat built-in discovery mechanism. Disabling the built-in discovery mechanism helps to reduce duplicate definitions that might open in the Cryostat web console if both the plug-in and Cryostat can access the same target application information.

You can set the following environment variable in your Red Hat OpenShift command-line console (CLI) to disable the Cryostat built-in discovery mechanism:

CRYOSTAT_DISABLE_BUILTIN_DISCOVERY=true

CRYOSTAT_DISABLE_BUILTIN_DISCOVERY=true

You can also consider preserving the setting for the Cryostat built-in discovery implementation and then complete one of the following actions:

The Pluggable Discovery API uses DisoveryRegistrationHandler to register a discovered plug-in with Cryostat. The handler manages GET and POST requests from the plug-in and Cryostat. If you do not register a plug-in with Cryostat, the plug-in cannot provide target information to Cryostat.

When your discovery plug-in program sends a POST request to Cryostat for registration purposes, Cryostat reads the callback URL and sends a GET request to the plug-in. If the plug-in correctly responds to the GET request then Cryostat accepts the registration request by responding to the initial POST request. This process ensures that the plug-in is active and available to Cryostat.

A failed request might indicate that the plug-in failed or went offline. In this case, the endpoint removes the plug-in’s information from the database on Red Hat OpenShift.

You can also specify id and token elements in your GET or POST request. These elements are optional, but you can consider using them in situations where you want to reuse registration information for a plug-in that was previously registered with Cryostat.

Cryostat creates a token for the registered plug-in, and this token contains expiry and authorization information. If a POST request contains valid id and token information, Cryostat can reuse the plug-in registration information and refreshes the token. If a request contains only an id element or a token element, you must re-register the plug-in with Cryostat.

After Cryostat sends a POST request to the plug-in’s callback component, the plug-in might send a POST request to Cryostat to refresh the plug-in’s registration details. The plug-in must include its id and token information in its request. Cryostat can then use DiscoveryRegistrationHandler to refresh the plug-in’s details. Cryostat sends a response that includes the updated token to the plug-in, and the plug-in can use this token for future requests to Cryostat.

{
    "realm": "my-plugin",
    "callback": "http://my-plugin.my-namespace.svc.local:1234/callback"
}

{
    "realm": "my-plugin",
    "callback": "http://my-plugin.my-namespace.svc.local:1234/callback"
}

{
"data": {
    "result": {
        "id": "922dd4f4-9d7c-4ae2-8982-0903868226a6",
        "token": "<key_value>"
    }
},
"meta": {
    "status": "Created",
    "type": "application/json"
}

{
"data": {
    "result": {
        "id": "922dd4f4-9d7c-4ae2-8982-0903868226a6",
        "token": "<key_value>"
    }
},
"meta": {
    "status": "Created",
    "type": "application/json"
}

The Pluggable Discovery API uses the DisoveryRegistrationCheckHandler handler to enable plug-ins to periodically check their own registration status on a Cryostat server instance.

The DisoveryRegistrationCheckHandler handler manages GET requests from the plug-in to Cryostat. By using the handler, external plug-ins can periodically verify the Cryostat server instance to which they are registered is still active and recognizes the prior registration of the plug-in.

Similar to how the Cryostat callback URL endpoint checks plug-in instances, where Cryostat reads the callback URL and sends a GET request to the plug-in, the DisoveryRegistrationCheckHandler handler works in the same way, but sends the request in the opposite direction. That is, the plug-in sends a GET request to the Cryostat server to check its registration status on the Cryostat server. If the request fails, for example, if an Unexpected 401 or Unexpected 404 error response is received, the plug-in can discard its existing registration information and attempt to register again.

http -v https://my-cryostat.my-namespace.cluster.local:8181/api/v2.2/discovery/<plugin-registration-id>?token=<current-plugin-registration-token>

$ http -v https://my-cryostat.my-namespace.cluster.local:8181/api/v2.2/discovery/<plugin-registration-id>?token=<current-plugin-registration-token>

HTTP/1.1 200 OK
content-encoding: gzip
content-length: 86
content-type: application/json

{
	"data": {
    	"result": null
	},
	"meta": {
    	"mimeType": "JSON",
    	"status": "OK"
	}
}

HTTP/1.1 200 OK
content-encoding: gzip
content-length: 86
content-type: application/json

{
	"data": {
    	"result": null
	},
	"meta": {
    	"mimeType": "JSON",
    	"status": "OK"
	}
}

HTTP/1.1 404 Not Found
content-encoding: gzip
content-length: 95
content-type: application/json

{
	"data": {
    	"reason": null
	},
	"meta": {
    	"status": "Not Found",
    	"type": "text/plain"
	}
}

HTTP/1.1 404 Not Found
content-encoding: gzip
content-length: 95
content-type: application/json

{
	"data": {
    	"reason": null
	},
	"meta": {
    	"status": "Not Found",
    	"type": "text/plain"
	}
}

The DiscoveryGetHandler collates deployment schemas and opens these schemas in a hierarchical tree view, so that Cryostat can interact with any registered discoverable plug-ins to integrate with a specific deployment schema.

When you create a deployment with at least one pod, and a service maps to the deployment or pod on Red Hat OpenShift, Red Hat OpenShift creates an endpoint object, /api/v2.2/discovery, for all combinations of the Pod IP address and the Service port. The DiscoveryGetHandler receives information from an endpoint GET request and then collates deployment schema information in JSON format.

The following example demonstrates how the handler opens results in a hierarchical tree view in JSON format. In the example, the root of the tree is the UNIVERSE node. This node contains child Realm node types, which stem from Cryostat’s built-in discovery mechanism and the plug-ins discovered by the Pluggable Discovery API.

{
    "data": {
        "result": {
            "children": [
                {
                    "children": [],
                    "labels": {},
                    "name": "Custom Targets",
                    "nodeType": "Realm"
                },
                {
                    "children": [
                        {
                            "labels": {},
                            "name": "service:jmx:rmi:///jndi/rmi://cryostat:9091/jmxrmi",
                            "nodeType": "JVM",
                            "target": {
                                "alias": "io.cryostat.Cryostat",
                                "annotations": {
                                    "cryostat": {
                                        "HOST": "cryostat",
                                        "JAVA_MAIN": "io.cryostat.Cryostat",
                                        "PORT": "9091",
                                        "REALM": "JDP"
                                    },
                                    "platform": {}
                                },
                                "connectUrl": "service:jmx:rmi:///jndi/rmi://cryostat:9091/jmxrmi",
                                "labels": {}
                            }
                        }
                    ],
                    "labels": {},
                    "name": "JDP",
                    "nodeType": "Realm"
                }
            ],
            "labels": {},
            "name": "Universe",
            "nodeType": "Universe"
        }
    },
    "meta": {
        "status": "OK",
        "type": "application/json"
    }
}

{
    "data": {
        "result": {
            "children": [
                {
                    "children": [],
                    "labels": {},
                    "name": "Custom Targets",
                    "nodeType": "Realm"
                },
                {
                    "children": [
                        {
                            "labels": {},
                            "name": "service:jmx:rmi:///jndi/rmi://cryostat:9091/jmxrmi",
                            "nodeType": "JVM",
                            "target": {
                                "alias": "io.cryostat.Cryostat",
                                "annotations": {
                                    "cryostat": {
                                        "HOST": "cryostat",
                                        "JAVA_MAIN": "io.cryostat.Cryostat",
                                        "PORT": "9091",
                                        "REALM": "JDP"
                                    },
                                    "platform": {}
                                },
                                "connectUrl": "service:jmx:rmi:///jndi/rmi://cryostat:9091/jmxrmi",
                                "labels": {}
                            }
                        }
                    ],
                    "labels": {},
                    "name": "JDP",
                    "nodeType": "Realm"
                }
            ],
            "labels": {},
            "name": "Universe",
            "nodeType": "Universe"
        }
    },
    "meta": {
        "status": "OK",
        "type": "application/json"
    }
}

A plug-in registered with Cryostat sends a POST /api/v2.2/discovery/:id request to DiscoveryPostHandler. The id parameter of the request refers to the ID of the registered plug-in. This handler processes any subtrees associated with the plug-in and then maps a deployment schema.

The Cryostat POST request defines a subtree REALM node, so that the plug-in handler can publish its child node types in the REALM node during the request process. The plug-in takes responsibility for providing the correct information for theCryostat REALM node and for placing node types in the correct subtree positions. The plug-in must provide a token with the POST request that it sends to the DiscoveryPostHandler, so that the plug-in can bypass the authorization header check that the plug-in previously passed.

[
    {
        "labels": {},
        "nodeType": "JVM",
        "name": "service:jmx:rmi:///jndi/rmi://myapp.svc.local:9091/jmxrmi",
        "nodeType": "JVM",
        "target": {
            "alias": "com.MyApp",
            "annotations": {
                "cryostat": {},
                "platform": {}
        },
            "connectUrl": "service:jmx:rmi:///jndi/rmi://myapp.svc.local:9091/jmxrmi",
            "labels": {}
        }
    }
]

[
    {
        "labels": {},
        "nodeType": "JVM",
        "name": "service:jmx:rmi:///jndi/rmi://myapp.svc.local:9091/jmxrmi",
        "nodeType": "JVM",
        "target": {
            "alias": "com.MyApp",
            "annotations": {
                "cryostat": {},
                "platform": {}
        },
            "connectUrl": "service:jmx:rmi:///jndi/rmi://myapp.svc.local:9091/jmxrmi",
            "labels": {}
        }
    }
]

{
    "data": {
        "result": null
    },
    "meta": {
        "mimeType": "JSON",
        "status": "OK"
    }
}

{
    "data": {
        "result": null
    },
    "meta": {
        "mimeType": "JSON",
        "status": "OK"
    }
}

If a plug-in registered with Cryostat as a plug-in, and then the plug-in needs to shutdown, the plug-in typically issues a request to un-register itself as a plug-in from Cryostat. Cryostat sends a DELETE /api/v2.2/discovery/:id?token=:token request to DiscoveryDeregistrationHandler that then unregisters the plug-in from Cryostat.

The handler removes both the plug-in’s REALM subtree from the discovery schema and the plug-in’s registration information from Cryostat.

{
     "data": {
        "result": "bcc0f3a6-dc48-402e-a3d6-9fbb63beff78"
    },
    "meta": {
        "mimeType": "JSON",
        "status": "OK"
    }
}

{
     "data": {
        "result": "bcc0f3a6-dc48-402e-a3d6-9fbb63beff78"
    },
    "meta": {
        "mimeType": "JSON",
        "status": "OK"
    }
}

The Pluggable Discovery API returns messages when any of its handlers either completes a task or experiences issues when registering a plug-in with Cryostat.

A handler can return any of the following message types when the handler attempts to register a plug-in with Cryostat based on GET and POST requests:

You can use an HTTP client, such as HTTPie, to interact with the GraphQL API for generating custom queries.

When using the API to create a query, you must specify specific values for data types and fields, so that a function can use these values to accurately locate the data you require.

Consider a use case where you could automate a workflow that performs multiple operations in a single query. As an example, an HTTPie client request could send a query to Cryostat so that Cryostat could perform the following tasks:

The following examples demonstrate using the HTTP client to interact with the GraphQL API for generating a simple query and a complex query on Cryostat data.

https :8181/api/v1/targets

$ https :8181/api/v1/targets
HTTP/1.1 200 OK
content-encoding: gzip
content-length: 223
content-type: application/json

 {
	targetNodes {
    		name
    		nodeType
    		labels
    		target {
        		alias
        		serviceUri
    		}
	}
}

The previous example sets specific values for the targetNodes element, such as name. After you run the query, the query returns a list of any target JVMs that match the criteria specified.

https -v :8181/api/v2.2/graphql query=@graphql/target-nodes-query.graphql

$ https -v :8181/api/v2.2/graphql query=@graphql/target-nodes-query.graphql
POST /api/v2.2/graphql HTTP/1.1
Accept: application/json, /;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 171
Content-Type: application/json
Host: localhost:8181
User-Agent: HTTPie/2.6.0

 {
environmentNodes(filter: { name: "<application_pod_name>" }) {
    		descendantTargets {
        		doStartRecording(recording: {
            		name: "myrecording",
            	template: "Profiling",
            	templateType: "TARGET",
     		       	duration: 30
          	 	}) {
          			name
          			state
        		}
    		}
	}
}

The previous example sets the following specific values for the environmentNodes element, and the example does not return JVM target applications:

After you run the query, the query returns information about the JFR recording that you started, such as a list of active application nodes.

For Red Hat OpenShift, this query would return Deployment and DeploymentConfig API objects, and pods that interact with Cryostat.

The complex query demonstrates how the GraphQL API can perform a single API request that returns any JVM objects that interact with Cryostat and Red Hat OpenShift. The API request then starts a JFR recording on those returned objects.

Cryostat in combination with the JMC Agent can provide you with more information when you need to diagnose issues with your running JVM application.

The JMC Agent JAR file must be in the same Red Hat OpenShift container as the target JVM application. Otherwise, Cryostat cannot use the JMC Agent functionality on the application.

From the Cryostat web console, you can upload probe templates and then insert these templates into the JVM application. You can remove these template probes at a later stage, if required. A probe template describes a set of objects that Cryostat can process, so that Cryostat can complete a sequence of JMC Agent operations on the JVM application.

When you start a target JVM application with the JMC Agent, Cryostat automatically detects if the application is running with the JMC Agent.

Revised on 2023-12-12 18:07:31 UTC