Chapter 9. Diagnose and resolve serverless workflow issues

Procedure

1. If the workflow operation fails, examine the container log of the specific workflow instance to determine the cause by running the following command:

   $ oc logs my-workflow-xy73lj

   Copy to Clipboard Toggle word wrap

2. If the workflow fails to reach an HTTPS endpoint, check the pod log for an SSL certificate verification failure. This occurs if the target endpoint uses a Certificate Authority (CA) that the workflow cannot verify. The resulting error resembles the following:

   sun.security.provider.certpath.SunCertPathBuilderException - unable to find valid certification path to requested target

   Copy to Clipboard Toggle word wrap
3. To resolve the SSL certificate error, load the additional CA certificate into the running workflow container.

Procedure

1. Identify required namespaces.

   • Retrieve the namespace value where RHDH is running using oc get backstage -A.

   • Identify the SonataFlow Services Namespace by checking for either a sonataflowclusterplatform or sonataflowplatform instance.

     Note

     By default, the SonataFlow namespace must be the same as the RHDH namespace.

2. If the workflow is deployed to a namespace outside the core SonataFlow services, configure network policies to permit the necessary inter-namespace traffic.

   # Example NetworkPolicy configuration to ingress traffic into the workflow namespace
   apiVersion: networking.k8s.io/v1
   kind: NetworkPolicy
   metadata:
     name: {{ .Release.Name }}-allow-infra-ns-to-workflow-ns
     # Sonataflow and Workflows are using the RHDH target namespace.
     namespace: {{ .Release.Namespace | quote }}
   spec:
     podSelector: {}
     ingress:
       - from:
         - namespaceSelector:
             matchLabels:
               # Allow knative events to be delivered to workflows.
               kubernetes.io/metadata.name: knative-eventing
         - namespaceSelector:
             matchLabels:
               # Allow auxiliary knative function for workflow (such as m2k-save-transformation)
               kubernetes.io/metadata.name: knative-serving
         - namespaceSelector:
             matchLabels:
               # Allow communication between the serverless logic operator and the workflow namespace.
               kubernetes.io/metadata.name: openshift-serverless-logic

   Copy to Clipboard Toggle word wrap

3. Add SonataFlowClusterPlatform Custom Resource as shown in the following configuration:

   oc create -f - <<EOF
   apiVersion: sonataflow.org/v1alpha08
   kind: SonataFlowClusterPlatform
   metadata:
     name: cluster-platform
   spec:
     platformRef:
       name: sonataflow-platform
       namespace: $RHDH_NAMESPACE

   Copy to Clipboard Toggle word wrap

4. To allow communication between RHDH namespace and the workflow namespace, create the following network policies:

   1. Allow RHDH services to accept traffic from workflows. Create an additional network policy within the RHDH instance namespace as shown in the following configuration::

      oc create -f - <<EOF
      apiVersion: networking.k8s.io/v1
      kind: NetworkPolicy
      metadata:
        name: allow-external-workflows-to-rhdh
        # Namespace where network policies are deployed
        namespace: $RHDH_NAMESPACE
      spec:
        podSelector: {}
        ingress:
          - from:
            - namespaceSelector:
                matchLabels:
                  # Allow SonataFlow services to communicate with new/additional workflow namespace.
                  kubernetes.io/metadata.name: $ADDITIONAL_WORKFLOW_NAMESPACE

      Copy to Clipboard Toggle word wrap

   2. Allow traffic from RHDH, SonataFlow and Knative. Create a network policy within the additional workflow namespace as shown in the following configuration:

      oc create -f - <<EOF
      apiVersion: networking.k8s.io/v1
      kind: NetworkPolicy
      metadata:
        name: allow-rhdh-and-knative-to-workflows
        namespace: $ADDITIONAL_WORKFLOW_NAMESPACE
      spec:
        podSelector: {}
        ingress:
          - from:
            - namespaceSelector:
                matchLabels:
                  # Allows traffic from pods in the {product-very-short} namespace.
                  kubernetes.io/metadata.name: $RHDH_NAMESPACE
            - namespaceSelector:
                matchLabels:
                  # Allows traffic from pods in the Knative Eventing namespace.
                  kubernetes.io/metadata.name: knative-eventing
            - namespaceSelector:
                matchLabels:
                  # Allows traffic from pods in the Knative Serving namespace.
                  kubernetes.io/metadata.name: knative-serving

      Copy to Clipboard Toggle word wrap
5. (Optional) Create an allow-intra-namespace policy in the workflow namespace to enable unrestricted communication among all pods within that namespace.

6. If workflow persistence is required, perform the following configuration steps:

   1. Create a dedicated PostgreSQL Secret containing database credentials within the workflow namespace as shown in the following configuration:

      oc get secret sonataflow-psql-postgresql -n <your_namespace> -o yaml > secret.yaml
      sed -i '/namespace: <your_namespace>/d' secret.yaml
      oc apply -f secret.yaml -n $ADDITIONAL_NAMESPACE

      Copy to Clipboard Toggle word wrap

   2. Configure the workflow serviceRef property to correctly reference the PostgreSQL service namespace as shown in the following configuration:

      apiVersion: sonataflow.org/v1alpha08
      kind: SonataFlow
        ...
      spec:
        ...
        persistence:
          postgresql:
            secretRef:
              name: sonataflow-psql-postgresql
              passwordKey: postgres-password
              userKey: postgres-username
            serviceRef:
              databaseName: sonataflow
              databaseSchema: greeting
              name: sonataflow-psql-postgresql
              namespace: $POSTGRESQL_NAMESPACE
              port: 5432

      Copy to Clipboard Toggle word wrap

      namespace

      Enter the namespace where the PostgreSQL server is deployed.

7. If the sonataflow-platform-data-index-service cannot connect to the PostgreSQL database on startup, perform the following diagnostic checks:

   1. Verify that the PostgreSQL Pod has fully transitioned to a running and operational status. Allow additional time for database initialization before expecting related service pods (DataIndex, JobService) to establish a connection.
   2. If the PostgreSQL Server operates in a dedicated namespace (for example, outside RHDH), verify that network policies are configured to allow ingress traffic from the SonataFlow services namespace. Network policies might prevent the Data Index and Job Service pods from connecting to the database.

Procedure

1. Verify if the workflow uses GitOps profile. The RHDH Orchestrator UI displays only the workflows that use this profile. Make sure the workflow definition and the SonataFlow manifests use the GitOps profile.

2. Verify that the workflow pod has started and is ready. The readiness of a workflow pod depends on its successful registration with the Data Index. When a workflow initializes, it performs the following actions:

   1. It attempts to create its schema in the database (if persistence is active).

   2. It attempts to register itself to the Data Index. The workflow pod remains in an unready state until it successfully registers to the Data Index.

      Check the workflow deployment for additional status and error messages that might be unavailable in the pod log.

3. Check if the workflow pod can reach the Data Index service. Connect to the workflows pod and send the following GraphQL request to the Data Index:

   curl -g -k  -X POST  -H "Content-Type: application/json" \
                       -d '{"query":"query{ ProcessDefinitions  { id, serviceUrl, endpoint } }"}' \
                       http://sonataflow-platform-data-index-service.<your_namespace>/graphql

   Copy to Clipboard Toggle word wrap

   Use the Data Index service and namespace as defined in your environment. By default, this is the same namespace where RHDH is installed. If your SonataFlow resources are installed in a separate namespace, use <your_namespace>. Check if the RHDH pod can reach the workflow service by running the following command:

   curl http://<workflow_service>.<workflow_namespace>/management/processes

   Copy to Clipboard Toggle word wrap

4. Connect to the RHDH pod. Verify its connection to the Data Index service and inspect the RHDH pod logs for messages from the Orchestrator plugin.

   To inspect the logs, identify the RHDH pod and run the following oc logs command:

   oc get pods -n <your_namespace>
   oc logs <rhdh_pod_name> -n <your_namespace>

   Copy to Clipboard Toggle word wrap

   You must find messages indicating it is attempting to fetch workflow information from the Data Index, similar to the following:

   {"level":"\u001b[32minfo\u001b[39m","message":"fetchWorkflowInfos() called: http://sonataflow-platform-data-index-service.<your_namespace>","plugin":"orchestrator","service":"backstage","span_id":"fca4ab29f0a7aef9","timestamp":"2025-08-04 17:58:26","trace_flags":"01","trace_id":"5408d4b06373ff8fb34769083ef771dd"}

   Copy to Clipboard Toggle word wrap

   Notice the "plugin":"orchestrator" that can help to filter the messages.

5. Make sure the Data Index properties are set in the -managed-props ConfigMap of the workflow as shown in the following configuration:

   kogito.data-index.health-enabled = true
   kogito.data-index.url = http://sonataflow-platform-data-index-service.<your_namespace>
   ...
   mp.messaging.outgoing.kogito-processdefinitions-events.url = http://sonataflow-platform-data-index-service.<your_namespace>/definitions
   mp.messaging.outgoing.kogito-processinstances-events.url = http://sonataflow-platform-data-index-service.<your_namespace>/processes

   Copy to Clipboard Toggle word wrap
   Note

   The -managed-props ConfigMap is located in the same namespace as the workflow and is generated by the Openshift Serverless Logic (OSL) Operator.

   These properties, along with similar settings for the Job Services, indicate that the (OSL) Operator successfully registered the Data Index service.

6. Confirm that the workflow is registered in the Data Index database. Connect to the database used by the Data Index and run the following command from the PSQL instance pod:

   PGPASSWORD=<psql password> psql -h localhost -p 5432 -U < user> -d sonataflow

   Copy to Clipboard Toggle word wrap

   Replace <psql password> and <user> with your database credentials.

   Run the following SQL commands to query the registered workflow definitions:

   sonataflow=# SET search_path TO "sonataflow-platform-data-index-service";
   sonataflow=# select id, name from definitions;

   Copy to Clipboard Toggle word wrap

   You must see your workflows listed in the query results.

7. Make sure you have enabled Data Index and Job Service in the SonataFlowPlatform custom resource (CR) as shown in the following configuration:

   services:
       dataIndex:
         enabled: true
       jobService:
         enabled: true

   Copy to Clipboard Toggle word wrap

   If you fail to enable the Data Index and the Job Services in the SonataFlowPlatform custom resource (CR), the Orchestrator plugin fails to fetch the available workflows.

   Note

   You can also manually edit the SonataFlowPlatform CR instance to trigger the re-creation of workflow-related manifests.

8. Set the RBAC permissions correctly. For more information, see RBAC documentation.