Testing guide Camel K

Firstly, we need to install and get familiar with jbang and camel CLIs. You can follow the official documentation about Camel JBang to install the CLIs to your local environment. After this, we can see how to test an Integration for Camel K with Camel JBang.

The first application we develop is a simple one, and it defines the process you must follow when testing any Integration that is eventually deployed in Kubernetes via Camel K. verify the target version of Camel in your Camel K installation. With this information we can ensure to test locally against the same version that we will be later deploying in a cluster.

$ kamel version -a -v | grep Runtime
Runtime Version: 3.8.1

$ kubectl get camelcatalog camel-catalog-3.8.1 -o yaml | grep camel\.version
      camel.version: 3.8.1

The commands above are useful to find out what is the Camel version used by the runtime in your cluster Camel K installation. Our target is Camel version 3.18.3.

The easiest way to initialize a Camel route is to run camel init command:

$ camel init HelloJBang.java

At this stage, we can edit the file with the logic we need for our integration, or, simply run it:

$ camel run HelloJBang.java
2022-11-23 12:11:05.407  INFO 52841 --- [           main] org.apache.camel.main.MainSupport        : Apache Camel (JBang) 3.18.1 is starting
2022-11-23 12:11:05.470  INFO 52841 --- [           main] org.apache.camel.main.MainSupport        : Using Java 11.0.17 with PID 52841. Started by squake in /home/squake/workspace/jbang/camel-blog
2022-11-23 12:11:07.537  INFO 52841 --- [           main] e.camel.impl.engine.AbstractCamelContext : Apache Camel 3.18.1 (CamelJBang) is starting
2022-11-23 12:11:07.675  INFO 52841 --- [           main] e.camel.impl.engine.AbstractCamelContext : Routes startup (started:1)
2022-11-23 12:11:07.676  INFO 52841 --- [           main] e.camel.impl.engine.AbstractCamelContext :     Started java (timer://java)
2022-11-23 12:11:07.676  INFO 52841 --- [           main] e.camel.impl.engine.AbstractCamelContext : Apache Camel 3.18.1 (CamelJBang) started in 397ms (build:118ms init:140ms start:139ms JVM-uptime:3s)
2022-11-23 12:11:08.705  INFO 52841 --- [ - timer://java] HelloJBang.java:14                       : Hello Camel from java
2022-11-23 12:11:09.676  INFO 52841 --- [ - timer://java] HelloJBang.java:14                       : Hello Camel from java
...

A local java process starts with a Camel application running. No need to create a Maven project, all the boilerplate is on Camel JBang! However, you may notice that the Camel version used is different from the one we want to target. This is because your Camel JBang is using a different version of Camel. No worry, we can re-run this application but specifying the Camel version we want to run:

$ jbang run -Dcamel.jbang.version=3.18.3 camel@apache/camel run HelloJBang.java
...
[1] 2022-11-23 11:13:02,825 INFO  [org.apa.cam.imp.eng.AbstractCamelContext] (main) Apache Camel 3.18.3 (camel-1) started in 70ms (build:0ms init:61ms start:9ms)
...

The next step is to run it in a Kubernetes cluster where Camel K is installed.

Let us use the Camel K plugin for Camel JBang here instead of kamel CLI. This way, you can use the same JBang tooling to both run the Camel K integration locally and on the K8s cluster with the operator. The JBang plugin documentation can be found here: Camel JBang Kubernetes.

You see that the Camel K operator takes care to do the necessary transformation and build the Integration and related resources according to the expected lifecycle. Once this is live, you can follow up with the operations you usually do on a deployed Integration.

The benefit of this process is that you need not worry about the remote cluster until you are satisfied with your Integration tuned locally.

Once your Integration ready, you must take care about the kind of tuning, related to cluster deployment. Having this, you need not worry on deployment details at an early stage of the development. Or you can even have a separation of roles in your company where the domain expert may develop the integration locally and the cluster expert may do the deployment at a later stage.

Let us see an example about how to develop an integration that will later need some fine tuning in the cluster.

import org.apache.camel.builder.RouteBuilder;

public class MyJBangRoute extends RouteBuilder {

    @Override
    public void configure() throws Exception {
        from("file:/tmp/input")
            .convertBodyTo(String.class)
            .log("Processing file ${headers.CamelFileName} with content: ${body}")
            /*
            .filter(simple("${body} !contains 'checked'"))
                .log("WARN not checked: ${body}")
                .to("file:/tmp/discarded")
            .end()
            .to("file:/tmp/output");
            */
            .choice()
                .when(simple("${body} !contains 'checked'"))
                    .log("WARN not checked!")
                    .to("file:/tmp/discarded")
                .otherwise()
                    .to("file:/tmp/output")
            .end();
    }
}

There is a process that is in charge to write files into a directory. You must filter those files based on their content. We have left the code comments on purpose because it was the way we developed iteratively. We tested something locally with Camel JBang, until we came to the final version of the integration. We had tested the Filter EIP but while testing we needed a Content Based Router EIP instead. It must sound a familiar process as it happens probably every time we develop something.

Now that we are ready, we run a last round of testing locally via Camel JBang:

$ jbang run -Dcamel.jbang.version=3.18.3 camel@apache/camel run MyJBangRoute.java
2022-11-23 12:19:11.516  INFO 55909 --- [           main] org.apache.camel.main.MainSupport        : Apache Camel (JBang) 3.18.3 is starting
2022-11-23 12:19:11.592  INFO 55909 --- [           main] org.apache.camel.main.MainSupport        : Using Java 11.0.17 with PID 55909. Started by squake in /home/squake/workspace/jbang/camel-blog
2022-11-23 12:19:14.020  INFO 55909 --- [           main] e.camel.impl.engine.AbstractCamelContext : Apache Camel 3.18.3 (CamelJBang) is starting
2022-11-23 12:19:14.220  INFO 55909 --- [           main] e.camel.impl.engine.AbstractCamelContext : Routes startup (started:1)
2022-11-23 12:19:14.220  INFO 55909 --- [           main] e.camel.impl.engine.AbstractCamelContext :     Started route1 (file:///tmp/input)
2022-11-23 12:19:14.220  INFO 55909 --- [           main] e.camel.impl.engine.AbstractCamelContext : Apache Camel 3.18.3 (CamelJBang) started in 677ms (build:133ms init:344ms start:200ms JVM-uptime:3s)
2022-11-23 12:19:27.757  INFO 55909 --- [le:///tmp/input] MyJBangRoute.java:11                     : Processing file file_1669202367381 with content: some entry

2022-11-23 12:19:27.758  INFO 55909 --- [le:///tmp/input] MyJBangRoute:21                          : WARN not checked!
2022-11-23 12:19:32.276  INFO 55909 --- [le:///tmp/input] MyJBangRoute.java:11                     : Processing file file_1669202372252 with content: some entry checked

We have tested adding files on the input directory. Ready to promote to my development cluster!

Use the Camel K JBang plugin here to run the integration on K8s so you do not need to switch tooling.

Run the following command:

camel k run MyJBangRoute.java

The Integration started correctly, but we are using a file system that is local to the Pod where the Integration is running.

$ kamel run MyJBangRoute.java --volume my-pv-claim-input:/tmp/input --volume my-pv-claim-output:/tmp/output --volume my-pv-claim-discarded:/tmp/discarded --dev
...
[1] 2022-11-23 11:39:26,281 INFO  [route1] (Camel (camel-1) thread #1 - file:///tmp/input) Processing file file_1669203565971 with content: some entry
[1]
[1] 2022-11-23 11:39:26,303 INFO  [route1] (Camel (camel-1) thread #1 - file:///tmp/input) WARN not checked!
[1] 2022-11-23 11:39:32,322 INFO  [route1] (Camel (camel-1) thread #1 - file:///tmp/input) Processing file file_1669203571981 with content: some entry checked

Another benefit of Camel JBang is the ability to test a Kamelet locally. Until now, the easiest possibility to test a Kamelet was to upload to a Kubernetes cluster and to run some Integration using it via Camel K.

Let us develop a simple Kamelet for this scope. It is a Coffee source we are using to generate random coffee events.

apiVersion: camel.apache.org/v1
kind: Kamelet
metadata:
  name: coffee-source
  annotations:
    camel.apache.org/kamelet.support.level: "Stable"
    camel.apache.org/catalog.version: "4.7.0-SNAPSHOT"
    camel.apache.org/kamelet.icon: "data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiIHN0YW5kYWxvbmU9Im5vIj8+CjxzdmcgeG1sbnM6cmRmPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5LzAyLzIyLXJkZi1zeW50YXgtbnMjIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIGhlaWdodD0iOTJwdCIgd2lkdGg9IjkycHQiIHZlcnNpb249IjEuMCIgeG1sbnM6Y2M9Imh0dHA6Ly9jcmVhdGl2ZWNvbW1vbnMub3JnL25zIyIgeG1sbnM6eGxpbms9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkveGxpbmsiIHhtbG5zOmRjPSJodHRwOi8vcHVybC5vcmcvZGMvZWxlbWVudHMvMS4xLyI+Cgk8ZGVmcz4KCQk8bGluZWFyR3JhZGllbnQgaWQ9ImEiPgoJCQk8c3RvcCBzdG9wLWNvbG9yPSIjZmZmZmZmIiBzdG9wLW9wYWNpdHk9Ii41IiBvZmZzZXQ9IjAiLz4KCQkJPHN0b3Agc3RvcC1jb2xvcj0iI2ZmZmZmZiIgc3RvcC1vcGFjaXR5PSIuMSIgb2Zmc2V0PSIxIi8+CgkJPC9saW5lYXJHcmFkaWVudD4KCQk8bGluZWFyR3JhZGllbnQgaWQ9ImQiIHkyPSI2Mi4yOTkiIHhsaW5rOmhyZWY9IiNhIiBncmFkaWVudFVuaXRzPSJ1c2VyU3BhY2VPblVzZSIgeTE9IjMzLjYxIiBncmFkaWVudFRyYW5zZm9ybT0ibWF0cml4KC43ODQ3OSAwIDAgMS4yNzQyIC0yNS42OTEgLTguNTYzNSkiIHgyPSI5NS42ODkiIHgxPSI1OS4wOTkiLz4KCQk8bGluZWFyR3JhZGllbnQgaWQ9ImMiIHkyPSIyNDEuMDkiIHhsaW5rOmhyZWY9IiNhIiBncmFkaWVudFVuaXRzPSJ1c2VyU3BhY2VPblVzZSIgeTE9IjIwOC4wNCIgZ3JhZGllbnRUcmFuc2Zvcm09Im1hdHJpeCgxLjk3NzcgMCAwIC41MDU2MyAtMjUuNjkxIC04LjU2MzUpIiB4Mj0iMjguMTc5IiB4MT0iMTcuNDAyIi8+CgkJPGxpbmVhckdyYWRpZW50IGlkPSJiIiB5Mj0iODAuOTA5IiB4bGluazpocmVmPSIjYSIgZ3JhZGllbnRVbml0cz0idXNlclNwYWNlT25Vc2UiIHkxPSI1NS45ODgiIGdyYWRpZW50VHJhbnNmb3JtPSJtYXRyaXgoMS41NDY5IDAgMCAuNjQ2NDcgLTI1LjY5MSAtOC41NjM1KSIgeDI9Ijg3LjA3NCIgeDE9IjcwLjA2MyIvPgoJPC9kZWZzPgoJPHBhdGggc3Ryb2tlLWxpbmVqb2luPSJyb3VuZCIgZD0ibTEyLjQ2MyAyNC44ODZjMi4zNTIgMS4yMjYgMjIuMzY4IDUuNDg4IDMzLjk3MiA1LjIyNiAxNi41MjcgMC4yNjIgMzAuMzEzLTYuMDQ5IDMyLjkyNy03LjA1NSAwIDEuNDMzLTIuMzA3IDEwLjI3My0yLjYxNCAxNS42NzkgMCA1LjQ0OCAxLjgzIDI4LjQxNSAyLjA5MSAzMy43MTEgMC44NjggNi4xNzggMi43MDQgMTMuODYxIDQuNDQzIDE5LjA3NyAxLjgyOSAzLjU1My0yMy41NjMgOS44NTYtMzQuNzU3IDEwLjQ1Ni0xMi42MDIgMC43OC0zOC45MzctNC4zNzUtMzcuMzY5LTguMzY2IDAtMy45NjggMy42NTktMTMuMzgzIDMuNjU5LTE5LjU5OSAwLjUyMi02LjAyNS0wLjI2Mi0yMy4yNzMtMC4yNjItMzAuODM2LTAuMjYxLTYuNzgtMS4wNTMtMTIuNTYxLTIuMDktMTguMjkzeiIgZmlsbC1ydWxlPSJldmVub2RkIiBzdHJva2U9IiMwMDAwMDAiIHN0cm9rZS13aWR0aD0iMXB0IiBmaWxsPSIjZmJkOTAwIi8+Cgk8cGF0aCBkPSJtMTAuNjMzIDk0LjY1OWMtNS41ODUxLTEuMzMxLTcuODc4NiAxMC4xMTEtMS44Mjg4IDEyLjAyMSA2LjM2NzggMy43NSAyOS43MDMgNy4wNiAzOS4xOTkgNi4yNyAxMS4xMDEtMC4yNiAzMS4xOTItNC40NCAzNS44MDEtOC4zNiA2LjEzNC0zLjkyIDUuNDY2LTEzLjA2NiAwLTEyLjAyMS0zLjI3OCAzLjY1OC0yNi42OTkgOC44ODEtMzYuNTg1IDkuNDExLTkuMjIzIDAuNzgtMzAuNzQ5LTIuNTMtMzYuNTg2LTcuMzIxeiIgZmlsbC1ydWxlPSJldmVub2RkIiBzdHJva2U9IiMwMDAwMDAiIHN0cm9rZS13aWR0aD0iMXB0IiBmaWxsPSIjZmJmM2JmIi8+Cgk8cGF0aCBzdHJva2UtbGluZWpvaW49ImJldmVsIiBkPSJtNzcuMzgyIDM0LjA0NmMxLjI0NS0zLjIxMiA5LjYzOS02Ljk3MiAxMi4zNjQtNy41MTYgNC42ODYtMS4wNSAxMi4zODQtMS4zODggMTYuNzY0IDQuMjggNy45NCAxMC4zMjMgNi43NiAyOC42MjYgMi44NiAzNC42MzgtMi43OCA1LjEwNC05LjM3MSAxMC4yODItMTQuNjM1IDExLjg3OC01LjE1MSAxLjUzMy0xMi43MDcgMi42NjEtMTQuMzMzIDMuNzExLTAuMzUtMS4yOTYtMS4zMjctNy4zODgtMS4zOC05LjA3MSAxLjk1IDAuMTI4IDcuNDg5LTAuODkzIDExLjY5NS0xLjg2OCAzLjkwMi0wLjg5OSA2LjQ1LTMuMjc0IDkuMzMzLTYuMjIyIDUtNC43IDQuMzUtMjEuMTYgMC41NC0yNS4wNTctMi4yMzMtMi4yNjItNi44NDktMy45MDQtOS45MTUtMy4zMjMtNC45OTIgMS4wMzItMTMuNjc3IDcuMzY2LTEzLjY3NyA2Ljk4LTAuNTA4LTIuMDgtMC4yNS02LjE1OSAwLjM4NC04LjQzeiIgZmlsbC1ydWxlPSJldmVub2RkIiBzdHJva2U9IiMwMDAwMDAiIHN0cm9rZS13aWR0aD0iMS4yNSIgZmlsbD0iI2ZiZjNiZiIvPgoJPHBhdGggc3Ryb2tlLWxpbmVqb2luPSJyb3VuZCIgZD0ibTMyLjAyMiAzOC4zNjhjMS42NTUgMS4yMDYtMS4zNTUgMTYuOTU1LTAuOTQyIDI4LjEzMSAwLjQxNCAxNC4yOTUgMS40NDQgMjMuNTI4LTAuNTIxIDI0LjYzNS0zLjEwOCAxLjY3NS05LjkwMS0wLjEzNS0xMi4wNDYtMi40Mi0xLjI3My0xLjUwNyAxLjgwNi0xMC4yNCAyLjAxMy0xNi40MjktMC40MTQtOC43MTEtMS43MDMtMzMuMzAzLTAuNDYxLTM0Ljc3OCAyLjI1Mi0yLjA1MyA5LjY4MS0xLjE1MiAxMS45NTcgMC44NjF6IiBmaWxsLXJ1bGU9ImV2ZW5vZGQiIHN0cm9rZT0iIzAwMDAwMCIgc3Ryb2tlLXdpZHRoPSIxLjI1IiBmaWxsPSIjZmJlNjAwIi8+Cgk8cGF0aCBkPSJtNDAuNjEyIDM5LjAzN2MtMS40NzggMS40MjQtMC4wNjMgMTkuNjI1LTAuMDYzIDIyLjU1OSAwLjMwNSAzLjgwOC0xLjEwMSAyNy40NTItMC4xNzggMjguOTU0IDEuODQ4IDIuMTIyIDEwLjIxNiAyLjQ0MiAxMy4wMDEtMC4zNTYgMS41MDUtMS44NzUtMC40NzgtMjIuNTQ0LTAuNDc4LTI3LjY4IDAtNS41MSAxLjQwNy0yMi4wNTItMC40NC0yMy41OC0yLjAzMy0yLjE0OS04LjQ0LTMuMTgtMTEuODQyIDAuMTAzeiIgZmlsbC1ydWxlPSJldmVub2RkIiBzdHJva2U9IiMwMDAwMDAiIHN0cm9rZS13aWR0aD0iMXB0IiBmaWxsPSIjZmJlNjAwIi8+Cgk8cGF0aCBzdHJva2UtbGluZWpvaW49InJvdW5kIiBkPSJtNjAuMzAxIDM3LjU5M2MtMS42NTggMS4yNTYgMS4xNzkgMTUuOCAxLjE5NCAyNi45ODIgMC4xMzcgMTQuMjk5LTEuMjQ1IDI0LjY2MiAwLjgyNCAyNS43MDkgMy4yNjggMS41NzggMTAuODgxLTEuNTQyIDEzLTMuODkxIDEuMjUzLTEuNTQ1LTEuNDExLTEwLjE3OS0yLjA4Mi0xNi4zNTgtMC45ODQtOC4xNjQgMC4xNDgtMzMuMTI4LTEuMTg5LTM0LjU2NC0yLjQwMi0xLjk4NC05LjQ4MiAwLjA0LTExLjc0NyAyLjEyMnoiIGZpbGwtcnVsZT0iZXZlbm9kZCIgc3Ryb2tlPSIjMDAwMDAwIiBzdHJva2Utd2lkdGg9IjEuMjUiIGZpbGw9IiNmYmU2MDAiLz4KCTxwYXRoIGQ9Im01My41ODIgMzEuMTJjLTQuOTg5IDEuMTA5LTM2LjU4OC0zLjE0MS0zOS43MjktNC44MDQgMC45MjQgNC42MiAzLjE0MSA0NS4yNzIgMS42NjMgNDkuODkyIDAuMTg1IDIuMDMyLTMuODggMTUuMTUyLTMuNjk1IDE3LjkyNCAxNy4xODQtNjguMzcgMzkuNzI4LTQ4Ljk2OCA0MS43NjEtNjMuMDEyeiIgZmlsbC1ydWxlPSJldmVub2RkIiBmaWxsPSJ1cmwoI2QpIi8+Cgk8cGF0aCBkPSJtMTAuMDI3IDk1LjMwOWMtMy4wNTE1LTAuODk3LTUuMjA1MyA2LjgyMS0yLjg3MiA5LjE1MSA1Ljc0MyAyLjY5IDEzLjI4Mi0yLjMzIDM4LjIzLTEuNjEtMTIuNzQzLTAuMzYtMzEuNTg5LTIuODc0LTM1LjM1OC03LjU0MXoiIGZpbGwtcnVsZT0iZXZlbm9kZCIgZmlsbD0idXJsKCNjKSIvPgoJPHBhdGggZD0ibTc4LjU5IDMzLjU2N2M0LjQ4Ny00LjQ4OCA4Ljc5NC01LjU2NCAxMy45OTktNi40NjIgOC43OTEtMi4zMzMgMTQuOTAxIDMuNzY5IDE2Ljg3MSAxMS44NDYtNC40OS03LjE3OS0xMC4yMy04LjI1Ni0xNC4xNzgtOC40MzYtNC4xMjggMC43MTgtMTUuNzk1IDcuODk4LTE2Ljg3MiA5LjE1NHMtMC43MTgtNC4xMjggMC4xOC02LjEwMnoiIGZpbGwtcnVsZT0iZXZlbm9kZCIgZmlsbD0idXJsKCNiKSIvPgoJPHBhdGggc3Ryb2tlLWxpbmVqb2luPSJyb3VuZCIgZD0ibTExLjQwOCA3Ny4zNGMyLjM4MzIgMS4xNTkgNC4yODExLTEuNTY5MyAzLjQ2NDktMy4wMzAzIDAuOTE1MDMgMC4wODY1OCAxLjc5NDgtMC4zMjU0IDEuNzk0OC0xLjc5NDggMC43MjA0NC0wLjcyMDQ0LTAuMzY0NjEtMS44NTQ0LTAuMzY0NjEtMi43MzU3LTAuOTkzNTQtMC45OTM1NCAwLjAwNTYtMi4xNjUgMC4wMDU2LTMuNzI1NyAwLTEuNTUzNSAwLjg5NzQyLTIuNTAyNCAwLjg5NzQyLTQuMTI4MSAwLTIuMzYxMSAyLjA1OTQtMS4xODA3IDAuODk3NDItNC42NjY2IDEuMDg4Mi0wLjQyNDU1IDIuMjc0MS0xLjQ4NDUgMC44OTc0Mi0yLjY5MjMgMi4xNjAxLTAuMjM5NTIgMy4yMTg2LTIuMzU0MiAwLjUzODQ1LTQuNjY2NiA0LjA3MzQgMC00LjIzMDItOC43MzA1IDIuNjkyMy02Ljk5OTkgMi4yMjItMC41NTU1MSAxLjc5NDgtMi4yMTUxIDEuNzk0OC00LjMwNzYgMi44NzE3IDMuOTQ4NyA2Ljg5NTQgMi42MjEzIDcuNTM4MyAwIDEuMzQ4NiA0LjM5OTggMTAuNTkgMi41ODY5IDEwLjU5LTIuODcxNyAwLjE3OTQ4IDYuNzUwMiA3LjExNzcgMy40MDQ2IDguNDM1OCAzLjk0ODYtMS42MTU0IDEuODY2MiAxLjU4NDEgOS4wNzk2IDQuMzA3NiA5LjE1MzctNi4zMDk3IDQuNzMyMy01LjE3MjkgMTMuMDAxIDIuNTEyOCAxNC41MzggMy44OTM4IDAgNS4zODQ1LTMuMjc4NSA1LjM4NDUtNy44OTczIDEuMjU2NCAyLjY0NDcgNi45NzIgNC4yNzk3IDYuOTk5OS0wLjE3OTQ4IDIuODcxNyA1LjU0NDYgNi40OTU5LTEuNDcwNCA0LjMwNzYtMi4xNTM4IDUuMDI1NiAxLjkwNTcgMy4yMTI4LTYuOTgxMSAxLjM3ODUtOS4wNTYgMi44NzE4LTAuOTE0NDggMS44MzQ2LTcuNjE4NCAwLjA1NzQtOS43ODk4IDIuNjIxMiAyLjY2NTIgNi43Mzg1LTAuODMxMTIgNi4yODItNS45MjMgMS4yMjggMy40NjcxIDkuMTQ3NS0wLjM2ODI4IDMuNzY5Mi04LjQzNTggMC0xLjU0NTEtNC40ODcxLTEuNzQ4OC01LjU2NC0wLjUzODQ1LTAuMDE1NDEtNS40NDYxLTQuMDk5Ny05LjY5MjEtNi45OTk5LTguNjE1MiAxLjc5OS0yLjY5MzItOS4wNDgtNC44OTk5LTExLjMwOC0wLjUzOSAxLjM1MS01LjcwMTItMTMuODEtOS4zMzM2LTE0LjE3OS02LjEwMjktMS43NDgtMi41MTI4LTExLjc3MS0yLjU1ODYtMTQuNzE4IDYuMjgxOSAwLTQuODYwNi0xNi4zMDktNi45OTk5LTE1Ljk3NCAwLjM1ODk3LTMuNDg5OS0yLjQzMzEtOS4yMjc0IDAuMzU4OTctOC43OTQ3IDMuMjMwNy01LjM4NDUtMi43MDM0LTcuODQyIDkuNTYxMS0zLjQxMDIgMTAuMjMxLTIuNTEyOCAyLjI2MjQtMi42OTIzIDExLjMxMSAwLjUzODQ1IDExLjEyOC0xLjk3NDMgMi4xMjk3LTAuODk3NDIgOC40MzY2IDEuMjU2NCA4LjYxNTItMS42Nzk0IDIuMzIwNiAwLjI0NTcgMTMuNjc0IDcuMTc5NCAxMS44NDYgMCAyLjUyMzQgMC43MDg3NyA0LjY5NDEtMC4xNzk0OCA3LjM1ODggMCAxLjU0NTUtMC44OTc0MiAyLjg1MjgtMC44OTc0MiA0LjQ4NzEgMC4zNzIwNiAwLjc0NDEyLTEuMjU5NyAyLjcyNDQgMC41Mzg0NSAzLjk0ODYtNC4yMTY3IDEuNzU5My0zLjMwMjQgNC40NjQyLTEuNjcwMSA1LjcyMjZ6IiBmaWxsLXJ1bGU9ImV2ZW5vZGQiIHN0cm9rZT0iIzAwMDAwMCIgc3Ryb2tlLXdpZHRoPSIxcHQiIGZpbGw9IiNmZmZmZmYiLz4KCTxwYXRoIHN0cm9rZS1saW5lam9pbj0icm91bmQiIGQ9Im0xMS4zMTcgMzIuNTc0Yy0xLjUwOTgtMS42NSAxLjIyMS03LjA0IDQuMjQyLTYuNzYzIDAuNjg5LTIuNDc0IDIuNTg2LTIuODkyIDQuNjg4LTIuMTg3LTEuMDQ4LTIuMDQ1IDEuNTAzLTMuOTkyIDMuNzUtMS42ODIgMS41MTctMi42MjIgNC42NzctNC42NDUgNi4zNTYtMy4yMzEtMC4xMzItMy4zNzMgNi4wNjMtNi43OTQgOC4zMzEtMy44MzcgMCAwLjYwNi0wLjM2MiAxLjg3NSAwIDEuODc1IiBzdHJva2U9IiMwMDAwMDAiIHN0cm9rZS1saW5lY2FwPSJyb3VuZCIgc3Ryb2tlLXdpZHRoPSIxcHQiIGZpbGw9Im5vbmUiLz4KCTxwYXRoIHN0cm9rZS1saW5lam9pbj0icm91bmQiIGQ9Im00OC4zNzIgMjIuMzc0Yy0wLjEwNC00LjcyMSAxNC4wMDktOC41OTEgMTEuMjUtMC4zMTMgMS4yNjktMC42MzQgNi44NzUtMS4yOTkgNS44NDQgMi4zMTQgNC4xMjMtMC40NjYgMTAuMzkgMS4xMDQgNi42NjIgNi42ODggMi4zOTYgMS44MDYgMS4zMzEgNi42OTYtMC4zMTkgNS4wNjEiIHN0cm9rZT0iIzAwMDAwMCIgc3Ryb2tlLWxpbmVjYXA9InJvdW5kIiBzdHJva2Utd2lkdGg9IjFwdCIgZmlsbD0ibm9uZSIvPgo8L3N2Zz4K"
    camel.apache.org/provider: "Apache Software Foundation"
    camel.apache.org/kamelet.group: "Coffees"
    camel.apache.org/kamelet.namespace: "Dataset"
  labels:
    camel.apache.org/kamelet.type: "source"
spec:
  definition:
    title: "Coffee Source"
    description: "Produces periodic events about coffees!"
    type: object
    properties:
      period:
        title: Period
        description: The time interval between two events
        type: integer
        default: 5000
  types:
    out:
      mediaType: application/json
  dependencies:
    - "camel:timer"
    - "camel:http"
    - "camel:kamelet"
  template:
    from:
      uri: "timer:coffee"
      parameters:
        period: "{{period}}"
      steps:
        - to: https://random-data-api.com/api/coffee/random_coffee
        - removeHeaders:
            pattern: '*'
        - to: "kamelet:sink"

To test it, we can use a simple Integration to log its content:

- from:
    uri: "kamelet:coffee-source?period=5000"
    steps:
      - log: "${body}"

Now we can run:

$ camel run --local-kamelet-dir=</path/to/local/kamelets/dir> coffee-integration.yaml
2022-11-24 11:27:29.634  INFO 39527 --- [           main] org.apache.camel.main.MainSupport        : Apache Camel (JBang) 3.18.1 is starting
2022-11-24 11:27:29.706  INFO 39527 --- [           main] org.apache.camel.main.MainSupport        : Using Java 11.0.17 with PID 39527. Started by squake in /home/squake/workspace/jbang/camel-blog
2022-11-24 11:27:31.391  INFO 39527 --- [           main] e.camel.impl.engine.AbstractCamelContext : Apache Camel 3.18.1 (CamelJBang) is starting
2022-11-24 11:27:31.590  INFO 39527 --- [           main] org.apache.camel.main.BaseMainSupport    : Property-placeholders summary
2022-11-24 11:27:31.590  INFO 39527 --- [           main] org.apache.camel.main.BaseMainSupport    :     [coffee-source.kamelet.yaml]     period=5000
2022-11-24 11:27:31.590  INFO 39527 --- [           main] org.apache.camel.main.BaseMainSupport    :     [coffee-source.kamelet.yaml]     templateId=coffee-source
2022-11-24 11:27:31.591  INFO 39527 --- [           main] e.camel.impl.engine.AbstractCamelContext : Routes startup (started:2)
2022-11-24 11:27:31.591  INFO 39527 --- [           main] e.camel.impl.engine.AbstractCamelContext :     Started route1 (kamelet://coffee-source)
2022-11-24 11:27:31.591  INFO 39527 --- [           main] e.camel.impl.engine.AbstractCamelContext :     Started coffee-source-1 (timer://coffee)
2022-11-24 11:27:31.591  INFO 39527 --- [           main] e.camel.impl.engine.AbstractCamelContext : Apache Camel 3.18.1 (CamelJBang) started in 1s143ms (build:125ms init:819ms start:199ms JVM-uptime:2s)
2022-11-24 11:27:33.297  INFO 39527 --- [ - timer://coffee] coffee-integration.yaml:4                  : {"id":3648,"uid":"712d4f54-3314-4129-844e-9915002ecbb7","blend_name":"Winter Cowboy","origin":"Lekempti, Ethiopia","variety":"Agaro","notes":"delicate, juicy, sundried tomato, fresh bread, lemonade","intensifier":"juicy"}

This is a boost while you are programming a Kamelet, because you can have a quick feedback without the need of a cluster. Once ready, you can continue your development as usual uploading the Kamelet to the cluster and using in your Camel K integrations.

YAKS is an Open Source test automation platform that leverages Behavior Driven Development concepts for running tests locally and on Cloud infrastructure (For example: Kubernetes or OpenShift). This means that the testing tool is able to run your tests both as local tests and natively on Kubernetes. The framework is specifically designed to verify Serverless and Microservice applications and aims for integration testing with the application under test, up and running in a production-like environment. A typical YAKS test uses the same infrastructure as the application under test and exchanges data/events over different messaging transports (For example: Http REST, Knative eventing, Kafka, JMS and many more).

As YAKS itself is written in Java the runtime uses a Java virtual machine with build tools such as Maven and integrates with well known Java testing frameworks such as JUnit, Cucumber and Citrus to run the tests.

Here is a sample Camel K integration that we like to test in the following. The integration exposes a Http service to the user. The service accepts client Http POST requests that add fruit model objects. The Camel K route applies content based routing to store the fruits in different AWS S3 buckets.

In the test scenario YAKS is going to invoke the Camel K service and verify that the message content has been sent to the right AWS S3 bucket.

Here is a sample fruit model object that is subject to be stored in AWS S3:

{
  "id": 1000,
  "name": "Pineapple",
  "category":{
    "id": "1",
    "name":"tropical"
  },
  "nutrition":{
    "calories": 50,
    "sugar": 9
  },
  "status": "AVAILABLE",
  "price": 1.59,
  "tags": ["sweet"]
}

Here is the Camel K integration route:

from('platform-http:/fruits')
    .log('received fruit ${body}')
    .unmarshal().json()
    .removeHeaders("*")
    .setHeader("CamelAwsS3Key", constant("fruit.json"))
    .choice()
        .when().simple('${body[nutrition][sugar]} <= 5')
            .setHeader("CamelAwsS3BucketName", constant("low-sugar"))
        .when().simple('${body[nutrition][sugar]} > 5 && ${body[nutrition][sugar]} <= 10')
            .setHeader("CamelAwsS3BucketName", constant("medium-sugar"))
        .otherwise()
            .setHeader("CamelAwsS3BucketName", constant("high-sugar"))
    .end()
    .marshal().json()
    .log('sending ${body}')
    .to("aws2-s3://noop?$parameters")

The route uses content based routing EAP based on the nutrition sugar rating of a given fruit in order to send the fruits to different AWS S3 buckets (low-sugar, medium-sugar, high-sugar).

In the following the test case for this integration needs to invoke the exposed service with different fruits and verify its outcome on AWS S3.

In the beginning, let us just write the test and run it locally. For now, we do not care how to deploy the application under test in the Cloud infrastructure as everything is running on the local machine using JBang.

JBang is a fantastic way to just start coding and running Java code and also Camel K integrations (also, see this former blog post about how JBang integrates with Camel K).

YAKS as a framework brings a set of ready-to-use domain specific languages (XML, YAML, Groovy, BDD Cucumber steps) for writing tests to verify your deployed services.

This post uses the Behavior Driven Development integration via Cucumber. So the YAKS test is a single feature file that uses BDD Gherkin syntax like this:

Feature: Camel K Fruit Store

  Background:
    Given URL: http://localhost:8080

  Scenario: Create infrastructure
    # Start AWS S3 container
    Given Enable service S3
    Given start LocalStack container

    # Create Camel K integration
    Given Camel K integration property file aws-s3-credentials.properties
    When load Camel K integration fruit-service.groovy
    Then Camel K integration fruit-service should print Started route1 (platform-http:///fruits)

  Scenario: Verify fruit service
    # Invoke Camel K service
    Given HTTP request body: yaks:readFile('pineapple.json')
    And HTTP request header Content-Type="application/json"
    When send POST /fruits
    Then receive HTTP 200 OK

    # Verify uploaded S3 file
    Given New global Camel context
    Given load to Camel registry amazonS3Client.groovy
    Given Camel exchange message header CamelAwsS3Key="fruit.json"
    Given receive Camel exchange from("aws2-s3://medium-sugar?amazonS3Client=#amazonS3Client&deleteAfterRead=true") with body: yaks:readFile('pineapple.json')

Let us walk through the test step by step. Firstly, the feature file uses the usual Given-When-Then BDD syntax to give context, describe the actions and verify the outcome. Each step calls a specific YAKS action that is provided out of the box by the framework. The user is able to choose from a huge set of steps that automatically perform actions like sending/receiving Http requests/responses, starting Testcontainers, running Camel routes, connecting to a database, publishing events on Kafka or Knative brokers and many more.

In the first scenario the test automatically prepares some required infrastructure. The YAKS test starts a Localstack Testcontainer to have an AWS S3 test instance running (Given start LocalStack container). Then the test loads and starts the Camel K integration under test (When load Camel K integration fruit-service.groovy) and waits for it to properly start. In local testing this step starts the Camel K integration using JBang. Later the post will also run the test in a Kubernetes environment.

Now the infrastructure is up and running and the test is able to load the fruit model object as Http request body (Given HTTP request body: yaks:readFile('pineapple.json')) and invoke the Camel K service (When send POST /fruits). The test waits for the Http response and verifies its 200 OK status.

In the last step, the test verifies that the fruit object has been added to the right AWS S3 bucket (medium-sugar). As YAKS itself is not able to connect to AWS S3 the test uses Apache Camel for this step. The test creates a Camel context, loads a AWS client and connects to AWS S3 with a temporary Camel route (Given receive Camel exchange from("aws2-s3://medium-sugar?amazonS3Client=#amazonS3Client&deleteAfterRead=true")). With this Apache Camel integration YAKS is able to use the complete 300+ Camel components for sending and receiving messages to various messaging transports. The Camel exchange body must be the same fruit model object (yaks:readFile('pineapple.json') as posted in the initial Http request.

YAKS uses the powerful message payload validation capabilities provided by Citrus for this message content verification. The validation is able to compare message contents of type XML, Json, plaintext and many more.

This completes the test case. You can now run this test with Cucumber and JUnit for instance. The easiest way though to directly run tests with YAKS is to use the YAKS command line client. You need not set up a whole project with Maven dependencies and so on. Just write the test file and run with:

$ yaks run fruit-service.feature --local

You should see some log output like this:

INFO |
INFO | ------------------------------------------------------------------------
INFO |        .__  __
INFO |   ____ |__|/  |________ __ __  ______
INFO | _/ ___\|  \   __\_  __ \  |  \/  ___/
INFO | \  \___|  ||  |  |  | \/  |  /\___ \
INFO |  \___  >__||__|  |__|  |____//____  >
INFO |      \/                           \/
INFO |
INFO | C I T R U S  T E S T S  3.4.0
INFO |
INFO | ------------------------------------------------------------------------
INFO |

Scenario: Create infrastructure # fruit-service.feature:6
  Given URL: http://localhost:8080
  Given Enable service S3
  [...]

Scenario: Verify fruit service # fruit-service.feature:20
  Given URL: http://localhost:8080
  Given HTTP request body: yaks:readFile('pineapple.json')
  [...]

Scenario: Remove infrastructure  # fruit-service.feature:31
  Given URL: http://localhost:8080
  Given delete Camel K integration fruit-service
  Given stop LocalStack container

3 Scenarios (3 passed)
18 Steps (18 passed)
0m18,051s


INFO | ------------------------------------------------------------------------
INFO |
INFO | CITRUS TEST RESULTS
INFO |
INFO |  Create infrastructure .......................................... SUCCESS
INFO |  Verify fruit service ........................................... SUCCESS
INFO |  Remove infrastructure .......................................... SUCCESS
INFO |
INFO | TOTAL: 3
INFO | FAILED: 0 (0.0%)
INFO | SUCCESS:    3 (100.0%)
INFO |
INFO | ------------------------------------------------------------------------

3 Scenarios (3 passed)
18 Steps (18 passed)
0m18,051s


Test results: Total: 0, Passed: 1, Failed: 0, Errors: 0, Skipped: 0
    fruit-service (fruit-service.feature): Passed

YAKS is able to run tests both locally and as part of a Kubernetes cluster. When running tests on Cloud infrastructure YAKS leverages the Operator SDK and provides a specific operator to manage the test case resources on the cluster. Each time you declare a test in the form of a custom resource, the YAKS operator automatically takes care of preparing the proper runtime in order to execute the test as a Kubernetes Pod.

Writing a Serverless or Microservices application for instance with Camel K is very declarative. As a developer you just write the Camel route and run it as an integration via the Camel K operator directly on the cluster. The declarative approach as well as the nature of Serverless applications make us rely on a given runtime infrastructure, and it is essential to verify the applications also on that infrastructure. So it is only natural to also move the verifying tests into this very same Cloud infrastructure. This is why YAKS also brings your tests to the Cloud infrastructure for integration and end-to-end testing.

So here is how it works. You are able to run the very same YAKS test that has been run locally also as a Pod in Kubernetes.

YAKS provides a Kubernetes operator and a set of CRDs (custom resources) that we need to install on the cluster. The best way to install YAKS is to use the OperatorHub or the yaks CLI tools that you can download from the YAKS GitHub release pages.

With the yaks-client binary simply run this install command:

$ yaks install

This command prepares your Kubernetes cluster for running tests with YAKS. It will take care of installing the YAKS custom resource definitions, setting up role permissions and creating the YAKS operator in a global operator namespace.

Now that the YAKS operator is up and running you can run the very same test from local testing also on the Cloud infrastructure. The only thing that needs to be done is to adjust the Http endpoint URL of the Camel K integration from http://localhost:8080 to http://fruit-service.${YAKS_NAMESPACE}

$ yaks run fruit-service.feature

Instead of using local JBang tooling to run the test locally now the YAKS CLI connects to the Kubernetes cluster to create the test as a custom resource. From there the YAKS operator takes over preparing the test runtime and running the test as a Pod.

The test did prepare some infrastructure, in particular the Camel K integration and the AWS S3 Localstack Testcontainer instance. How does that work inside Kubernetes? YAKS completely takes care of it. The Camel K integration is run with the Camel K operator running on the same Kubernetes cluster. And the Testcontainer AWS S3 instance is automatically run as a Pod in Kubernetes. Even connection settings are handled automatically. It just works!

You will see some similar test log output when running the test remotely and the test performs its actions and its validation exactly the same as locally.

You can also review the test Pod outcome with:

$ yaks ls

This is an example output you should get:

NAME           PHASE   TOTAL  PASSED  FAILED  SKIPPED  ERRORS
fruit-service  Passed  3      3       0       0        0

The whole demo code is available on this GitHub repository. It also shows how to integrate the tests in a GitHub CI actions workflow, so you can run the tests automatically with every code change.

Apache Camel K is a lightweight integration framework built from Apache Camel that runs natively on Kubernetes and is specifically designed for serverless and microservice architectures.

Users of Camel K can instantly run integration code written in Camel DSL on their preferred cloud (Kubernetes or OpenShift).

If the subject under test is a Camel K integration, you can leverage the YAKS Camel K bindings that provide useful steps for managing Camel K integrations.

Working with Camel K integrations

Given create Camel K integration helloworld.groovy
"""
from('timer:tick?period=1000')
  .setBody().constant('Hello world from Camel K!')
  .to('log:info')
"""
Given Camel K integration helloworld is running
Then Camel K integration helloworld should print Hello world from Camel K!

The YAKS framework provides the Camel K extension library by default. You can create a new Camel K integration and check the status of the integration (e.g. running).

The following sections describe the available Camel K steps in detail.

YAKS_CAMELK_API_VERSION=v1alpha1

Given create Camel K integration {name}.groovy
"""
<<Camel DSL>>
"""

Given create Camel K integration {name}.groovy with configuration:
  | dependencies | mvn:org.foo:foo:1.0,mvn:org.bar:bar:0.9 |
  | traits       | quarkus.native=true,quarkus.enabled=true,route.enabled=true |
  | properties   | foo.key=value,bar.key=value |
  | source       | <<Camel DSL>> |

Given load Camel K integration {name}.groovy

Given delete Camel K integration {name}

Given Camel K integration {name} is running

Given Camel K resource polling configuration
    | maxAttempts          | 10   |
    | delayBetweenAttempts | 1000 |

Given Camel K integration {name} should print {log-message}

Given Camel K resource polling configuration
    | maxAttempts          | 10   |
    | delayBetweenAttempts | 1000 |

Given Camel K integration {name} should not print {log-message}

Given Disable auto removal of Camel K resources

Given Enable auto removal of Camel K resources

Kamelets are a form of predefined Camel route templates implemented in Camel K. Usually a Kamelet encapsulates a certain functionality (e.g. send messages to an endpoint). Additionaly Kamelets define a set of properties that the user needs to provide when using the Kamelet.

YAKS provides steps to manage Kamelets.

YAKS_KAMELET_API_VERSION=v1

Given Kamelet dataType in="{mediaType}"

Given Kamelet title "{title}"

Given Kamelet template
"""
from:
  uri: timer:tick
  parameters:
    period: "#property:period"
  steps:
  - set-body:
      constant: "{{message}}"
  - to: "kamelet:sink"
"""

Given Kamelet property definition message
  | type     | string        |
  | required | true          |
  | example  | "hello world" |
  | default  | "hello"       |

Given Kamelet source timer.yaml
"""
<<YAML>>
"""

Given create Kamelet {name}

Given create Kamelet {name} with template
"""
<<YAML>>
"""

Given load Kamelet {name}.kamelet.yaml

Given delete Kamelet {name}

Given Kamelet {name} is available$

You can bind a Kamelet as a source to a sink. This concept is described with Pipes. YAKS as a framework is able to create and verify Pipes in combination with Kamelets.

Given bind Kamelet {name} to uri {uri}

Given bind Kamelet {kamelet} to Kafka topic {topic}

Given bind Kamelet {kamelet} to Knative channel {channel}

Given bind Kamelet {kamelet} to Knative channel {channel} of kind {kind}

Given Pipe source properties
  | {property}  | {value} |

Given Pipe source properties
  | message  | "Hello world" |

Given Pipe sink properties
  | {property}  | {value} |

Given create Pipe {name}

Given load Pipe {name}.yaml

Given delete Pipe {name}

Given Pipe {name} is available$

Given Disable auto removal of Kamelet resources

Given Enable auto removal of Kamelet resources