Chapter 2. Connecting to Kafka with Kamelets

Set up Kafka:

Create Kamelet Bindings:

Apply the Kamelet Bindings as resources to the project.

The Camel K operator generates a separate Camel K integration for each Kamelet Binding.

In an editor of your choice, create a YAML file with the following basic structure:

apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
  name:
spec:
  source:
  sink:

Add a name for the Kamelet Binding. For this example, the name is coffees-to-kafka because the binding connects the coffee-source Kamelet to the kafka-sink Kamelet.

apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
  name: coffees-to-kafka
spec:
  source:
  sink:

For the Kamelet Binding’s source, specify a data source Kamelet (for example, the coffee-source Kamelet produces events that contain data about coffee) and configure any parameters for the Kamelet.

apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
  name: coffees-to-kafka
spec:
  source:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1alpha1
      name: coffee-source
    properties:
      period: 5000
  sink:

For the Kamelet Binding’s sink, specify the kafka-sink Kamelet and its required properties.

For example, when the Kafka cluster is on AMQ Streams, set the securityProtocol property to “PLAINTEXT”:

apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
  name: coffees-to-kafka
spec:
  source:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1alpha1
      name: coffee-source
    properties:
      period: 5000
  sink:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1alpha1
      name: kafka-sink
    properties:
      bootstrapServers: "broker.url:9092"
      password: "testpassword"
      topic: "test-topic"
      user: "testuser"
      securityProtocol: "PLAINTEXT"

Add the Kamelet Binding as a resource to your OpenShift namespace:

oc apply -f <kamelet binding filename>

For example:

oc apply -f coffees-to-kafka.yaml

The Camel K operator generates and runs a Camel K integration by using the KameletBinding resource. It might take a few minutes to build.

To see the status of the KameletBinding resource:

oc get kameletbindings

To see the status of their integrations:

oc get integrations

To view the integration’s log:

kamel logs <integration> -n <project>

For example:

kamel logs coffees-to-kafka -n my-camel-k-kafka

In an editor of your choice, create a YAML file with the following basic structure:

apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
  name:
spec:
  source:
  sink:

Add a name for the Kamelet Binding. For this example, the name is kafka-to-log because the binding connects the kafka-source Kamelet to the log-sink Kamelet.

apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
  name: kafka-to-log
spec:
  source:
  sink:

For the Kamelet Binding’s sink, specify the data consumer Kamelet (for example, the log-sink Kamelet) and configure any parameters for the Kamelet, for example:

apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
  name: kafka-to-log
spec:
  source:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1alpha1
      name: kafka-source
    properties:
      bootstrapServers: "broker.url:9092"
      password: "testpassword"
      topic: "test-topic"
      user: "testuser"
      securityProtocol: "PLAINTEXT" // only for AMQ streams
  sink:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1alpha1
      name: log-sink
    properties:
      showStreams: true

Add the Kamelet Binding as a resource to your OpenShift namespace:

oc apply -f <kamelet binding filename>

For example:

oc apply -f kafka-to-log.yaml

The Camel K operator generates and runs a Camel K integration by using the KameletBinding resource. It might take a few minutes to build.

To see the status of the KameletBinding resource:

oc get kameletbindings

To see the status of their integrations:

oc get integrations

To view the integration’s log:

kamel logs <integration> -n <project>

For example:

kamel logs kafka-to-log -n my-camel-k-kafka

In the output, you should see coffee events, for example:

INFO  [log-sink-E80C5C904418150-0000000000000001] (Camel (camel-1) thread #0 - timer://tick) {"id":7259,"uid":"a4ecb7c2-05b8-4a49-b0d2-d1e8db5bc5e2","blend_name":"Postmodern Symphony","origin":"Huila, Colombia","variety":"Kona","notes":"delicate, chewy, black currant, red apple, star fruit","intensifier":"balanced"}

To stop a running integration, delete the associated Kamelet Binding resource:

oc delete kameletbindings/<kameletbinding-name>

For example:

oc delete kameletbindings/kafka-to-log