Chapter 6. AWS Kinesis Firehose Sink
Send message to an AWS Kinesis Firehose Stream.
6.1. Authentication methods Copy linkLink copied to clipboard!
In this Kamelet you can avoid using explicit static credentials by specifying the useDefaultCredentialsProvider option and set it to true.
The order of evaluation for Default Credentials Provider is the following:
-
Java system properties -
aws.accessKeyIdandaws.secretKey. -
Environment variables -
AWS_ACCESS_KEY_IDandAWS_SECRET_ACCESS_KEY. - Web Identity Token from AWS STS.
- The shared credentials and config files.
-
Amazon ECS container credentials - loaded from the Amazon ECS if the environment variable
AWS_CONTAINER_CREDENTIALS_RELATIVE_URIis set. - Amazon EC2 Instance profile credentials.
You can also use the Profile Credentials Provider, by setting the useProfileCredentialsProvider option to true and profileCredentialsName to the profile name.
Only one of access key/secret key or default credentials provider could be used
For more information, see the AWS credentials documentation
6.2. Configuration Options Copy linkLink copied to clipboard!
The following table summarizes the configuration options available for the aws-kinesis-firehose-sink Kamelet:
| Property | Name | Description | Type | Default | Example |
|---|---|---|---|---|---|
| region * | AWS Region | The AWS region to access. | string | ||
| streamName * | Stream Name | The name of the stream we want to send to data to. | string | ||
| accessKey | Access Key | The access key obtained from AWS. | string | ||
| overrideEndpoint | Endpoint Overwrite |
Select this option to override the endpoint URI. To use this option, you must also provide a URI for the | boolean | False | |
| profileCredentialsName | Profile Credentials Name | If using a profile credentials provider this parameter sets the profile name. | string | ||
| secretKey | Secret Key | The secret key obtained from AWS. | string | ||
| sessionToken | Session Token | Amazon AWS Session Token used when the user needs to assume a IAM role. | string | ||
| uriEndpointOverride | Overwrite Endpoint URI |
The overriding endpoint URI. To use this option, you must also select the | string | ||
| useDefaultCredentialsProvider | Default Credentials Provider | Set whether the Kinesis Firehose client should expect to load credentials through a default credentials provider or to expect static credentials to be passed in. | boolean | False | |
| useProfileCredentialsProvider | Profile Credentials Provider | Set whether the Kinesis Firehose client should expect to load credentials through a profile credentials provider. | boolean | False | |
| useSessionCredentials | Session Credentials | Set whether the Kinesis Firehose client should expect to use Session Credentials. This is useful in situation in which the user needs to assume a IAM role for doing operations in Kinesis Firehose. | boolean | False |
* = Fields marked with an asterisk are mandatory.
6.3. Dependencies Copy linkLink copied to clipboard!
6.3.1. Quarkus dependencies Copy linkLink copied to clipboard!
<dependencies>
<dependency>
<groupId>org.apache.camel.kamelets</groupId>
<artifactId>camel-kamelets-utils</artifactId>
<version>4.8.5</version>
</dependency>
<dependency>
<groupId>org.apache.camel.quarkus</groupId>
<artifactId>camel-quarkus-aws-kinesis-firehose</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel.quarkus</groupId>
<artifactId>camel-quarkus-aws2-kinesis</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel.quarkus</groupId>
<artifactId>camel-quarkus-kamelet</artifactId>
</dependency>
</dependencies>
6.4. Usage Copy linkLink copied to clipboard!
6.4.1. Camel JBang usage Copy linkLink copied to clipboard!
6.4.1.1. Prerequisites for JBang Copy linkLink copied to clipboard!
- Install JBang.
You have executed the following command:
jbang app install camel@apache/camel
6.4.1.2. Running a route with JBang Copy linkLink copied to clipboard!
Suppose you have a file named route.yaml with this content:
- route:
from:
uri: "kamelet:timer-source"
parameters:
period: 10000
message: 'test'
steps:
- to:
uri: "kamelet:log-sink"
You can now run it directly through the following command.
camel run route.yaml
6.4.2. Knative Sink Copy linkLink copied to clipboard!
You can use the aws-kinesis-firehose-sink Kamelet as a Knative sink by binding it to a Knative object.
aws-kinesis-firehose-sink-binding.yaml
apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
name: aws-kinesis-firehose-sink-binding
spec:
source:
ref:
kind: Channel
apiVersion: messaging.knative.dev/v1
name: mychannel
sink:
ref:
kind: Kamelet
apiVersion: camel.apache.org/v1
name: aws-kinesis-firehose-sink
properties:
accessKey: "The Access Key"
region: "eu-west-1"
secretKey: "The Secret Key"
streamName: "The Stream name"
6.4.3. Kafka Sink Copy linkLink copied to clipboard!
You can use the aws-kinesis-firehose-sink Kamelet as a Kafka sink by binding it to a Kafka topic.
aws-kinesis-firehose-sink-binding.yaml
apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
name: aws-kinesis-firehose-sink-binding
spec:
source:
ref:
kind: KafkaTopic
apiVersion: kafka.strimzi.io/v1beta1
name: my-topic
sink:
ref:
kind: Kamelet
apiVersion: camel.apache.org/v1
name: aws-kinesis-firehose-sink
properties:
accessKey: "The Access Key"
region: "eu-west-1"
secretKey: "The Secret Key"
streamName: "The Stream name"