Chapter 7. AWS Kinesis Sink
Send data to AWS Kinesis.
7.1. Authentication methods Copy linkLink copied to clipboard!
In this Kamelet you can avoid using explicit static credentials by specifying the useDefaultCredentialsProvider option and set it to true.
The order of evaluation for Default Credentials Provider is the following:
-
Java system properties -
aws.accessKeyIdandaws.secretKey. -
Environment variables -
AWS_ACCESS_KEY_IDandAWS_SECRET_ACCESS_KEY. - Web Identity Token from AWS STS.
- The shared credentials and config files.
-
Amazon ECS container credentials - loaded from the Amazon ECS if the environment variable
AWS_CONTAINER_CREDENTIALS_RELATIVE_URIis set. - Amazon EC2 Instance profile credentials.
You can also use the Profile Credentials Provider, by setting the useProfileCredentialsProvider option to true and profileCredentialsName to the profile name.
Only one of access key/secret key or default credentials provider could be used
For more information, see the AWS credentials documentation
7.2. Optional Headers Copy linkLink copied to clipboard!
In the header, you can optionally set the file / ce-partition property to set the Kinesis partition key.
If you do not set the property in the header, the Kamelet uses the exchange ID for the partition key.
You can also set the sequence-number / ce-sequencenumber property in the header to specify the Sequence number.
7.3. Configuration Options Copy linkLink copied to clipboard!
The following table summarizes the configuration options available for the aws-kinesis-sink Kamelet:
| Property | Name | Description | Type | Default | Example |
|---|---|---|---|---|---|
| region * | AWS Region | The AWS region to access. | string | ||
| stream * | Stream Name | The Kinesis stream that you want to access. The Kinesis stream that you specify must already exist. | string | ||
| accessKey | Access Key | The access key obtained from AWS. | string | ||
| overrideEndpoint | Endpoint Overwrite |
Select this option to override the endpoint URI. To use this option, you must also provide a URI for the | boolean | False | |
| profileCredentialsName | Profile Credentials Name | If using a profile credentials provider this parameter sets the profile name. | string | ||
| secretKey | Secret Key | The secret key obtained from AWS. | string | ||
| sessionToken | Session Token | Amazon AWS Session Token used when the user needs to assume a IAM role. | string | ||
| uriEndpointOverride | Overwrite Endpoint URI |
The overriding endpoint URI. To use this option, you must also select the | string | ||
| useDefaultCredentialsProvider | Default Credentials Provider | If true, the Kinesis client loads credentials through a default credentials provider. If false, it uses the basic authentication method (access key and secret key). | boolean | False | |
| useProfileCredentialsProvider | Profile Credentials Provider | Set whether the Kinesis client should expect to load credentials through a profile credentials provider. | boolean | False | |
| useSessionCredentials | Session Credentials | Set whether the Kinesis client should expect to use Session Credentials. This is useful in situation in which the user needs to assume a IAM role for doing operations in Kinesis. | boolean | False |
* = Fields marked with an asterisk are mandatory.
7.4. Dependencies Copy linkLink copied to clipboard!
7.4.1. Quarkus dependencies Copy linkLink copied to clipboard!
<dependencies>
<dependency>
<groupId>org.apache.camel.kamelets</groupId>
<artifactId>camel-kamelets-utils</artifactId>
<version>4.8.5</version>
</dependency>
<dependency>
<groupId>org.apache.camel.quarkus</groupId>
<artifactId>camel-quarkus-aws-kinesis</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel.quarkus</groupId>
<artifactId>camel-quarkus-aws2-kinesis</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel.quarkus</groupId>
<artifactId>camel-quarkus-kamelet</artifactId>
</dependency>
</dependencies>
7.5. Usage Copy linkLink copied to clipboard!
7.5.1. Camel JBang usage Copy linkLink copied to clipboard!
7.5.1.1. Prerequisites for JBang Copy linkLink copied to clipboard!
- Install JBang.
You have executed the following command:
jbang app install camel@apache/camel
7.5.1.2. Running a route with JBang Copy linkLink copied to clipboard!
Suppose you have a file named route.yaml with this content:
- route:
from:
uri: "kamelet:timer-source"
parameters:
period: 10000
message: 'test'
steps:
- to:
uri: "kamelet:log-sink"
You can now run it directly through the following command.
camel run route.yaml
7.5.2. Knative Sink Copy linkLink copied to clipboard!
You can use the aws-kinesis-sink Kamelet as a Knative sink by binding it to a Knative object.
aws-kinesis-sink-binding.yaml
apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
name: aws-kinesis-sink-binding
spec:
source:
ref:
kind: Channel
apiVersion: messaging.knative.dev/v1
name: mychannel
sink:
ref:
kind: Kamelet
apiVersion: camel.apache.org/v1
name: aws-kinesis-sink
properties:
accessKey: "The Access Key"
region: "eu-west-1"
secretKey: "The Secret Key"
stream: "The Stream Name"
7.5.3. Kafka Sink Copy linkLink copied to clipboard!
You can use the aws-kinesis-sink Kamelet as a Kafka sink by binding it to a Kafka topic.
aws-kinesis-sink-binding.yaml
apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
name: aws-kinesis-sink-binding
spec:
source:
ref:
kind: KafkaTopic
apiVersion: kafka.strimzi.io/v1beta1
name: my-topic
sink:
ref:
kind: Kamelet
apiVersion: camel.apache.org/v1
name: aws-kinesis-sink
properties:
accessKey: "The Access Key"
region: "eu-west-1"
secretKey: "The Secret Key"
stream: "The Stream Name"