Red Hat Summit Red Hat Summit지원콘솔개발자평가판 시작연락처

이 콘텐츠는 선택한 언어로 제공되지 않습니다.

Chapter 17. AWS S3 Streaming upload Sink

Upload data to AWS S3 in streaming upload mode.

17.1. Authentication methods
링크 복사

In this Kamelet you can avoid using explicit static credentials by specifying the useDefaultCredentialsProvider option and set it to true.

The order of evaluation for Default Credentials Provider is the following:

  • Java system properties - aws.accessKeyId and aws.secretKey.
  • Environment variables - AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.
  • Web Identity Token from AWS STS.
  • The shared credentials and config files.
  • Amazon ECS container credentials - loaded from the Amazon ECS if the environment variable AWS_CONTAINER_CREDENTIALS_RELATIVE_URI is set.
  • Amazon EC2 Instance profile credentials.

You can also use the Profile Credentials Provider, by setting the useProfileCredentialsProvider option to true and profileCredentialsName to the profile name.

Only one of access key/secret key or default credentials provider could be used

For more information, see the AWS credentials documentation

17.2. Optional Headers
링크 복사

In the header, you can optionally set the file / ce-file property to specify the name of the file to upload.

If you do not set the property in the header, the Kamelet uses the exchange ID for the file name.

17.3. Configuration Options
링크 복사

The following table summarizes the configuration options available for the aws-s3-streaming-upload-sink Kamelet:

Expand
PropertyNameDescriptionTypeDefaultExample

bucketNameOrArn *

Bucket Name

The S3 Bucket name or Amazon Resource Name (ARN)..

string

  

keyName *

Key Name

Setting the key name for an element in the bucket through endpoint parameter. In Streaming Upload, with the default configuration, this is the base for the progressive creation of files.

string

  

region *

AWS Region

The AWS region to access.

string

  

accessKey

Access Key

The access key obtained from AWS.

string

  

autoCreateBucket

Autocreate Bucket

Setting the autocreation of the S3 bucket bucketName.

boolean

False

 

batchMessageNumber

Batch Message Number

The number of messages composing a batch in streaming upload mode.

integer

10

 

batchSize

Batch Size

The batch size (in bytes) in streaming upload mode.

integer

1000000

 

forcePathStyle

Force Path Style

Forces path style when accessing AWS S3 buckets.

boolean

False

 

namingStrategy

Naming Strategy

The naming strategy to use in streaming upload mode. There are 2 enums and the value can be one of progressive, random.

string

progressive

 

overrideEndpoint

Endpoint Overwrite

Select this option to override the endpoint URI. To use this option, you must also provide a URI for the uriEndpointOverride option.

boolean

False

 

profileCredentialsName

Profile Credentials Name

If using a profile credentials provider this parameter sets the profile name.

string

  

restartingPolicy

Restarting Policy

The restarting policy to use in streaming upload mode. There are 2 enums and the value can be one of override, lastPart.

string

lastPart

 

secretKey

Secret Key

The secret key obtained from AWS.

string

  

sessionToken

Session Token

Amazon AWS Session Token used when the user needs to assume a IAM role.

string

  

streamingUploadTimeout

Streaming Upload Timeout

While streaming upload mode is true, this option set the timeout to complete upload.

integer

  

uriEndpointOverride

Overwrite Endpoint URI

The overriding endpoint URI. To use this option, you must also select the overrideEndpoint option.

string

  

useDefaultCredentialsProvider

Default Credentials Provider

Set whether the S3 client should expect to load credentials through a default credentials provider or to expect static credentials to be passed in.

boolean

False

 

useProfileCredentialsProvider

Profile Credentials Provider

Set whether the S3 client should expect to load credentials through a profile credentials provider.

boolean

False

 

useSessionCredentials

Session Credentials

Set whether the S3 client should expect to use Session Credentials. This is useful in situation in which the user needs to assume a IAM role for doing operations in S3.

boolean

False

 

* = Fields marked with an asterisk are mandatory.

17.4. Dependencies
링크 복사

17.4.1. Quarkus dependencies
링크 복사

<dependencies>
  <dependency>
    <groupId>org.apache.camel.kamelets</groupId>
    <artifactId>camel-kamelets-utils</artifactId>
    <version>4.8.5</version>
  </dependency>
  <dependency>
    <groupId>org.apache.camel.quarkus</groupId>
    <artifactId>camel-quarkus-aws-s3-streaming-upload</artifactId>
  </dependency>
  <dependency>
    <groupId>org.apache.camel.quarkus</groupId>
    <artifactId>camel-quarkus-aws2-s3</artifactId>
  </dependency>
  <dependency>
    <groupId>org.apache.camel.quarkus</groupId>
    <artifactId>camel-quarkus-kamelet</artifactId>
  </dependency>
</dependencies>
Copy to Clipboard Toggle word wrap

17.5. Usage
링크 복사

17.5.1. Camel JBang usage
링크 복사

17.5.1.1. Prerequisites for JBang
링크 복사

  • Install JBang.

  • You have executed the following command: 

    jbang app install camel@apache/camel
    Copy to Clipboard Toggle word wrap

17.5.1.2. Running a route with JBang
링크 복사

Suppose you have a file named route.yaml with this content: 

- route:
    from:
      uri: "kamelet:timer-source"
      parameters:
        period: 10000
        message: 'test'
      steps:
        - to:
            uri: "kamelet:log-sink"
Copy to Clipboard Toggle word wrap

You can now run it directly through the following command. 

camel run route.yaml
Copy to Clipboard Toggle word wrap

17.5.2. Knative Sink
링크 복사

You can use the aws-s3-streaming-upload-sink Kamelet as a Knative sink by binding it to a Knative object.

aws-s3-streaming-upload-sink-binding.yaml

apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
  name: aws-s3-streaming-upload-sink-binding
spec:
  source:
    ref:
      kind: Channel
      apiVersion: messaging.knative.dev/v1
      name: mychannel
  sink:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1
      name: aws-s3-streaming-upload-sink
    properties:
      accessKey: "The Access Key"
      bucketNameOrArn: "The Bucket Name"
      keyName: "The Key Name"
      region: "eu-west-1"
      secretKey: "The Secret Key"
Copy to Clipboard Toggle word wrap

17.5.3. Kafka Sink
링크 복사

You can use the aws-s3-streaming-upload-sink Kamelet as a Kafka sink by binding it to a Kafka topic.

aws-s3-streaming-upload-sink-binding.yaml

apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
  name: aws-s3-streaming-upload-sink-binding
spec:
  source:
    ref:
      kind: KafkaTopic
      apiVersion: kafka.strimzi.io/v1beta1
      name: my-topic
  sink:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1
      name: aws-s3-streaming-upload-sink
    properties:
      accessKey: "The Access Key"
      bucketNameOrArn: "The Bucket Name"
      keyName: "The Key Name"
      region: "eu-west-1"
      secretKey: "The Secret Key"
Copy to Clipboard Toggle word wrap

17.6. Kamelets source file
링크 복사

https://github.com/jboss-fuse/camel-kamelets/blob/camel-kamelets-4.10.3-branch/kamelets/aws-s3-streaming-upload-sink.kamelet.yaml

Red Hat logoGithubredditYoutubeTwitter

자세한 정보

평가판, 구매 및 판매

커뮤니티

Red Hat 문서 정보

Red Hat을 사용하는 고객은 신뢰할 수 있는 콘텐츠가 포함된 제품과 서비스를 통해 혁신하고 목표를 달성할 수 있습니다. 최신 업데이트를 확인하세요.

보다 포괄적 수용을 위한 오픈 소스 용어 교체

Red Hat은 코드, 문서, 웹 속성에서 문제가 있는 언어를 교체하기 위해 최선을 다하고 있습니다. 자세한 내용은 다음을 참조하세요.Red Hat 블로그.

Red Hat 소개

Red Hat은 기업이 핵심 데이터 센터에서 네트워크 에지에 이르기까지 플랫폼과 환경 전반에서 더 쉽게 작업할 수 있도록 강화된 솔루션을 제공합니다.

Theme

© 2026 Red Hat맨 위로 이동