Chapter 32. AWS S3 Storage Service Component

bucketNameOrArn

Required Bucket name or ARN

String

amazonS3Client (common)

Reference to a com.amazonaws.services.sqs.AmazonS3 in the link:https://camel.apache.org/registry.htmlRegistry.

AmazonS3

pathStyleAccess (common)

Whether or not the S3 client should use path style access

false

boolean

policy (common)

The policy for this queue to set in the com.amazonaws.services.s3.AmazonS3#setBucketPolicy() method.

String

proxyHost (common)

To define a proxy host when instantiating the SQS client

String

proxyPort (common)

Specify a proxy port to be used inside the client definition.

Integer

region (common)

The region in which S3 client needs to work

String

useIAMCredentials (common)

Set whether the S3 client should expect to load credentials on an EC2 instance or to expect static credentials to be passed in.

false

boolean

encryptionMaterials (common)

The encryption materials to use in case of Symmetric/Asymmetric client usage

EncryptionMaterials

useEncryption (common)

Define if encryption must be used or not

false

boolean

bridgeErrorHandler (consumer)

Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.

false

boolean

deleteAfterRead (consumer)

Delete objects from S3 after they have been retrieved. The delete is only performed if the Exchange is committed. If a rollback occurs, the object is not deleted. If this option is false, then the same objects will be retrieve over and over again on the polls. Therefore you need to use the Idempotent Consumer EIP in the route to filter out duplicates. You can filter using the S3Constants#BUCKET_NAME and S3Constants#KEY headers, or only the S3Constants#KEY header.

true

boolean

fileName (consumer)

To get the object from the bucket with the given file name

String

includeBody (consumer)

If it is true, the exchange body will be set to a stream to the contents of the file. If false, the headers will be set with the S3 object metadata, but the body will be null. This option is strongly related to autocloseBody option. In case of setting includeBody to true and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically.

true

boolean

maxConnections (consumer)

Set the maxConnections parameter in the S3 client configuration

60

int

maxMessagesPerPoll (consumer)

Gets the maximum number of messages as a limit to poll at each polling. Is default unlimited, but use 0 or negative number to disable it as unlimited.

10

int

prefix (consumer)

The prefix which is used in the com.amazonaws.services.s3.model.ListObjectsRequest to only consume objects we are interested in.

String

sendEmptyMessageWhenIdle (consumer)

If the polling consumer did not poll any files, you can enable this option to send an empty message (no body) instead.

false

boolean

autocloseBody (consumer)

If this option is true and includeBody is true, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to true and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically.

true

boolean

exceptionHandler (consumer)

To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this option is not in use. By default the consumer will deal with exceptions, that will be logged at WARN or ERROR level and ignored.

ExceptionHandler

exchangePattern (consumer)

Sets the exchange pattern when the consumer creates an exchange.

ExchangePattern

pollStrategy (consumer)

A pluggable org.apache.camel.PollingConsumerPollingStrategy allowing you to provide your custom implementation to control error handling usually occurred during the poll operation before an Exchange have been created and being routed in Camel.

PollingConsumerPoll Strategy

deleteAfterWrite (producer)

Delete file object after the S3 file has been uploaded

false

boolean

multiPartUpload (producer)

If it is true, camel will upload the file with multi part format, the part size is decided by the option of partSize

false

boolean

operation (producer)

The operation to do in case the user don’t want to do only an upload

S3Operations

partSize (producer)

Setup the partSize which is used in multi part upload, the default size is 25M.

26214400

long

serverSideEncryption (producer)

Sets the server-side encryption algorithm when encrypting the object using AWS-managed keys. For example use AES256.

String

storageClass (producer)

The storage class to set in the com.amazonaws.services.s3.model.PutObjectRequest request.

String

awsKMSKeyId (producer)

Define the id of KMS key to use in case KMS is enabled

String

useAwsKMS (producer)

Define if KMS must be used or not

false

boolean

synchronous (advanced)

Sets whether synchronous processing should be strictly used, or Camel is allowed to use asynchronous processing (if supported).

false

boolean

accelerateModeEnabled ( advanced)

Define if Accelerate Mode enabled is true or false

false

boolean

chunkedEncodingDisabled ( advanced)

Define if disabled Chunked Encoding is true or false

false

boolean

dualstackEnabled ( advanced)

Define if Dualstack enabled is true or false

false

boolean

forceGlobalBucketAccess Enabled ( advanced)

Define if Force Global Bucket Access enabled is true or false

false

boolean

payloadSigningEnabled ( advanced)

Define if Payload Signing enabled is true or false

false

boolean

backoffErrorThreshold (scheduler)

The number of subsequent error polls (failed due some error) that should happen before the backoffMultipler should kick-in.

int

backoffIdleThreshold (scheduler)

The number of subsequent idle polls that should happen before the backoffMultipler should kick-in.

int

backoffMultiplier (scheduler)

To let the scheduled polling consumer backoff if there has been a number of subsequent idles/errors in a row. The multiplier is then the number of polls that will be skipped before the next actual attempt is happening again. When this option is in use then backoffIdleThreshold and/or backoffErrorThreshold must also be configured.

int

delay (scheduler)

Milliseconds before the next poll. You can also specify time values using units, such as 60s (60 seconds), 5m30s (5 minutes and 30 seconds), and 1h (1 hour).

500

long

greedy (scheduler)

If greedy is enabled, then the ScheduledPollConsumer will run immediately again, if the previous run polled 1 or more messages.

false

boolean

initialDelay (scheduler)

Milliseconds before the first poll starts. You can also specify time values using units, such as 60s (60 seconds), 5m30s (5 minutes and 30 seconds), and 1h (1 hour).

1000

long

runLoggingLevel (scheduler)

The consumer logs a start/complete log line when it polls. This option allows you to configure the logging level for that.

TRACE

LoggingLevel

scheduledExecutorService (scheduler)

Allows for configuring a custom/shared thread pool to use for the consumer. By default each consumer has its own single threaded thread pool.

ScheduledExecutor Service

scheduler (scheduler)

To use a cron scheduler from either camel-spring or camel-quartz2 component

none

ScheduledPollConsumer Scheduler

schedulerProperties (scheduler)

To configure additional properties when using a custom scheduler or any of the Quartz2, Spring based scheduler.

Map

startScheduler (scheduler)

Whether the scheduler should be auto started.

true

boolean

timeUnit (scheduler)

Time unit for initialDelay and delay options.

MILLISECONDS

TimeUnit

useFixedDelay (scheduler)

Controls if fixed delay or fixed rate is used. See ScheduledExecutorService in JDK for details.

true

boolean

accessKey (security)

Amazon AWS Access Key

String

secretKey (security)

Amazon AWS Secret Key

String