Chapter 32. JPA
Since Camel 1.0
Both producer and consumer are supported.
The JPA component enables you to store and retrieve Java objects from persistent storage using EJB 3’s Java Persistence Architecture (JPA). Java Persistence Architecture (JPA) is a standard interface layer that wraps Object/Relational Mapping (ORM) products such as OpenJPA, Hibernate, TopLink.
Add the following dependency to your pom.xml
for this component:
<dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-jpa</artifactId> <version>3.20.1.redhat-00050</version> <!-- use the same version as your Camel core version --> </dependency>
32.1. Sending to the endpoint
You can store a Java entity bean in a database by sending it to a JPA producer endpoint. The body of the In
message is assumed to be an entity bean (that is, a POJO with an @Entity annotation on it) or a collection or array of entity beans.
If the body is a List of entities, use entityType=java.util.List
as a configuration passed to the producer endpoint.
If the body does not contain one of the previous listed types, put a Message Translator before the endpoint to perform the necessary conversion first.
You can use query
, namedQuery
or nativeQuery
for the producer as well. For the value of the parameters
, you can use the Simple expression which allows you to retrieve parameter values from Message body, header and etc. Those query can be used for retrieving a set of data with using SELECT
JPQL/SQL statement as well as executing bulk update/delete with using UPDATE
/DELETE
JPQL/SQL statement. Please note that you need to specify useExecuteUpdate
to true
if you execute UPDATE
/DELETE
with namedQuery
as camel doesn’t look into the named query unlike query
and nativeQuery
.
32.2. Consuming from the endpoint
Consuming messages from a JPA consumer endpoint removes (or updates) entity beans in the database. This allows you to use a database table as a logical queue: consumers take messages from the queue and then delete/update them to logically remove them from the queue.
If you do not wish to delete the entity bean when it has been processed (and when routing is done), you can specify consumeDelete=false
on the URI. This will result in the entity being processed each poll.
If you would rather perform some update on the entity to mark it as processed (such as to exclude it from a future query) then you can annotate a method with @Consumed which will be invoked on your entity bean when the entity bean when it has been processed (and when routing is done).
You can use @PreConsumed which will be invoked on your entity bean before it has been processed (before routing).
If you are consuming a lot (100K+) of rows and experience OutOfMemory problems you should set the maximumResults to sensible value.
32.3. URI format
jpa:entityClassName[?options]
For sending to the endpoint, the entityClassName
is optional. If specified, it helps the Type Converterto ensure the body is of the correct type.
For consuming, the entityClassName
is mandatory.
32.4. Configuring Options
Camel components are configured on two separate levels:
- component level
- endpoint level
32.4.1. Configuring Component Options
The component level is the highest level which holds general and common configurations that are inherited by the endpoints. For example a component may have security settings, credentials for authentication, urls for network connection and so forth.
Some components only have a few options, and others may have many. Because components typically have pre configured defaults that are commonly used, then you may often only need to configure a few options on a component; or none at all.
Configuring components can be done with the Component DSL, in a configuration file (application.properties|yaml), or directly with Java code.
32.4.2. Configuring Endpoint Options
Where you find yourself configuring the most is on endpoints, as endpoints often have many options, which allows you to configure what you need the endpoint to do. The options are also categorized into whether the endpoint is used as consumer (from) or as a producer (to), or used for both.
Configuring endpoints is most often done directly in the endpoint URI as path and query parameters. You can also use the Endpoint DSL as a type safe way of configuring endpoints.
A good practice when configuring options is to use Property Placeholders, which allows to not hardcode urls, port numbers, sensitive information, and other settings. In other words placeholders allows to externalize the configuration from your code, and gives more flexibility and reuse.
The following two sections lists all the options, firstly for the component followed by the endpoint.
32.4.3. Component Options
The JPA component supports 9 options, which are listed below.
Name | Description | Default | Type |
---|---|---|---|
aliases (common) | Maps an alias to a JPA entity class. The alias can then be used in the endpoint URI (instead of the fully qualified class name). | Map | |
entityManagerFactory (common) | To use the EntityManagerFactory. This is strongly recommended to configure. | EntityManagerFactory | |
joinTransaction (common) | The camel-jpa component will join transaction by default. You can use this option to turn this off, for example if you use LOCAL_RESOURCE and join transaction doesn’t work with your JPA provider. This option can also be set globally on the JpaComponent, instead of having to set it on all endpoints. | true | boolean |
sharedEntityManager (common) | Whether to use Spring’s SharedEntityManager for the consumer/producer. Note in most cases joinTransaction should be set to false as this is not an EXTENDED EntityManager. | false | boolean |
transactionManager (common) | To use the PlatformTransactionManager for managing transactions. | PlatformTransactionManager | |
transactionStrategy (common) | To use the TransactionStrategy for running the operations in a transaction. | TransactionStrategy | |
bridgeErrorHandler (consumer) | Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. | false | boolean |
lazyStartProducer (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. | false | boolean |
autowiredEnabled (advanced) | Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc. | true | boolean |
32.4.4. Endpoint Options
The JPA endpoint is configured using URI syntax:
jpa:entityType
with the following path and query parameters:
32.4.4.1. Path Parameters (1 parameters)
Name | Description | Default | Type |
---|---|---|---|
entityType (common) | Required Entity class name. | Class |
32.4.4.2. Query Parameters (44 parameters)
Name | Description | Default | Type |
---|---|---|---|
joinTransaction (common) | The camel-jpa component will join transaction by default. You can use this option to turn this off, for example if you use LOCAL_RESOURCE and join transaction doesn’t work with your JPA provider. This option can also be set globally on the JpaComponent, instead of having to set it on all endpoints. | true | boolean |
maximumResults (common) | Set the maximum number of results to retrieve on the Query. | -1 | int |
namedQuery (common) | To use a named query. | String | |
nativeQuery (common) | To use a custom native query. You may want to use the option resultClass also when using native queries. | String | |
persistenceUnit (common) | Required The JPA persistence unit used by default. | camel | String |
query (common) | To use a custom query. | String | |
resultClass (common) | Defines the type of the returned payload (we will call entityManager.createNativeQuery(nativeQuery, resultClass) instead of entityManager.createNativeQuery(nativeQuery)). Without this option, we will return an object array. Only has an affect when using in conjunction with native query when consuming data. | Class | |
consumeDelete (consumer) | If true, the entity is deleted after it is consumed; if false, the entity is not deleted. | true | boolean |
consumeLockEntity (consumer) | Specifies whether or not to set an exclusive lock on each entity bean while processing the results from polling. | true | boolean |
deleteHandler (consumer) | To use a custom DeleteHandler to delete the row after the consumer is done processing the exchange. | DeleteHandler | |
lockModeType (consumer) | To configure the lock mode on the consumer. Enum values:
| PESSIMISTIC_WRITE | LockModeType |
maxMessagesPerPoll (consumer) | An integer value to define the maximum number of messages to gather per poll. By default, no maximum is set. Can be used to avoid polling many thousands of messages when starting up the server. Set a value of 0 or negative to disable. | int | |
preDeleteHandler (consumer) | To use a custom Pre-DeleteHandler to delete the row after the consumer has read the entity. | DeleteHandler | |
sendEmptyMessageWhenIdle (consumer) | If the polling consumer did not poll any files, you can enable this option to send an empty message (no body) instead. | false | boolean |
skipLockedEntity (consumer) | To configure whether to use NOWAIT on lock and silently skip the entity. | false | boolean |
transacted (consumer) | Whether to run the consumer in transacted mode, by which all messages will either commit or rollback, when the entire batch has been processed. The default behavior (false) is to commit all the previously successfully processed messages, and only rollback the last failed message. | false | boolean |
bridgeErrorHandler (consumer (advanced)) | Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. | false | boolean |
exceptionHandler (consumer (advanced)) | To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this option is not in use. By default the consumer will deal with exceptions, that will be logged at WARN or ERROR level and ignored. | ExceptionHandler | |
exchangePattern (consumer (advanced)) | Sets the exchange pattern when the consumer creates an exchange. Enum values:
| ExchangePattern | |
parameters (consumer (advanced)) | This key/value mapping is used for building the query parameters. It is expected to be of the generic type java.util.Map where the keys are the named parameters of a given JPA query and the values are their corresponding effective values you want to select for. When it’s used for producer, Simple expression can be used as a parameter value. It allows you to retrieve parameter values from the message body, header and etc. | Map | |
pollStrategy (consumer (advanced)) | A pluggable org.apache.camel.PollingConsumerPollingStrategy allowing you to provide your custom implementation to control error handling usually occurred during the poll operation before an Exchange have been created and being routed in Camel. | PollingConsumerPollStrategy | |
findEntity (producer) | If enabled then the producer will find a single entity by using the message body as key and entityType as the class type. This can be used instead of a query to find a single entity. | false | boolean |
flushOnSend (producer) | Flushes the EntityManager after the entity bean has been persisted. | true | boolean |
remove (producer) | Indicates to use entityManager.remove(entity). | false | boolean |
useExecuteUpdate (producer) | To configure whether to use executeUpdate() when producer executes a query. When you use INSERT, UPDATE or DELETE statement as a named query, you need to specify this option to 'true'. | Boolean | |
usePersist (producer) | Indicates to use entityManager.persist(entity) instead of entityManager.merge(entity). Note: entityManager.persist(entity) doesn’t work for detached entities (where the EntityManager has to execute an UPDATE instead of an INSERT query)!. | false | boolean |
lazyStartProducer (producer (advanced)) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. | false | boolean |
usePassedInEntityManager (producer (advanced)) | If set to true, then Camel will use the EntityManager from the header JpaConstants.ENTITY_MANAGER instead of the configured entity manager on the component/endpoint. This allows end users to control which entity manager will be in use. | false | boolean |
entityManagerProperties (advanced) | Additional properties for the entity manager to use. | Map | |
sharedEntityManager (advanced) | Whether to use Spring’s SharedEntityManager for the consumer/producer. Note in most cases joinTransaction should be set to false as this is not an EXTENDED EntityManager. | false | boolean |
backoffErrorThreshold (scheduler) | The number of subsequent error polls (failed due some error) that should happen before the backoffMultipler should kick-in. | int | |
backoffIdleThreshold (scheduler) | The number of subsequent idle polls that should happen before the backoffMultipler should kick-in. | int | |
backoffMultiplier (scheduler) | To let the scheduled polling consumer backoff if there has been a number of subsequent idles/errors in a row. The multiplier is then the number of polls that will be skipped before the next actual attempt is happening again. When this option is in use then backoffIdleThreshold and/or backoffErrorThreshold must also be configured. | int | |
delay (scheduler) | Milliseconds before the next poll. | 500 | long |
greedy (scheduler) | If greedy is enabled, then the ScheduledPollConsumer will run immediately again, if the previous run polled 1 or more messages. | false | boolean |
initialDelay (scheduler) | Milliseconds before the first poll starts. | 1000 | long |
repeatCount (scheduler) | Specifies a maximum limit of number of fires. So if you set it to 1, the scheduler will only fire once. If you set it to 5, it will only fire five times. A value of zero or negative means fire forever. | 0 | long |
runLoggingLevel (scheduler) | The consumer logs a start/complete log line when it polls. This option allows you to configure the logging level for that. Enum values:
| TRACE | LoggingLevel |
scheduledExecutorService (scheduler) | Allows for configuring a custom/shared thread pool to use for the consumer. By default each consumer has its own single threaded thread pool. | ScheduledExecutorService | |
scheduler (scheduler) | To use a cron scheduler from either camel-spring or camel-quartz component. Use value spring or quartz for built in scheduler. | none | Object |
schedulerProperties (scheduler) | To configure additional properties when using a custom scheduler or any of the Quartz, Spring based scheduler. | Map | |
startScheduler (scheduler) | Whether the scheduler should be auto started. | true | boolean |
timeUnit (scheduler) | Time unit for initialDelay and delay options. Enum values:
| MILLISECONDS | TimeUnit |
useFixedDelay (scheduler) | Controls if fixed delay or fixed rate is used. See ScheduledExecutorService in JDK for details. | true | boolean |
32.5. Message Headers
The JPA component supports 2 message header(s), which are listed below:
Name | Description | Default | Type |
---|---|---|---|
CamelEntityManager (common) Constant: ENTITY_MANAGER | The JPA EntityManager object. | EntityManager | |
CamelJpaParameters (producer) Constant: link: JPA_PARAMETERS_HEADER | Alternative way for passing query parameters as an Exchange header. | Map |
32.6. Configuring EntityManagerFactory
It is recommended to configure the JPA component to use a specific EntityManagerFactory
instance. If failed to do so each JpaEndpoint
will auto create their own instance of EntityManagerFactory
which most often is not what you want.
For example, you can instantiate a JPA component that references the myEMFactory
entity manager factory, as follows:
<bean id="jpa" class="org.apache.camel.component.jpa.JpaComponent"> <property name="entityManagerFactory" ref="myEMFactory"/> </bean>
The JpaComponent
automatically looks up the EntityManagerFactory
from the Registry which means you do not need to configure this on the JpaComponent
as shown above. You only need to do so if there is ambiguity, in which case Camel will log a WARN.
32.7. Configuring TransactionManager
The JpaComponent
automatically looks up the TransactionManager
from the Registry. If Camel won’t find any TransactionManager
instance registered, it will also look up for the TransactionTemplate
and try to extract TransactionManager
from it.
If none TransactionTemplate
is available in the registry, JpaEndpoint
will auto create their own instance of TransactionManager
which most often is not what you want.
If more than single instance of the TransactionManager
is found, Camel will log a WARN. In such cases you might want to instantiate and explicitly configure a JPA component that references the myTransactionManager
transaction manager, as follows:
<bean id="jpa" class="org.apache.camel.component.jpa.JpaComponent"> <property name="entityManagerFactory" ref="myEMFactory"/> <property name="transactionManager" ref="myTransactionManager"/> </bean>
32.8. Using a consumer with a named query
For consuming only selected entities, you can use the namedQuery
URI query option. First, you have to define the named query in the JPA Entity class:
@Entity @NamedQuery(name = "step1", query = "select x from MultiSteps x where x.step = 1") public class MultiSteps { ... }
After that you can define a consumer uri as shown below:
from("jpa://org.apache.camel.examples.MultiSteps?namedQuery=step1") .to("bean:myBusinessLogic");
32.9. Using a consumer with a query
For consuming only selected entities, you can use the query
URI query option. You only have to define the query option:
from("jpa://org.apache.camel.examples.MultiSteps?query=select o from org.apache.camel.examples.MultiSteps o where o.step = 1") .to("bean:myBusinessLogic");
32.10. Using a consumer with a native query
For consuming only selected entities, you can use the nativeQuery
URI query option. You only have to define the native query option:
from("jpa://org.apache.camel.examples.MultiSteps?nativeQuery=select * from MultiSteps where step = 1") .to("bean:myBusinessLogic");
If you use the native query option, you will receive an object array in the message body.
32.11. Using a producer with a named query
For retrieving selected entities or execute bulk update/delete, you can use the namedQuery
URI query option. First, you have to define the named query in the JPA Entity class:
@Entity @NamedQuery(name = "step1", query = "select x from MultiSteps x where x.step = 1") public class MultiSteps { ... }
After that you can define a producer uri as shown below:
from("direct:namedQuery") .to("jpa://org.apache.camel.examples.MultiSteps?namedQuery=step1");
Note that you need to specify useExecuteUpdate
option to true
to execute UPDATE
/DELETE
statement as a named query.
32.12. Using a producer with a query
For retrieving selected entities or execute bulk update/delete, you can use the query
URI query option. You only have to define the query option:
from("direct:query") .to("jpa://org.apache.camel.examples.MultiSteps?query=select o from org.apache.camel.examples.MultiSteps o where o.step = 1");
32.13. Using a producer with a native query
For retrieving selected entities or execute bulk update/delete, you can use the nativeQuery
URI query option. You only have to define the native query option:
from("direct:nativeQuery") .to("jpa://org.apache.camel.examples.MultiSteps?resultClass=org.apache.camel.examples.MultiSteps&nativeQuery=select * from MultiSteps where step = 1");
If you use the native query option without specifying resultClass
, you will receive an object array in the message body.
32.14. Using the JPA-Based Idempotent Repository
The Idempotent Consumer from the EIP patterns is used to filter out duplicate messages. A JPA-based idempotent repository is provided.
To use the JPA based idempotent repository.
Procedure
-
Set up a
persistence-unit
in the persistence.xml file. -
Set up a
org.springframework.orm.jpa.JpaTemplate
which is used by theorg.apache.camel.processor.idempotent.jpa.JpaMessageIdRepository
. - Configure the error formatting macro: snippet: java.lang.IndexOutOfBoundsException: Index: 20, Size: 20
-
Configure the idempotent repository as
org.apache.camel.processor.idempotent.jpa.JpaMessageIdRepository
. - Create the JPA idempotent repository in the Spring XML file as shown below:
<camelContext xmlns="http://camel.apache.org/schema/spring"> <route id="JpaMessageIdRepositoryTest"> <from uri="direct:start" /> <idempotentConsumer idempotentRepository="jpaStore"> <header>messageId</header> <to uri="mock:result" /> </idempotentConsumer> </route> </camelContext>
When running this Camel component tests inside your IDE
If you run the tests of this component directly inside your IDE, and not through Maven, then you could see exceptions like these:
org.springframework.transaction.CannotCreateTransactionException: Could not open JPA EntityManager for transaction; nested exception is <openjpa-2.2.1-r422266:1396819 nonfatal user error> org.apache.openjpa.persistence.ArgumentException: This configuration disallows runtime optimization, but the following listed types were not enhanced at build time or at class load time with a javaagent: "org.apache.camel.examples.SendEmail". at org.springframework.orm.jpa.JpaTransactionManager.doBegin(JpaTransactionManager.java:427) at org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:371) at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:127) at org.apache.camel.processor.jpa.JpaRouteTest.cleanupRepository(JpaRouteTest.java:96) at org.apache.camel.processor.jpa.JpaRouteTest.createCamelContext(JpaRouteTest.java:67) at org.apache.camel.test.junit5.CamelTestSupport.doSetUp(CamelTestSupport.java:238) at org.apache.camel.test.junit5.CamelTestSupport.setUp(CamelTestSupport.java:208)
The problem here is that the source has been compiled or recompiled through your IDE and not through Maven, which would enhance the byte-code at build time. To overcome this you need to enable dynamic byte-code enhancement of OpenJPA. For example, assuming the current OpenJPA version being used in Camel is 2.2.1, to run the tests inside your IDE you would need to pass the following argument to the JVM:
-javaagent:<path_to_your_local_m2_cache>/org/apache/openjpa/openjpa/2.2.1/openjpa-2.2.1.jar
32.15. Spring Boot Auto-Configuration
When using jpa with Spring Boot make sure to use the following Maven dependency to have support for auto configuration:
<dependency> <groupId>org.apache.camel.springboot</groupId> <artifactId>camel-jpa-starter</artifactId> </dependency>
The component supports 10 options, which are listed below.
Name | Description | Default | Type |
---|---|---|---|
camel.component.jpa.aliases | Maps an alias to a JPA entity class. The alias can then be used in the endpoint URI (instead of the fully qualified class name). | Map | |
camel.component.jpa.autowired-enabled | Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc. | true | Boolean |
camel.component.jpa.bridge-error-handler | Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. | false | Boolean |
camel.component.jpa.enabled | Whether to enable auto configuration of the jpa component. This is enabled by default. | Boolean | |
camel.component.jpa.entity-manager-factory | To use the EntityManagerFactory. This is strongly recommended to configure. The option is a javax.persistence.EntityManagerFactory type. | EntityManagerFactory | |
camel.component.jpa.join-transaction | The camel-jpa component will join transaction by default. You can use this option to turn this off, for example if you use LOCAL_RESOURCE and join transaction doesn’t work with your JPA provider. This option can also be set globally on the JpaComponent, instead of having to set it on all endpoints. | true | Boolean |
camel.component.jpa.lazy-start-producer | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. | false | Boolean |
camel.component.jpa.shared-entity-manager | Whether to use Spring’s SharedEntityManager for the consumer/producer. Note in most cases joinTransaction should be set to false as this is not an EXTENDED EntityManager. | false | Boolean |
camel.component.jpa.transaction-manager | To use the PlatformTransactionManager for managing transactions. The option is a org.springframework.transaction.PlatformTransactionManager type. | PlatformTransactionManager | |
camel.component.jpa.transaction-strategy | To use the TransactionStrategy for running the operations in a transaction. The option is a org.apache.camel.component.jpa.TransactionStrategy type. | TransactionStrategy |