Chapter 77. CSV DataFormat

Available as of Camel version 1.3

The CSV Data Format uses Apache Commons CSV to handle CSV payloads (Comma Separated Values) such as those exported/imported by Excel.

77.1. Options

The CSV dataformat supports 28 options, which are listed below.

Name	Default	Java Type	Description
formatRef		`String`	The reference format to use, it will be updated with the other format options, the default value is CSVFormat.DEFAULT
formatName		`String`	The name of the format to use, the default value is CSVFormat.DEFAULT
commentMarkerDisabled	`false`	`Boolean`	Disables the comment marker of the reference format.
commentMarker		`String`	Sets the comment marker of the reference format.
delimiter		`String`	Sets the delimiter to use. The default value is , (comma)
escapeDisabled	`false`	`Boolean`	Use for disabling using escape character
escape		`String`	Sets the escape character to use
headerDisabled	`false`	`Boolean`	Use for disabling headers
header		`List`	To configure the CSV headers
allowMissingColumnNames	`false`	`Boolean`	Whether to allow missing column names.
ignoreEmptyLines	`false`	`Boolean`	Whether to ignore empty lines.
ignoreSurroundingSpaces	`false`	`Boolean`	Whether to ignore surrounding spaces
nullStringDisabled	`false`	`Boolean`	Used to disable null strings
nullString		`String`	Sets the null string
quoteDisabled	`false`	`Boolean`	Used to disable quotes
quote		`String`	Sets the quote which by default is
recordSeparatorDisabled		`String`	Used for disabling record separator
recordSeparator		`String`	Sets the record separator (aka new line) which by default is new line characters (CRLF)
skipHeaderRecord	`false`	`Boolean`	Whether to skip the header record in the output
quoteMode		`String`	Sets the quote mode
ignoreHeaderCase	`false`	`Boolean`	Sets whether or not to ignore case when accessing header names.
trim	`false`	`Boolean`	Sets whether or not to trim leading and trailing blanks.
trailingDelimiter	`false`	`Boolean`	Sets whether or not to add a trailing delimiter.
lazyLoad	`false`	`Boolean`	Whether the unmarshalling should produce an iterator that reads the lines on the fly or if all the lines must be read at one.
useMaps	`false`	`Boolean`	Whether the unmarshalling should produce maps (HashMap)for the lines values instead of lists. It requires to have header (either defined or collected).
useOrderedMaps	`false`	`Boolean`	Whether the unmarshalling should produce ordered maps (LinkedHashMap) for the lines values instead of lists. It requires to have header (either defined or collected).
recordConverterRef		`String`	Refers to a custom CsvRecordConverter to lookup from the registry to use.
contentTypeHeader	`false`	`Boolean`	Whether the data format should set the Content-Type header with the type from the data format if the data format is capable of doing so. For example application/xml for data formats marshalling to XML, or application/json for data formats marshalling to JSon etc.

77.2. Spring Boot Auto-Configuration

The component supports 29 options, which are listed below.

Name	Description	Default	Type
camel.dataformat.csv.allow-missing-column-names	Whether to allow missing column names.	false	Boolean
camel.dataformat.csv.comment-marker	Sets the comment marker of the reference format.		String
camel.dataformat.csv.comment-marker-disabled	Disables the comment marker of the reference format.	false	Boolean
camel.dataformat.csv.content-type-header	Whether the data format should set the Content-Type header with the type from the data format if the data format is capable of doing so. For example application/xml for data formats marshalling to XML, or application/json for data formats marshalling to JSon etc.	false	Boolean
camel.dataformat.csv.delimiter	Sets the delimiter to use. The default value is , (comma)		String
camel.dataformat.csv.enabled	Enable csv dataformat	true	Boolean
camel.dataformat.csv.escape	Sets the escape character to use		String
camel.dataformat.csv.escape-disabled	Use for disabling using escape character	false	Boolean
camel.dataformat.csv.format-name	The name of the format to use, the default value is CSVFormat.DEFAULT		String
camel.dataformat.csv.format-ref	The reference format to use, it will be updated with the other format options, the default value is CSVFormat.DEFAULT		String
camel.dataformat.csv.header	To configure the CSV headers		List
camel.dataformat.csv.header-disabled	Use for disabling headers	false	Boolean
camel.dataformat.csv.ignore-empty-lines	Whether to ignore empty lines.	false	Boolean
camel.dataformat.csv.ignore-header-case	Sets whether or not to ignore case when accessing header names.	false	Boolean
camel.dataformat.csv.ignore-surrounding-spaces	Whether to ignore surrounding spaces	false	Boolean
camel.dataformat.csv.lazy-load	Whether the unmarshalling should produce an iterator that reads the lines on the fly or if all the lines must be read at one.	false	Boolean
camel.dataformat.csv.null-string	Sets the null string		String
camel.dataformat.csv.null-string-disabled	Used to disable null strings	false	Boolean
camel.dataformat.csv.quote	Sets the quote which by default is		String
camel.dataformat.csv.quote-disabled	Used to disable quotes	false	Boolean
camel.dataformat.csv.quote-mode	Sets the quote mode		String
camel.dataformat.csv.record-converter-ref	Refers to a custom CsvRecordConverter to lookup from the registry to use.		String
camel.dataformat.csv.record-separator	Sets the record separator (aka new line) which by default is new line characters (CRLF)		String
camel.dataformat.csv.record-separator-disabled	Used for disabling record separator		String
camel.dataformat.csv.skip-header-record	Whether to skip the header record in the output	false	Boolean
camel.dataformat.csv.trailing-delimiter	Sets whether or not to add a trailing delimiter.	false	Boolean
camel.dataformat.csv.trim	Sets whether or not to trim leading and trailing blanks.	false	Boolean
camel.dataformat.csv.use-maps	Whether the unmarshalling should produce maps (HashMap)for the lines values instead of lists. It requires to have header (either defined or collected).	false	Boolean
camel.dataformat.csv.use-ordered-maps	Whether the unmarshalling should produce ordered maps (LinkedHashMap) for the lines values instead of lists. It requires to have header (either defined or collected).	false	Boolean

77.3. Marshalling a Map to CSV

The component allows you to marshal a Java Map (or any other message type that can be converted in a Map) into a CSV payload.

Considering the following body

Map<String, Object> body = new LinkedHashMap<>();
body.put("foo", "abc");
body.put("bar", 123);

and this Java route definition

from("direct:start")
    .marshal().csv()
    .to("mock:result");

or this XML route definition

<route>
    <from uri="direct:start" />
    <marshal>
        <csv />
    </marshal>
    <to uri="mock:result" />
</route>

then it will produce

abc,123

77.4. Unmarshalling a CSV message into a Java List

Unmarshalling will transform a CSV messsage into a Java List with CSV file lines (containing another List with all the field values).

An example: we have a CSV file with names of persons, their IQ and their current activity.

Jack Dalton, 115, mad at Averell
Joe Dalton, 105, calming Joe
William Dalton, 105, keeping Joe from killing Averell
Averell Dalton, 80, playing with Rantanplan
Lucky Luke, 120, capturing the Daltons

We can now use the CSV component to unmarshal this file:

from("file:src/test/resources/?fileName=daltons.csv&noop=true")
    .unmarshal().csv()
    .to("mock:daltons");

The resulting message will contain a List<List<String>> like…

List<List<String>> data = (List<List<String>>) exchange.getIn().getBody();
for (List<String> line : data) {
    LOG.debug(String.format("%s has an IQ of %s and is currently %s", line.get(0), line.get(1), line.get(2)));
}

77.5. Marshalling a List<Map> to CSV

Available as of Camel 2.1

If you have multiple rows of data you want to be marshalled into CSV format you can now store the message payload as a List<Map<String, Object>> object where the list contains a Map for each row.

77.6. File Poller of CSV, then unmarshaling

Given a bean which can handle the incoming data…

MyCsvHandler.java

// Some comments here
public void doHandleCsvData(List<List<String>> csvData)
{
    // do magic here
}

your route then looks as follows

<route>
        <!-- poll every 10 seconds -->
        <from uri="file:///some/path/to/pickup/csvfiles?delete=true&amp;consumer.delay=10000" />
        <unmarshal><csv /></unmarshal>
        <to uri="bean:myCsvHandler?method=doHandleCsvData" />
</route>

77.7. Marshaling with a pipe as delimiter

Considering the following body

Map<String, Object> body = new LinkedHashMap<>();
body.put("foo", "abc");
body.put("bar", 123);

and this Java route definition

// Camel version < 2.15
CsvDataFormat oldCSV = new CsvDataFormat();
oldCSV.setDelimiter("|");
from("direct:start")
    .marshal(oldCSV)
    .to("mock:result")

// Camel version >= 2.15
from("direct:start")
    .marshal(new CsvDataFormat().setDelimiter(&#39;|&#39;))
    .to("mock:result")

or this XML route definition

<route>
  <from uri="direct:start" />
  <marshal>
    <csv delimiter="|" />
  </marshal>
  <to uri="mock:result" />
</route>

then it will produce

abc|123

Using autogenColumns, configRef and strategyRef attributes inside XML # DSL

Available as of Camel 2.9.2 / 2.10 and deleted for Camel 2.15

You can customize the CSV Data Format to make use of your own CSVConfig and/or CSVStrategy. Also note that the default value of the autogenColumns option is true. The following example should illustrate this customization.

<route>
  <from uri="direct:start" />
  <marshal>
    <!-- make use of a strategy other than the default one which is 'org.apache.commons.csv.CSVStrategy.DEFAULT_STRATEGY' -->
    <csv autogenColumns="false" delimiter="|" configRef="csvConfig" strategyRef="excelStrategy" />
  </marshal>
  <convertBodyTo type="java.lang.String" />
  <to uri="mock:result" />
</route>

<bean id="csvConfig" class="org.apache.commons.csv.writer.CSVConfig">
  <property name="fields">
    <list>
      <bean class="org.apache.commons.csv.writer.CSVField">
        <property name="name" value="orderId" />
      </bean>
      <bean class="org.apache.commons.csv.writer.CSVField">
        <property name="name" value="amount" />
      </bean>
    </list>
  </property>
</bean>

<bean id="excelStrategy" class="org.springframework.beans.factory.config.FieldRetrievingFactoryBean">
  <property name="staticField" value="org.apache.commons.csv.CSVStrategy.EXCEL_STRATEGY" />
</bean>

77.8. Using skipFirstLine option while unmarshaling

Available as of Camel 2.10 and deleted for Camel 2.15

You can instruct the CSV Data Format to skip the first line which contains the CSV headers. Using the Spring/XML DSL:

<route>
  <from uri="direct:start" />
  <unmarshal>
    <csv skipFirstLine="true" />
  </unmarshal>
  <to uri="bean:myCsvHandler?method=doHandleCsv" />
</route>

Or the Java DSL:

CsvDataFormat csv = new CsvDataFormat();
csv.setSkipFirstLine(true);

from("direct:start")
  .unmarshal(csv)
.to("bean:myCsvHandler?method=doHandleCsv");

77.9. Unmarshaling with a pipe as delimiter

Using the Spring/XML DSL:

<route>
  <from uri="direct:start" />
  <unmarshal>
    <csv delimiter="|" />
  </unmarshal>
  <to uri="bean:myCsvHandler?method=doHandleCsv" />
</route>

Or the Java DSL:

CsvDataFormat csv = new CsvDataFormat();
CSVStrategy strategy = CSVStrategy.DEFAULT_STRATEGY;
strategy.setDelimiter('|');
csv.setStrategy(strategy);

from("direct:start")
  .unmarshal(csv)
  .to("bean:myCsvHandler?method=doHandleCsv");

CsvDataFormat csv = new CsvDataFormat();
csv.setDelimiter("|");

from("direct:start")
  .unmarshal(csv)
  .to("bean:myCsvHandler?method=doHandleCsv");

CsvDataFormat csv = new CsvDataFormat();
CSVConfig csvConfig = new CSVConfig();
csvConfig.setDelimiter(";");
csv.setConfig(csvConfig);

from("direct:start")
  .unmarshal(csv)
  .to("bean:myCsvHandler?method=doHandleCsv");

Issue in CSVConfig

It looks like that

CSVConfig csvConfig = new CSVConfig();
csvConfig.setDelimiter(';');

doesn’t work. You have to set the delimiter as a String!

77.10. Dependencies

To use CSV in your Camel routes you need to add a dependency on camel-csv, which implements this data format.

If you use Maven you can just add the following to your pom.xml, substituting the version number for the latest and greatest release (see the download page for the latest versions).

<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-csv</artifactId>
  <version>x.x.x</version>
</dependency>

Chapter 77. CSV DataFormat

77.1. Options

77.2. Spring Boot Auto-Configuration

77.3. Marshalling a Map to CSV

77.4. Unmarshalling a CSV message into a Java List

77.5. Marshalling a List<Map> to CSV

77.6. File Poller of CSV, then unmarshaling

77.7. Marshaling with a pipe as delimiter

77.8. Using skipFirstLine option while unmarshaling

77.9. Unmarshaling with a pipe as delimiter

77.10. Dependencies

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

Making open source more inclusive

About Red Hat

Red Hat legal and privacy links

Red Hat legal and privacy links