org.fastnate.data.csv

Class CsvDataImporter<E>

java.lang.Object

org.fastnate.data.properties.PropertyDataImporter

org.fastnate.data.csv.CsvDataImporter<E>

Type Parameters:

E - The type of the generated objects

public class CsvDataImporter<E>
extends org.fastnate.data.properties.PropertyDataImporter

An importer that reads entities of a specific class from a CSV file. One can either add his own column mapping or use the default properties mapping. Column names are handled case insensitive.

Author:

Tobias Liefke

Field Summary

Fields

Modifier and Type	Field and Description
static String	COLLECTION_DELIMITER The name of the property in the settings that contains the delimiter for elements of Collections respective Maps.
static String	COLUMN_DELIMITER The name of the property in the settings that contains the column delimiter.
static String	LINE_DELIMITER The name of the property in the settings that contains the line delimiter.
static String	MAP_DELIMITER The name of the property in the settings that contains the delimiter for the key and the value for each Map entry (the entries are delimited by COLLECTION_DELIMITER).

Constructor Summary

Constructors

Constructor and Description
CsvDataImporter(Class<E> entityClass) Creates a new instance of CsvDataImporter for a specific type.
CsvDataImporter(org.fastnate.generator.context.EntityClass<E> entityClass) Creates a new instance of CsvDataImporter for a specific type.
CsvDataImporter(org.fastnate.generator.context.EntityClass<E> entityClass, org.supercsv.prefs.CsvPreference csvSettings) Creates a new instance of CsvDataImporter.
CsvDataImporter(org.fastnate.generator.context.EntityClass<E> entityClass, org.supercsv.prefs.CsvPreference csvSettings, org.fastnate.data.EntityRegistration entityRegistration) Creates a new instance of CsvDataImporter.
CsvDataImporter(org.fastnate.generator.context.EntityClass<E> entityClass, org.fastnate.data.EntityRegistration entityRegistration) Creates a new instance of CsvDataImporter for a specific type.

Method Summary

Modifier and Type	Method and Description
void	addColumnMapping(String column, BiConsumer<E,String> propertyMapping) Defines the mapping from a column name to a property.
<T> void	addColumnMapping(String column, Class<T> propertyClass, org.fastnate.data.properties.PropertyConverter<T> converter, BiConsumer<E,T> propertyMapping) Defines the mapping from a column name to a property with an intermediate converter.
<T> void	addColumnMapping(String column, Function<String,T> converter, BiConsumer<E,T> propertyMapping) Defines the mapping from a column name to a property with an intermediate converter.
void	addDefaultColumnMapping(String columnName) Defines the mapping from a column name to a property by using the one from buildMapping(Property).
void	addDefaultColumnMapping(String columnName, String propertyName) Defines the mapping from a column name to a property by using the one from buildMapping(Property).
void	addIgnoredColumn(String column) Ignores a column during import.
void	addPostProcessor(Consumer<E> postProcessor) Adds a consumer that is called for each new entity.
protected boolean	applyColumn(E entity, String column, String value) Sets a property for an entity from a column value.
protected <T,V> BiConsumer<T,String>	buildMapping(org.fastnate.generator.context.Property<? super T,V> property) Builds the mapping for the given property.
protected List<? extends E>	createEntities(DataRow row) Builds one or more entities from the given row.
protected E	createEntity() Creates a new empty entity for the current converter.
protected E	createEntity(DataRow row) Builds one entity from the given row.
String	getCollectionDelimiter() The pattern to split Collection elements or Map entries, if mapProperties() or addDefaultColumnMapping(String) is used.
org.supercsv.prefs.CsvPreference	getCsvSettings() The settings to use when importing a file.
protected Charset	getDefaultEncoding() Defines the default encoding for CSV files, if it can't be determined from the BOM.
org.fastnate.generator.context.EntityClass<E>	getEntityClass() The description of the type of the created entities.
org.fastnate.data.EntityRegistration	getEntityRegistration()
String	getMapDelimiter() The pattern to split key and value of each entry in a Map, if mapProperties() or addDefaultColumnMapping(String) is used.
List<E>	importFile(org.fastnate.data.files.DataFile file) Reads entities from the given file.
boolean	isIgnoredColumn(String column) Indicates that the given column is ignored during import.
boolean	isIgnoreMissingColumns() Indicates to ignore any column that is not found in the CSV file.
boolean	isIgnoreUnknownColumns() Indicates to ignore any column that can't be mapped to a property.
void	mapProperties() Maps the name of each property of the entity class to the CSV columns.
void	mapTableColumns() Maps the database columns of all properties that are part of the entity table to CSV columns.
protected org.supercsv.io.CsvListReader	openCsvListReader(org.fastnate.data.files.DataFile importFile) Opens a CSV file.
BiConsumer<E,String>	removeColumnMapping(String column) Removes the mapping of the given column.
void	removeIgnoredColumn(String column) Removes an ignored column.
void	setCollectionDelimiter(String collectionDelimiter) The pattern to split Collection elements or Map entries, if mapProperties() or addDefaultColumnMapping(String) is used.
void	setCsvSettings(org.supercsv.prefs.CsvPreference csvSettings) The settings to use when importing a file.
void	setIgnoreMissingColumns(boolean ignoreMissingColumns) Indicates to ignore any column that is not found in the CSV file.
void	setIgnoreUnknownColumns(boolean ignoreUnknownColumns) Indicates to ignore any column that can't be mapped to a property.
void	setMapDelimiter(String mapDelimiter) The pattern to split key and value of each entry in a Map, if mapProperties() or addDefaultColumnMapping(String) is used.

Methods inherited from class org.fastnate.data.properties.PropertyDataImporter

addConverter, findConverter

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

COLUMN_DELIMITER

public static final String COLUMN_DELIMITER

The name of the property in the settings that contains the column delimiter. The default value for this property is ','.

See Also:

Constant Field Values

LINE_DELIMITER

public static final String LINE_DELIMITER

The name of the property in the settings that contains the line delimiter. The default value for this property is '\n'.

See Also:

Constant Field Values

COLLECTION_DELIMITER

public static final String COLLECTION_DELIMITER

The name of the property in the settings that contains the delimiter for elements of Collections respective Maps. The default value for this property is ','.

See Also:

Constant Field Values

MAP_DELIMITER

public static final String MAP_DELIMITER

The name of the property in the settings that contains the delimiter for the key and the value for each Map entry (the entries are delimited by COLLECTION_DELIMITER). The default value for this property is ':'.

See Also:

Constant Field Values

Constructor Detail

CsvDataImporter

public CsvDataImporter(Class<E> entityClass)

Creates a new instance of CsvDataImporter for a specific type. Will use its own context and EntityRegistration, so use only as standalone importer.

Parameters:

entityClass - the type of the created entities

CsvDataImporter

public CsvDataImporter(org.fastnate.generator.context.EntityClass<E> entityClass)

Creates a new instance of CsvDataImporter for a specific type. Will use its own EntityRegistration, so use only, if you don't need to exchange entity references between different importers. Will use the default CSV settings, by taking the properties from the context of the entity class into account.

Parameters:

entityClass - the description of the type of the created entities.

CsvDataImporter

public CsvDataImporter(org.fastnate.generator.context.EntityClass<E> entityClass,
                       org.supercsv.prefs.CsvPreference csvSettings)

Creates a new instance of CsvDataImporter. Will use its own EntityRegistration, so use only, if you don't need to exchange entity references between different importers.

Parameters:

entityClass - the description of the type of the created entities.

csvSettings - the settings to use

CsvDataImporter

public CsvDataImporter(org.fastnate.generator.context.EntityClass<E> entityClass,
                       org.supercsv.prefs.CsvPreference csvSettings,
                       org.fastnate.data.EntityRegistration entityRegistration)

Creates a new instance of CsvDataImporter.

Parameters:

entityClass - the description of the type of the created entities.

csvSettings - the settings to use

entityRegistration - used to store unique keys for entities

CsvDataImporter

public CsvDataImporter(org.fastnate.generator.context.EntityClass<E> entityClass,
                       org.fastnate.data.EntityRegistration entityRegistration)

Creates a new instance of CsvDataImporter for a specific type.

Parameters:

entityClass - the description of the type of the created entities.

entityRegistration - used to store unique keys for entities

Method Detail

addColumnMapping

public void addColumnMapping(String column,
                             BiConsumer<E,String> propertyMapping)

Defines the mapping from a column name to a property.

Parameters:

column - the name of the column

propertyMapping - converts and sets the value for this column

addColumnMapping

public <T> void addColumnMapping(String column,
                                 Class<T> propertyClass,
                                 org.fastnate.data.properties.PropertyConverter<T> converter,
                                 BiConsumer<E,T> propertyMapping)

Defines the mapping from a column name to a property with an intermediate converter.

Type Parameters:

T - the type of the property

Parameters:

column - the name of the column

propertyClass - the type of the property

converter - converts the content of the column to the property

propertyMapping - sets the value for this column

addColumnMapping

public <T> void addColumnMapping(String column,
                                 Function<String,T> converter,
                                 BiConsumer<E,T> propertyMapping)

Defines the mapping from a column name to a property with an intermediate converter.

Type Parameters:

T - the type of the property

Parameters:

column - the name of the column

converter - converts the content of the column to the property

propertyMapping - sets the value for this column

addDefaultColumnMapping

public void addDefaultColumnMapping(String columnName)

Defines the mapping from a column name to a property by using the one from buildMapping(Property).

Parameters:

columnName - the name of the column and property

addDefaultColumnMapping

public void addDefaultColumnMapping(String columnName,
                                    String propertyName)

Defines the mapping from a column name to a property by using the one from buildMapping(Property).

Parameters:

columnName - the name of the column and property

propertyName - the name of the property

addIgnoredColumn

public void addIgnoredColumn(String column)

Ignores a column during import. Ignored columns only need to be specified if isIgnoreUnknownColumns() is set to false.

Parameters:

column - the name of the column that is ignored

addPostProcessor

public void addPostProcessor(Consumer<E> postProcessor)

Adds a consumer that is called for each new entity.

Parameters:

postProcessor - the function to call for each entity

applyColumn

protected boolean applyColumn(E entity,
                              String column,
                              String value)

Sets a property for an entity from a column value.

Parameters:

entity - the entity to modify

column - the name of the current column (in lower case, as the mapping is stored in lower case, too)

value - the value of the column

Returns:

true if an appropriate property was found, false if a matching property was not found and #isIgnoreUnknownColumns() is true

Throws:

IllegalArgumentException - if a matching property was not found or it was not convertable

buildMapping

protected <T,V> BiConsumer<T,String> buildMapping(org.fastnate.generator.context.Property<? super T,V> property)

Builds the mapping for the given property.

Type Parameters:

V - the type of the property

Parameters:

property - the property to map

Returns:

the mapper of the property or null if no specific mapping can be created and the property should be ignored

createEntities

protected List<? extends E> createEntities(DataRow row)

Builds one or more entities from the given row.

Parameters:

row - contains the current row data

Returns:

the list of entities from that row

createEntity

protected E createEntity()

Creates a new empty entity for the current converter. Used by the default implementation of createEntities(DataRow).

Returns:

the new entity

createEntity

protected E createEntity(DataRow row)

Builds one entity from the given row.

Parameters:

row - contains the current row data

Returns:

the created and filled entity

getDefaultEncoding

protected Charset getDefaultEncoding()

Defines the default encoding for CSV files, if it can't be determined from the BOM.

Returns:

the default encoding

importFile

public List<E> importFile(org.fastnate.data.files.DataFile file)
                   throws IOException,
                          org.fastnate.data.DataImportException

Reads entities from the given file.

Parameters:

file - the file to import

Returns:

all found entities

Throws:

IOException - if the file was not accessible

org.fastnate.data.DataImportException - if the file contents was invalid

isIgnoredColumn

public boolean isIgnoredColumn(String column)

Indicates that the given column is ignored during import.

Parameters:

column - the name of the column

Returns:

true if the column is not imported

mapProperties

public void mapProperties()

Maps the name of each property of the entity class to the CSV columns. Useful if the CSV file is a property export.

mapTableColumns

public void mapTableColumns()

Maps the database columns of all properties that are part of the entity table to CSV columns. Useful if the CSV file is a database export.

openCsvListReader

protected org.supercsv.io.CsvListReader openCsvListReader(org.fastnate.data.files.DataFile importFile)
                                                   throws IOException

Opens a CSV file. If the given file ends with "gz", then the file is decompressed using a GZIPInputStream.

Parameters:

importFile - the CSV file

Returns:

the reader used to read the CSV file

Throws:

IOException - if the file was not accessible

removeColumnMapping

public BiConsumer<E,String> removeColumnMapping(String column)

Removes the mapping of the given column.

Parameters:

column - the column to remove

Returns:

the previous mapping or null if none was registered

removeIgnoredColumn

public void removeIgnoredColumn(String column)

Removes an ignored column.

Parameters:

column - the name of the column that is ignored

See Also:

addIgnoredColumn(String)

getEntityClass

public org.fastnate.generator.context.EntityClass<E> getEntityClass()

The description of the type of the created entities.

getEntityRegistration

public org.fastnate.data.EntityRegistration getEntityRegistration()

getCsvSettings

public org.supercsv.prefs.CsvPreference getCsvSettings()

The settings to use when importing a file.

getCollectionDelimiter

public String getCollectionDelimiter()

The pattern to split Collection elements or Map entries, if mapProperties() or addDefaultColumnMapping(String) is used. Defaults to the COLLECTION_DELIMITER property from the settings.

getMapDelimiter

public String getMapDelimiter()

The pattern to split key and value of each entry in a Map, if mapProperties() or addDefaultColumnMapping(String) is used. Defaults to the MAP_DELIMITER property from the settings.

isIgnoreMissingColumns

public boolean isIgnoreMissingColumns()

Indicates to ignore any column that is not found in the CSV file.

isIgnoreUnknownColumns

public boolean isIgnoreUnknownColumns()

Indicates to ignore any column that can't be mapped to a property.

setCsvSettings

public void setCsvSettings(org.supercsv.prefs.CsvPreference csvSettings)

The settings to use when importing a file.

setCollectionDelimiter

public void setCollectionDelimiter(String collectionDelimiter)

The pattern to split Collection elements or Map entries, if mapProperties() or addDefaultColumnMapping(String) is used. Defaults to the COLLECTION_DELIMITER property from the settings.

setMapDelimiter

public void setMapDelimiter(String mapDelimiter)

The pattern to split key and value of each entry in a Map, if mapProperties() or addDefaultColumnMapping(String) is used. Defaults to the MAP_DELIMITER property from the settings.

setIgnoreMissingColumns

public void setIgnoreMissingColumns(boolean ignoreMissingColumns)

Indicates to ignore any column that is not found in the CSV file.

setIgnoreUnknownColumns

public void setIgnoreUnknownColumns(boolean ignoreUnknownColumns)

Indicates to ignore any column that can't be mapped to a property.