Package org.fastnate.data.csv

Class CsvDataImporter<E>

java.lang.Object
org.fastnate.data.properties.PropertyDataImporter
org.fastnate.data.csv.CsvDataImporter<E>
Type Parameters:
E - The type of the generated objects
public class CsvDataImporter<E> extends org.fastnate.data.properties.PropertyDataImporter
An importer that reads entities of a specific class from a CSV file. One can either add his own column mapping or use the default properties mapping. Column names are handled case insensitive.
Author:
Tobias Liefke

  • Field Details

    • COLUMN_DELIMITER

      public static final String COLUMN_DELIMITER
      The name of the property in the settings that contains the column delimiter. The default value for this property is ','.
      See Also:

    • LINE_DELIMITER

      public static final String LINE_DELIMITER
      The name of the property in the settings that contains the line delimiter. The default value for this property is '\n'.
      See Also:

    • COLLECTION_DELIMITER

      public static final String COLLECTION_DELIMITER
      The name of the property in the settings that contains the delimiter for elements of Collections respective Maps. The default value for this property is ','.
      See Also:

    • MAP_DELIMITER

      public static final String MAP_DELIMITER
      The name of the property in the settings that contains the delimiter for the key and the value for each Map entry (the entries are delimited by COLLECTION_DELIMITER). The default value for this property is ':'.
      See Also:

  • Constructor Details

    • CsvDataImporter

      public CsvDataImporter(Class<E> entityClass)
      Creates a new instance of CsvDataImporter for a specific type. Will use its own context and EntityRegistration, so use only as standalone importer.
      Parameters:
      entityClass - the type of the created entities

    • CsvDataImporter

      public CsvDataImporter(org.fastnate.generator.context.EntityClass<E> entityClass)
      Creates a new instance of CsvDataImporter for a specific type. Will use its own EntityRegistration, so use only, if you don't need to exchange entity references between different importers. Will use the default CSV settings, by taking the properties from the context of the entity class into account.
      Parameters:
      entityClass - the description of the type of the created entities.

    • CsvDataImporter

      public CsvDataImporter(org.fastnate.generator.context.EntityClass<E> entityClass, org.supercsv.prefs.CsvPreference csvSettings)
      Creates a new instance of CsvDataImporter. Will use its own EntityRegistration, so use only, if you don't need to exchange entity references between different importers.
      Parameters:
      entityClass - the description of the type of the created entities.
      csvSettings - the settings to use

    • CsvDataImporter

      public CsvDataImporter(org.fastnate.generator.context.EntityClass<E> entityClass, org.supercsv.prefs.CsvPreference csvSettings, org.fastnate.data.EntityRegistration entityRegistration)
      Creates a new instance of CsvDataImporter.
      Parameters:
      entityClass - the description of the type of the created entities.
      csvSettings - the settings to use
      entityRegistration - used to store unique keys for entities

    • CsvDataImporter

      public CsvDataImporter(org.fastnate.generator.context.EntityClass<E> entityClass, org.fastnate.data.EntityRegistration entityRegistration)
      Creates a new instance of CsvDataImporter for a specific type.
      Parameters:
      entityClass - the description of the type of the created entities.
      entityRegistration - used to store unique keys for entities

  • Method Details

    • addColumnMapping

      public void addColumnMapping(String column, BiConsumer<E,String> propertyMapping)
      Defines the mapping from a column name to a property.
      Parameters:
      column - the name of the column
      propertyMapping - converts and sets the value for this column

    • addColumnMapping

      public <T> void addColumnMapping(String column, Class<T> propertyClass, org.fastnate.data.properties.PropertyConverter<T> converter, BiConsumer<E,T> propertyMapping)
      Defines the mapping from a column name to a property with an intermediate converter.
      Type Parameters:
      T - the type of the property
      Parameters:
      column - the name of the column
      propertyClass - the type of the property
      converter - converts the content of the column to the property
      propertyMapping - sets the value for this column

    • addColumnMapping

      public <T> void addColumnMapping(String column, Function<String,T> converter, BiConsumer<E,T> propertyMapping)
      Defines the mapping from a column name to a property with an intermediate converter.
      Type Parameters:
      T - the type of the property
      Parameters:
      column - the name of the column
      converter - converts the content of the column to the property
      propertyMapping - sets the value for this column

    • addDefaultColumnMapping

      public void addDefaultColumnMapping(String columnName)
      Defines the mapping from a column name to a property by using the one from buildMapping(Property).
      Parameters:
      columnName - the name of the column and property

    • addDefaultColumnMapping

      public void addDefaultColumnMapping(String columnName, String propertyName)
      Defines the mapping from a column name to a property by using the one from buildMapping(Property).
      Parameters:
      columnName - the name of the column and property
      propertyName - the name of the property

    • addIgnoredColumn

      public void addIgnoredColumn(String column)
      Ignores a column during import. Ignored columns only need to be specified if isIgnoreUnknownColumns() is set to false.
      Parameters:
      column - the name of the column that is ignored

    • addPostProcessor

      public void addPostProcessor(Consumer<E> postProcessor)
      Adds a consumer that is called for each new entity.
      Parameters:
      postProcessor - the function to call for each entity

    • applyColumn

      protected boolean applyColumn(E entity, String column, String value)
      Sets a property for an entity from a column value.
      Parameters:
      entity - the entity to modify
      column - the name of the current column (in lower case, as the mapping is stored in lower case, too)
      value - the value of the column
      Returns:
      true if an appropriate property was found, false if a matching property was not found and #isIgnoreUnknownColumns() is true
      Throws:
      IllegalArgumentException - if a matching property was not found or it was not convertable

    • buildMapping

      protected <T, V> BiConsumer<T,String> buildMapping(org.fastnate.generator.context.Property<? super T,V> property)
      Builds the mapping for the given property.
      Type Parameters:
      V - the type of the property
      Parameters:
      property - the property to map
      Returns:
      the mapper of the property or null if no specific mapping can be created and the property should be ignored

    • createEntities

      protected List<? extends E> createEntities(DataRow row)
      Builds one or more entities from the given row.
      Parameters:
      row - contains the current row data
      Returns:
      the list of entities from that row

    • createEntity

      protected E createEntity()
      Creates a new empty entity for the current converter. Used by the default implementation of createEntities(DataRow).
      Returns:
      the new entity

    • createEntity

      protected E createEntity(DataRow row)
      Builds one entity from the given row.
      Parameters:
      row - contains the current row data
      Returns:
      the created and filled entity

    • getDefaultEncoding

      protected Charset getDefaultEncoding()
      Defines the default encoding for CSV files, if it can't be determined from the BOM.
      Returns:
      the default encoding

    • importFile

      public List<E> importFile(org.fastnate.data.files.DataFile file) throws IOException, org.fastnate.data.DataImportException
      Reads entities from the given file.
      Parameters:
      file - the file to import
      Returns:
      all found entities
      Throws:
      IOException - if the file was not accessible
      org.fastnate.data.DataImportException - if the file contents was invalid

    • isIgnoredColumn

      public boolean isIgnoredColumn(String column)
      Indicates that the given column is ignored during import.
      Parameters:
      column - the name of the column
      Returns:
      true if the column is not imported

    • mapProperties

      public void mapProperties()
      Maps the name of each property of the entity class to the CSV columns. Useful if the CSV file is a property export.

    • mapTableColumns

      public void mapTableColumns()
      Maps the database columns of all properties that are part of the entity table to CSV columns. Useful if the CSV file is a database export.

    • openCsvListReader

      protected org.supercsv.io.CsvListReader openCsvListReader(org.fastnate.data.files.DataFile importFile) throws IOException
      Opens a CSV file. If the given file ends with "gz", then the file is decompressed using a GZIPInputStream.
      Parameters:
      importFile - the CSV file
      Returns:
      the reader used to read the CSV file
      Throws:
      IOException - if the file was not accessible

    • removeColumnMapping

      public BiConsumer<E,String> removeColumnMapping(String column)
      Removes the mapping of the given column.
      Parameters:
      column - the column to remove
      Returns:
      the previous mapping or null if none was registered

    • removeIgnoredColumn

      public void removeIgnoredColumn(String column)
      Removes an ignored column.
      Parameters:
      column - the name of the column that is ignored
      See Also:

    • getEntityClass

      public org.fastnate.generator.context.EntityClass<E> getEntityClass()
      The description of the type of the created entities.

    • getEntityRegistration

      public org.fastnate.data.EntityRegistration getEntityRegistration()

    • getCsvSettings

      public org.supercsv.prefs.CsvPreference getCsvSettings()
      The settings to use when importing a file.

    • getCollectionDelimiter

      public String getCollectionDelimiter()
      The pattern to split Collection elements or Map entries, if mapProperties() or addDefaultColumnMapping(String) is used. Defaults to the COLLECTION_DELIMITER property from the settings.

    • getMapDelimiter

      public String getMapDelimiter()
      The pattern to split key and value of each entry in a Map, if mapProperties() or addDefaultColumnMapping(String) is used. Defaults to the MAP_DELIMITER property from the settings.

    • isIgnoreMissingColumns

      public boolean isIgnoreMissingColumns()
      Indicates to ignore any column that is not found in the CSV file.

    • isIgnoreUnknownColumns

      public boolean isIgnoreUnknownColumns()
      Indicates to ignore any column that can't be mapped to a property.

    • setCsvSettings

      public void setCsvSettings(org.supercsv.prefs.CsvPreference csvSettings)
      The settings to use when importing a file.

    • setCollectionDelimiter

      public void setCollectionDelimiter(String collectionDelimiter)
      The pattern to split Collection elements or Map entries, if mapProperties() or addDefaultColumnMapping(String) is used. Defaults to the COLLECTION_DELIMITER property from the settings.

    • setMapDelimiter

      public void setMapDelimiter(String mapDelimiter)
      The pattern to split key and value of each entry in a Map, if mapProperties() or addDefaultColumnMapping(String) is used. Defaults to the MAP_DELIMITER property from the settings.

    • setIgnoreMissingColumns

      public void setIgnoreMissingColumns(boolean ignoreMissingColumns)
      Indicates to ignore any column that is not found in the CSV file.

    • setIgnoreUnknownColumns

      public void setIgnoreUnknownColumns(boolean ignoreUnknownColumns)
      Indicates to ignore any column that can't be mapped to a property.