Challenges
Index, fixed-position, LDIF, XML, web log, and many other file formats are not compatible with spreadsheets, database loaders, and other applications that require delimited data files in comma-separated values (CSV) format. Conversely, files exported in CSV format may need to be processed and re-cast into other file formats for mainframe, tape, data interchange, and reporting applications.
Beyond converting to and from CSV files, you may also need to integrate and transform the data within them, report on it or wrangle it for analytics and data visualization, and govern it for quality, security, and DevOps.
Solutions
IRI delivers CSV and other file handling functionality across its product line. Choose based on need, noting that the IRI Voracity platform includes everything below.
CSV File Conversion Only
Use IRI NextForm or IRI Voracity to convert files in CSV to other formats (e.g., LDIF, ODBC, XML, text, etc.), or from other formats to CSV. NextForm also supports data type conversion at the field level, and record layout remapping.
NextForm file definitions also work in the SortCL program in IRI CoSort and Voracity. That means you can re-use your metadata in fast data transformation and reporting jobs when upgrading.
CSV Data Transformation & Conversion
The SortCL program in CoSort or Voracity can:
- transform CSV files (i.e. sort, join, aggregate, cross-calculate, etc.)
- convert CSV files to other file or table formats and create CSV files from those formats, and convert data types as needed at the same time
- report from CSV file sources into detail, summary, or delta (CDC and SCD) targets ... or hand-off wrangled subsets to your BI tool, DB loader, or applications like SalesForce.
CSV Data Cleansing, Masking & Testing
The SortCL program in CoSort or Voracity can also improve data quality; i.e., it can scan, filter, validate, de-duplicate, cleanse, unify, and otherwise reconcile and redefine the values and layouts of CSV data.
Use IRI FieldShield or IRI DarkShield to protect fields in production CSV files with encryption, masking, etc. See this technical example!
Use IRI RowGen or Voracity if you need test data in CSV file formats. RowGen uses the same layout metadata as CoSort (SortCL) and NextForm, so you can easily move between test data generation and real data transformation.
All of the above (and many other!) use cases are performed through the same simple 4GL job-script metadata for file layout and field manipulation. Automatically create, graphically modify, and team-share these metadata assets in a free, familiar IDE built on Eclipse™ called IRI Workbench.
