Using the Metadata Discovery Wizard
IRI’s data management tools share a familiar and self-documenting metadata language called SortCL. All these tools — including CoSort, FieldShield, NextForm, and RowGen — require data definition file (DDF) layouts with /FIELD specifications for each data source so you can map your data and manage your metadata.
These DDF files can define both sources and targets, and are completely re-usable with the same product (e.g. multiple CoSort SortCL jobs), or in other IRI applications (e.g. RowGen, for creating test data in production formats).
The common graphical IDE for IRI software, IRI Workbench, helps you produce data definition file (DDF) metadata repositories from both third-party metadata, and from automated self-discovery. DDFs are also text scripts you can easily create by hand and modify in any editor.
Specifically, the “Discover Metadata” wizard is an available profiling tool in Workbench that helps you view the contents of your files and databases, confirm their layouts, and create the DDFs that describe them. Other available profiling tools in the Workbench allow you to produce statistical information, forensic metadata, and new extracts with pattern-searched values from databases and unstructured document (dark data) sources.
Launch the Discover Metadata Wizard and specify whether the source you’re trying to define is a flat file (fixed or delimited, including character-separated fields, Excel sheets and ASN.1 CDRs), or an ODBC-connected database table. Browsing connects to and opens the source directly in the data preview window, allowing you to view the file or table and accept, modify, or create SortCL fields.
In the cases of tables or delimited files, field layouts will be automatically defined, speeding DDF creation.
The field start and ending positions, data types, file formats, and record lengths for fixed files can be hand-defined with an ergonomic offset slider.
You can modify file, but not table, layouts, including re-specifying field names and data types. Contact firstname.lastname@example.org for help using this wizard.
[…] first used IRI Workbench to open the LinkSpeedQuery.txt file. We used the new Sort Job Wizard to discover and define the metadata for that file, define a new sort key, and map the data to a new […]