Test Database Targets
IRI RowGen creates and loads structurally and referentially correct test data. It uses existing (or client-provided) DDLs for:
- DB2 UDB
- SQL Server
and packaged Apps (e.g., PeopleSoft, Informatica) using the above RDBs. It also supports the generation of synthetic subsets where you select (and save) the include or omit criteria for each target.
Use the 'Create DB Test Data' wizard in the IRI Workbench GUI for RowGen to do the following automatically:
table descriptions and integrity constraints are translated to test data generation definitions that reflect source structures, data types, and dependencies
pre-sorted flat files containing the test data are created for loading, while the definition scripts remain for easy modification and re-use
target tables are rapidly bulk loaded in the order necessary to maintain the same table relationships (referential integrity) that exist in production
Define table sizes, column separators, and index values. Apply field-level rules to one or more tables that:
- randomly select data from set files (custom, single-column, tab-delimited, or database-column-extracted)
- contain specified string data (literal values)
- draw data from any numeric range in provided (linear, normal, percent) or custom distributions
- display custom 'compound data values' created one component at a time (e.g., phone number format)
Apply these same generation rules to individual table, flat-file, and custom report targets. Use the RowGen GUI's 'Create Custom Test Data' wizard, or script the field definitions directly.
Whether created by the GUI or by hand, RowGen jobs are available for re-use, integration, and batch execution.
Test File Targets
RowGen gives you complete control over the generation of target test formats.
Use the 'Create Custom Test Data' wizard in the IRI Workbench GUI for RowGen, or work directly in the RowGen Control Language (RCL) job scripts that the wizard produces to combine data generation and formatting.
RCL scripts use the data and job definition syntax of IRI's CoSort SortCL program to describe the precise layout, size, and content of the target file(s).
Specify the size, position, separator, and data type of each field element. Define conditional selection criteria or ranges for set file values. See the discussion of field value customization below.
Create one or more test files at a time in these formats:
- CSV (or other-delimited)
- ELF (W3C Extended Log Format)
- LDIF (LDAP)
- Line Sequential
- Mainframe Variable Block
- Micro Focus Variable Length COBOL (Small/Large)
- Micro Focus I-SAM
- Micro Focus (Acucubol) Vision
- Record Sequential
- Text, fixed or delimited
- Variable Sequential
Generate as many records per file, and as many files, as you specify. Format each output in custom field, record, and report layouts. Specify transformations like sorting, aggregation, and cross-calculation. See Intelligent Column (Field) Values below.
*You can write and modify RCL job scripts with any text editor, in the RowGen GUI's syntax-aware script editor, or through GUI dialogs which interact with the script.
Test Report Targets
Create custom reports with test data inside. RowGen has the same reporting features for test data that SortCL has for real data:
Build test reports with custom header and footer records. Define detail and summary layouts. Specify environment variable and special constants.
RowGen is also ideal for franchising (preparing transformed) test data for specialized BI tools, and for providing better, faster, and bigger test data to applications like SAP and SalesForce.
Intelligent Column (Field) Values
Generate any number of rows with any number of columns, in fixed or delimited position. Choose the data type to generate for each field from these groups:
- ASCII & EBCDIC Characters
- Numeric, Whole, Currency, IP Address
- Alpha & EBCDIC Digits
- RM and MF COBOL Numerics
- Other Binary Numerics
- US, Europe, ISO, Japan Timestamps
- Unicode & Multi-byte
Define new data types with the compound (composite) data value wizard in the IRI Workbench GUI for RowGen. Specify, or randomly select, field values from one or more real set files, or literal values and ranges. Select a frequency distribution and preview the pattern in gnuplot.
When using set files, choose from several random selection methods (including inter-column permutation). This delivers both realism and de-identification when sets contain real names.
See field-level rules above for customizing and applying field-level intelligence to the test data. You can also combine random generation and selection to fine tune the test data. In fact, RowGen has many ways to enhance realism without using real data. See:
RowGen users will find many different points of integration into their current operations.
RowGen jobs easily fit into applications and batch streams. RowGen Control Language (.RCL) specifications are in text scripts called from the command line. The jobs are simple to edit and will run anywhere RowGen is licensed.
Integrating RowGen is also easier because of the self-documenting nature and cross-compatibility of its metadata. The data definition file (.DDF) format RowGen uses is common to all IRI software tools, and is supported throughout the IRI Workbench GUI, built on Eclipse™. Metadata discovery and conversion functions produce new DDFs that RowGen and other IRI tools can use*, and free version control plug-ins in Eclipse support for data lineage and metadata impact analysis.
The universality of metadata throughout the IRI product line also allows you to easily transition between test data generation and real data processing. Jobs in CoSort SortCL (data transformation and reporting), FieldShield (data masking), and NextForm use the same DDF.
RowGen DDF is also supported in AnalytixDS Mapping Manager and Meta Integration Technology Inc.'s Meta Integration Model Bridge (MIMB). Users of those packages can automatically convert the metadata in their DB, ETL, modeling, and related applications to RowGen DDF. This makes it easier to populate existing applications with test tables, files, and reports used in production, and with even broader value ranges for stress testing.
Use RowGen's built-in data transformation functions, such as expressions and aggregation, to test the functionality of your applications. For example, compare the results of your program's aggregation routine with the summary values that RowGen produced while creating the test data. See this case study as well.
Test targets can be files, named or unnamed pipes (stdout), one or more custom output procedures, and ODBC. Direct file output to database loaders (pre-sorted!) or your applications. Again, RowGen outputs can range from simple, one-field flat files, to EDW databases, to elaborately-formatted HTML reports complete with real-time system data.
*RowGen DDFs are also created by IRI's Fast Extract (FACT) tool, and by IRI-supplied metadata converters for COBOL copybooks, CSV file headers, SQL*Loader control files and ELF web log files.