{"id":3598,"date":"2013-02-04T22:39:55","date_gmt":"2013-02-04T22:39:55","guid":{"rendered":"http:\/\/www.iri.com\/blog\/?p=3598"},"modified":"2021-09-07T09:08:58","modified_gmt":"2021-09-07T13:08:58","slug":"test-data-generator-custom-data-job-wizard-rowgen-v3","status":"publish","type":"post","link":"https:\/\/www.iri.com\/blog\/test-data\/test-data-generator-custom-data-job-wizard-rowgen-v3\/","title":{"rendered":"Introducing the RowGen Test File Wizard"},"content":{"rendered":"

A test data generator<\/a> is an important part of the setup process for DevOps and data architects prototyping database and data warehouse operations, testing applications, benchmarking different platforms, and outsourcing work formats. \u00a0These users should not rely on production data in those processes because that risks exposing personally identifiable information (PII), and\/or it may not reflect the characteristics of future production data.<\/p>\n

IRI RowGen<\/a>\u00a0allows these architects to synthesize new, and realistic\u00a0test data<\/a>\u00a0targets in flat file formats, and relational databases … in a single operation that supports referential integrity, applies\u00a0custom formats, and, is safe. The populated test\u00a0data reflects\u00a0the kind of\u00a0production values that users\u00a0would\u00a0encounter in normal database or ETL operations.<\/p>\n

RowGen users can leverage purpose-built job wizards in the IRI Workbench GUI<\/a>\u00a0(built on Eclipse\u2122) to simplify and streamline test data generation. One of the most useful wizards is the New Test Data Job<\/em> wizard, and is the subject of this article. If you’re interested in generating structurally and referentially correct test data for SQL database targets, see this article<\/a>.<\/p>\n

The file-centric test data wizard constructs individual RowGen jobs with\u00a0one or more custom data layouts, to produce one or more targets all in the same pass \u00a0(while\u00a0sorting on\u00a0key fields). Users can define their own field\/column layouts piece-by-piece, or they can incorporate existing metadata that already describes all or most of the test target data structures.<\/p>\n

RowGen and its wizards support several ways to make data more realistic<\/a>, including the random selection of values from\u00a0set<\/em> files (such as from an existing database column of values in a text file), or by using any of the available mini-wizards for creating fit-for-purpose set files on the fly.<\/p>\n

Set files can consist of any combination of literal values and\/or ranges and ensure that only unique values in a set file are selected from top to bottom, and support valid pairs<\/a> in the test data.<\/p>\n

File & Field Format Specification<\/strong><\/p>\n

In the RowGen test data generation wizard, the vehicle for specifying the file format can be defined in the source (and\/or target) specification dialog, where a menu of options is offered; e.g.,<\/p>\n

\"\"<\/a>\"\"
\n<\/a>The most basic form for values that the test data generator synthesizes consists of randomly generated characters that conform to the declared data type. Alternatively of course, are random selection from set files. Either way, this can be defined field-by-field, in the input \/ generation phase of a job; e.g,<\/p>\n

\"\"<\/a><\/p>\n

Here the user can see basic information about the custom layout, re-position or clear fields, and optionally declare a field separator and frame character. When an existing field is double-clicked, the Test Field Data Attributes<\/em> dialog opens (see below) where the user can specify attributes for that field, including: \u00a0name, position within the record, data type, min- and max-size, upper- vs. lower-case letters, left vs. right alignment, etc.<\/p>\n

The\u00a0dialog offers additional options for enhancing field-level realism such as value endianness, and the ability to specify a percentage of null values to be generated among the full set of values.<\/p>\n

\"\"<\/a><\/p>\n

And from within the Test Field Data Attributes<\/em> dialog, users can click ‘define’ to select an existing, or define a new, set file if they do not want random generation of values based solely on the declared data type.<\/p>\n

Additional Attributes<\/strong><\/p>\n

In the New Test Data Job wizard, there are several ways for the test data generator to add data realism to a field, including the following set file creation mini-wizards:<\/p>\n