Evaluating the performance of hardware platforms and software applications requires the use of realistic production data. Files and tables must be the right sizes, and contain the right data types, record formats, and counts.
Standard benchmarks promulgated by organizations like the Transaction Processing Performance Council may similarly require a wide range of pre-determined volumes and layouts of test data.
Large files and tables can take a long time to build and load without the right tools and techniques. Culling sample data from production can take time, and violate data privacy rules.
The IRI RowGen product -- or IRI Voracity data management platform which includes RowGen -- can synthesize safe, big test data files and database tables in any structured format. With RowGen, you can generate a complete and consistent battery of files for different software and hardware platforms, helping you find the best processing paradigms for your environment. In many cases, multiple output targets can be automatically generated and loaded simultaneously.
RowGen can create any number (and size) of files or relational tables with any amount of columns in any fixed or delimited position, with more than 100 different data types available. The test records and fields can be selected and transformed to emulate production data, and values can recur or change in successive runs through random seed management.
Where benchmarking of database prototypes, Data Vault architectures, or data warehouse ETL operations are required, RowGen will respect the layout and relationships of production tables from existing DDL. It will build a batch script you can run to rapidly create and populate test DB targets that are structurally and referentially correct.
Each field within the test targets contains either randomly generated data, or data randomly selected from set files or numeric ranges to be more realistic.