Request Info
Overview BI Analyst DBA CIO/IT Manager ETL/Solution Architect Compliance Officer Programmer & ISV

Big data transformations, conversions, protections, and reports -- all in one place, product, and pass.

Architectural Challenges
IRI Solutions

Structured big data integration, staging, and conversion tasks take too long, or rely on complex transform designs in:

  • Costly ETL or sort tools
  • Shell or 3GL programs
  • SQL joins or procedures
  • Data quality or BI apps

IRI Voracity, the total data management platform built on CoSort, Hadoop, and Eclipse.

The metadata definition language and default processing engine for data manipulation in Voracity and all IRI software is the Sort Control Language (SortCL) program first designed for IRI CoSort users. SortCL uses an open fourth-generation language (4GL) to define, combine, and optimally perform all transformations and mappings in the same job script and I/O pass.

SortCLjobs are built automatically in the free Eclipse GUI that also allows you ro graphically modify and visually stage them in an ETL palette, schedule and run them in or outside the GUI, and team-share them in repositories like EGit.Voracity big data users can also seamlessly drive SortCL transformation jobs in Hadoop MapReduce, Spark, Storm and Tez in the IRI Workbench GUI, built on Eclipse™.

IRI's standalone NextForm product is a SortCL spin-off, purpose-built for converting: data types, endianness, record layouts, file formats, and databases. The IRI Workbench GUI, built on Eclipse™, supports Voracity ETL and standalone SortCL and NextForm operations.

Changed data capture (CDC) and other reports run in separate steps or products. In high volume, they are too slow.

Use the multi-source bulk/batch change data capture wizard in Voracity and schedule its jobs in near real-time with a high-resolution setting in the task scheduler. 

You can also build SortCL jobs that transform and remap data within custom detail and summary reports. The reports can be used for:

  • billing and customer data integration
  • clickstream and other analytics (BI)
  • flagging and operating on inserts, updates, deletes and no changes
  • slowly changing dimensions (SCD)
  • using lookups and 'fuzzy logic' criteria for spotting trends

SortCL also feeds BIRT reports in memory within the Eclipse GUI, IRI Workbench.

To comply with privacy regulations, you need data security, but your development and outsourced processes need access to realistic data. Most encryption methods only protect data in one database, or far more than just the fields you need to protect. Most data masking programs do too little.
Voracity or SortCL jobs that transform, migrate, and report on data (above) can also simultaneously protect data at risk. Apply field-level encryption, canned and custom data masking, de-ID, hashing, randomization, sub-string, expression, pseudonymization, shifting and custom functions. Integrate key management, tokenization and audit.
IRI FieldShield is purpose-built software for content-aware data loss prevention and data privacy law (PCI, HIPAA, etc.) compliance. The IRI Workbench GU supports both users the same way.
Test data needs to look real, but using production data for testing is unsafe, unavailable, or otherwise unacceptable. Other tools require sub-setting and masking, and lack big data speed and versatility.

Voracity or SortCL jobs that transform, migrate, report on, and secure data (above) can also be used to generate safe, intelligent test data, without needing real databases or files. The same 4GL behind SortCL data definitions and manipulations is behind the IRI RowGen test data package (which is included in Voracity, and also provides test data for ETL task previews!).

Switch easily between test data generation and real data processing and presentation. In the IRI Workbench (Eclipse GUI), use SortCL or RowGen to create structurally and referentially correct test data for database tables, plus custom file and report formats, with a mix of randomly selected/generated data.

Other features include:

  • all-pairs and joined (valid) pairs testing
  • canned and custom value occurrence (frequency) distribution
  • composite (or master) data type construction
  • automated parsing, generation, and DB-population jobs that run on any platform
Master Data Management (MDM) hubs are too costly and complex.

Use the consolidation or registry-style MDM wizard in IRI Voracity to fine and reconcile disparate values and save (or propagate) the unified master values.

You can also simplify reference data and metadata, master data and master metadata through: the shared syntax of SortCL and its data definition file (DDF) format, the SortCL /TEMPLATE (composite value) function, and future graphical IRI Workbench hub for:

  • reference data => local and remote files, DB tables, SET files
  • reference metadata => DDF files
  • master data => clean, unify, and create master data values
  • master metadata => master data DDF and/or fragment files
  • transformation, protection, and test data generation rules
Very large database (VLDB) utility operations like unload, load, migration, query, and reorgs are too slow.
The IRI FACT (Fast Extract) tool for Oracle, DB2, Sybase, SQL Server, MySQL, Tibero, and Altibase, unloads tables to flat files in parallel. FACT also creates SortCL DDF and loader control file metadata for optional ETL or offline reorg (unload/sort/reload) operations. CoSort pre-sorts speed loads. This facilitates the maintenance of huge fact tables in optimal query order, resulting in quicker lookup and response performance.
A legacy sort tool is too slow, expensive, or inflexible.
IRI CoSort has plug 'n play sort replacements, sort parm conversion routines, and/or business incentives for leaving your current sort.

Request More Information

* indicates a required field.
IRI does NOT share your information.