Chat
Request Info
Download
Download
Overview DataStage ETI*Solution Informatica OWB & ODI Pentaho Talend Others

Challenges


Most ETL and ELT tools, along with the database engines they use, cannot transform big data efficiently without:

  • an expensive parallel processing edition
  • taking database or system resources from others
  • a complex, hard-to-maintain Hadoop environment
  • a 6/7-figure hardware appliance or server upgrades
  • pushing the problem to an even more expensive database

It is the large sort, join, and aggregation jobs that can take too long. Subsequent tasks like loading, analytics, or BI displays also suffer. And these E, T, and L steps are typically performed in separate steps, I/O passes, products, or constantly changing cloud configurations.

Solutions


If you use other ETL or ELT software, IRI software can speed their major processes on Unix, Linux, or Windows:

Operation Solution Supports Advantages
Extraction
Fast Extract (FACT)
Oracle, DB2, Sybase, MySQL, SQL Server, Altibase, Greenplum, Teradata, Tibero
Native DB drivers, parallel unloads, portable flat file output data, simple job scripts, easy to invoke
Transformation
CoSort's SortCL Program
DB-agnostic, all flat files, Informatica, DataStage, and any 'system command' call
Multi-threading, task and I/O consolidation, local and remote execution in LUW file systems or Hadoop, plus automatic metadata and job creation
Loading
Any CoSort sort engine
All RDBMS loaders, ODBC, and JDBC
Stream pre-CoSorted data after E or T jobs to shave up to 90% off load time
ETL & ELT
The IRI Voracity platform
More than 125 legacy and modern small and big data sources and targets
All of the above in a total data management environment combining data discovery, integration, migration, governance, and analytics in Eclipse

Design your jobs in the free IRI Workbench GUI for all the IRI software utilities (or Voracity platform). With CoSort, for example, you can transform big data faster, alongside incumbent platforms using flat files in the local file system or HDFS cluster. If you do not use those tools, consider what Voracity or CoSort's SortCL program can do instead, especially in line with IRI's Fast Extract (FACT) utility and your database loader (with direct, pre-sorted files).

See how you can optimize sorts (and downstream joins, aggregates, and loads) for Informatica or DataStage. CoSort is also called at the API or script level in Kalido, ETI, Software AG Natural, SAS, and TeraStream.

Use the metadata and worfklows you have, and just call IRI software from your tool to speed, and/or combine unloads, data transformations and operations like:

  • Sorts
  • Joins
  • Aggregates
  • Lookups
  • Perl-Compatible Regular Expressions
  • Data-type and file-format conversions
  • Field/column encryption and masking
  • Detail, delta (CDC) and summary reports
  • Row-column pivoting
  • Slowly changing dimension reports
  • Test data generation

You can also call IRI jobs from the shell (as a batch executable or ETL tool command),, via API or Eclipse GUI -- and flow data back and forth through files, pipes, or procedures as needed. In the IRI Workbench GUI environment, you can build the invidiual job specs or complete ELT or ELT flows connecting CoSort (and FACT) to your sources and targets.

AnaytiX DS and Meta Integration Model Bridge (MIMB) software or services can also convert metadata defined in popular ETL tools (like Informatica's .xml and DataStage .dsx repositories) into equivalent SortCL data (and/or job) specs. This automatic metadata replication preserves existing investments, facilitates SortCL job creation, and reduces migration time.


Request More Information

* indicates a required field.
IRI does NOT share your information.