PowerCenter transforms of very large data volumes require partitioning and can run slower than desired, even after consulting and tuning. Bottlenecks may occur during large sort, join, aggregation, load, or unload operations. Informatica's initial "pushdown optimization" options shift the burden into an already-busy database (Oracle) or very expensive/complex platform (Teradata).
Another serious need is the protection of sensitive production data moving through Informatica data warehouse, data mart, or test operations. You may need to apply role-based data protections or generate large volumes of realistic, referentially correct test data to prototype applications and populate specific targets.
To speed transforms, reports, and field-level protections in general, consider the use of CoSort SortCL programs alongside your PowerCenter or PowerMart operations.
The American Stock Exchange uses CoSort as a "pushdown optimization" solution to dramatically transformation performance. On a 4-CPU IBM p640 that took PowerCenter 20m35s to sort, SortCL did the same job in 1m:19s. This represented only a nominal incremental software investment, and did not tax their database.
Run large sorts, joins, aggregations, and loads in the file system, where it's much faster. Plus, convert file and data types, mask PII, and generate custom reports -- all at the same time (in the same job script and I/O pass).
Click here to learn more.
You may not want to use the Dynamic Data Masking software that Informatica acquired. IRI's data-centric security product, FieldShield has more methods for protecting fields in structured datasets, and uses simple, portable, Eclipse-supported job scripts.
Your business rules dictate the feature you choose to apply to each column: format-preserving AES-256, Open SSL and GPG encryption, lookup-value substitution (pseudonymization), character masking, custom expression logic, user field functions, and more.
Do you need test data for Informatica ETL prototyping? Use IRI RowGen to generate it rapidly and affordably. With RowGen, you can build realistic, referentially correct test data to populate target tables, data marts, flat files, and production reports, while leveraging your database data model (.DDL) files and Informatica metadata.
You can now convert ETL jobs in Informatica to Voracity automatically through AnalytiX DS Code-Automation Frameworks (CATfx) technology. Both Informatica and Voracity metadata are modeled in AnalytiX DS Mapping Manager so you can move projects between the two platforms with ease. Voracity is both several times faster and less expensive than Informatica PowerCenter, and with this proven technology, it is finally possible to switch.