Informatica Pushdown Optimization with CoSort

by Jason Koivu on January 7, 2013

Big data transformation and integration can happen outside the database in an extract, transform, load (ETL) operation, or inside the database in an ELT operation: http://www.iri.com/blog/data-transformation2/etl-vs-elt-we-posit-you-judge/ One example of an ELT operation would be Informatica’s Pushdown Optimization option to allow users to perform transformations in a relational database like Oracle or Teradata. According to Informatica, this [...]

{ 2 comments }

CoSort Acceleration for Cognos Business Intelligence

by Jason Koivu on February 27, 2013

There are a number of business intelligence tools available today than can transform raw data into meaningful information. Because this process can be complex and involve large volumes of data however, it makes sense to use the right technologies at each step in the process … tools and techniques that combine well to deliver the [...]

{ 2 comments }

Weighted Distribution of Test Data Values in RowGen

February 12, 2013 Data Transformation

Realistic test data has a number of advantages over real data for anyone creating or changing a database, prototyping ETL operations, or testing applications. First, synthetic data do not expose personally identifying information (PII) like credit card, social security numbers, birth dates, etc. Second, realistic test data shows how the system will behave with real [...]

Read the full article →

RowGen 3′s Custom Test Data Wizard

February 4, 2013 Test Data

A test data generator is an important part of the setup process when database architects are creating database and data warehouse operations, prototyping applications, benchmarking different platforms, and outsourcing work formats.  These architects should never use real production data in the testing process because that runs the risk of exposing personally identifiable information, proprietary information, [...]

Read the full article →

VSE2SCL Converts Mainframe Sort Parms to CoSort SortCL

January 24, 2013 Sort Migration

VSE, virtual storage extended, is an operating system for IBM mainframe computers and the scripting language, or job control language (JCL), instructs the system how to run batch jobs or start subsystems.  As data needs continue to grow, database architects look for solutions to their big data problems.  They are often looking for faster processing, [...]

Read the full article →

Select/Filter to Reduce Data Bulk

January 21, 2013 Big Data

One of the best ways to speed up big data processing operations is to not process so much data in the first place; i.e. to eliminate unnecessary data ahead of time. Data can be culled en masse by specifying the collection, processing, or output of only a certain number of rows, or more intelligently with [...]

Read the full article →

MVS2SCL Converts z/OS JCL Sort Parms to CoSort SortCL

January 21, 2013 Sort Migration

MVS, multiple virtual storage, is an operating system for IBM mainframe computers and the scripting language, or job control language (JCL), instructs the system how to run batch jobs or start subsystems.  The growth and development of technology continues to accelerate for all computers, including mainframes, and users need the fastest systems available that are [...]

Read the full article →

One Big Data Manipulator, Three Business Intelligence Paths

January 2, 2013 Big Data

As CoSort integrates and stages big data from a variety of sources, it plays a natural role in preparing (or franchising) data for business intelligence (BI). CoSort not only transforms data for loading data warehouse tables, it can report at the same time, or feed data in filtered, aggregated, sorted, and properly formatted subsets (like [...]

Read the full article →

RowGen v3 Automates Database Test Data Generation

December 19, 2012 Test Data

The value of good test data to DBAs is well known: “Testing of database-intensive applications has unique challenges that stem from hidden dependencies, subtle differences in data semantics, target database schemes, and implicit business rules. These challenges become even more difficult when the application involves integrated and heterogeneous databases or confidential data. Proper test data [...]

Read the full article →

ETL vs ELT: We Posit, You Judge

December 13, 2012 Big Data

Full disclosure: As this article is authored by an ETL-centric company with its strong suit in processing big data outside of databases, what follows will not seem objective to many. Nevertheless it is still meant to present food for thought, and opens the floor to comments and future discussions across the IRI blog site. Since [...]

Read the full article →

← Previous Entries