CoSort Captures Changed Data in Files and Writes Delta Reports.

Home » Solutions » Business Intelligence » CDC Delta Reporting

Capture Changed Data (Delta Reporting)

Faster Data Acquisition, Retention, Refreshment. Easier File Compares.

Challenges:

Operational data undergoes constant changes, additions, and deletions relative to historical data. Data warehouse maintenance and business groups need to compare files, as well as capture, isolate, and analyze changes in data.

Unfortunately, many data acquisition tools and methods used for comparing files and processing changed data are cumbersome, error-prone, and limited to one database's sources or performance. Regular reads and updates of the entire database is an impractical alternative to refresh the warehouse, especially as data volumes grow.

In other words, the feasibility of doing in-database work for deltas shrink as table sizes grow.

Solutions:

Trap (capture) your database and legacy data changes off-line, in faster, more functional CoSort Sort Control Language (SortCL) job scripts that compare files with integrated join and reporting logic. This removes a major workload from the DBMS (which relies on triggers to update CDC tables).

Start with a SELECT query in the compatible Fast Extract (FACT) tool for Oracle, for example, and dump only those rows generated after a certain timestamp to a flat file. Alternatively, perform an unqualified extract to save unloading time.

The CoSort SortCL tool can then conditionally join this file to one or more other archive files, and produce multiple output targets. You can capture the inserts, updates or deltes in custom reports, feed a (pre-sorted) database loader, and/or populate other applications with the deltas -- all at the same time.

You can also count the number of new, modified, or missing records in your reports, or consult SortCL's runtime statistics (which count inner and outer matches at each join). Label, log, and analyze changes in the transaction data to spot red flags and assess trends forensically.

SortCL can, at the same time, also:

• re-format
• cross-calculate
• rank

the changed data, and use its field-level commands or functions to help you:

• transform
• cleanse
• protect
• present

the changed data.

Thus, you can cost-effectively capture changes in your data, and leverage simultaneous transformation and reporting opportunities, during high-volume file compare and data refresh operations.

IRI, The CoSort Company