IRI Blog Articles

Diving Deeper into Data Management

 

 

Post image for 🎥 Single-Pass Sort, Join and Aggregation Transformations

🎥 Single-Pass Sort, Join and Aggregation Transformations

by Susan Gegner

CoSort can sort, join, and aggregate in the same job. In fact, CoSort’s SortCL Program supports the consolidation of these three major data warehouse transformation functions in the same job script and I-O pass, along with reporting, protection, and many other data manipulations that would otherwise require separate steps, and possibly separate products. This kind of task consolidation is not only more efficient at runtime, it saves disk space, simplifies job design, and minimizes code maintenance.

In this demonstration, we are completing a sort, join, and aggregation in the same job script and in one I-O pass. We are using this demonstration to show how a user might pull the components together for a billing statement by using the combined work of the sort, join, and aggregation processes. We take transaction files from a store, join those files with customer information records and then aggregate the sums of transactions for each customer so that the output records could be used for input into billing statements.

Print Friendly

{ 0 comments… add one now }

Leave a Comment

Previous post:

Next post: