{"id":9539,"date":"2010-01-01T00:00:42","date_gmt":"2010-01-01T05:00:42","guid":{"rendered":"http:\/\/www.iri.com\/blog\/?p=9539"},"modified":"2017-11-06T15:10:29","modified_gmt":"2017-11-06T20:10:29","slug":"single-pass-sort-join-and-aggregation-transformations","status":"publish","type":"post","link":"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/","title":{"rendered":"Single-Pass Sort, Join and Aggregation Transformations [video]"},"content":{"rendered":"<p>CoSort can sort, join, and aggregate in the same job. In fact, CoSort&#8217;s SortCL Program supports the consolidation of these three major data warehouse transformation functions in the same job script and I-O pass, along with reporting, protection, and many other data manipulations that would otherwise require separate steps, and possibly separate products. This kind of task consolidation is not only more efficient at runtime, it saves disk space, simplifies job design, and minimizes code maintenance.<\/p>\n<p>In this demonstration, we are completing a sort, join, and aggregation in the same job script and in one I-O pass. We are using this demonstration to show how a user might pull the components together for a billing statement by using the combined work of the sort, join, and aggregation processes. We take transaction files from a store, join those files with customer information records and then aggregate the sums of transactions for each customer so that the output records could be used for input into billing statements.<\/p>\n<p style=\"text-align: center;\"><iframe loading=\"lazy\" width=\"1140\" height=\"641\" src=\"https:\/\/www.youtube.com\/embed\/esI9oZQwGTM?feature=oembed\" frameborder=\"0\" gesture=\"media\" allowfullscreen><\/iframe><\/p>\n","protected":false},"excerpt":{"rendered":"<p>CoSort can sort, join, and aggregate in the same job. In fact, CoSort&#8217;s SortCL Program supports the consolidation of these three major data warehouse transformation functions in the same job script and I-O pass, along with reporting, protection, and many other data manipulations that would otherwise require separate steps, and possibly separate products. This kind<\/p>\n<div><a class=\"btn-filled btn\" href=\"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/\" title=\"Single-Pass Sort, Join and Aggregation Transformations [video]\">Read More<\/a><\/div>\n","protected":false},"author":10,"featured_media":11794,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[1,1030],"tags":[],"class_list":["post-9539","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-transformation2","category-vlog"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v23.4 (Yoast SEO v23.4) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Single-Pass Sort, Join and Aggregation Transformations [video] - IRI<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Single-Pass Sort, Join and Aggregation Transformations [video]\" \/>\n<meta property=\"og:description\" content=\"CoSort can sort, join, and aggregate in the same job. In fact, CoSort&#8217;s SortCL Program supports the consolidation of these three major data warehouse transformation functions in the same job script and I-O pass, along with reporting, protection, and many other data manipulations that would otherwise require separate steps, and possibly separate products. This kindRead More\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/\" \/>\n<meta property=\"og:site_name\" content=\"IRI\" \/>\n<meta property=\"article:published_time\" content=\"2010-01-01T05:00:42+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2017-11-06T20:10:29+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2010\/01\/single-pass-sort.png\" \/>\n\t<meta property=\"og:image:width\" content=\"760\" \/>\n\t<meta property=\"og:image:height\" content=\"468\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Susan Gegner\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Susan Gegner\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/\"},\"author\":{\"name\":\"Susan Gegner\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/87be5da567628ab9396ca81170f36d63\"},\"headline\":\"Single-Pass Sort, Join and Aggregation Transformations [video]\",\"datePublished\":\"2010-01-01T05:00:42+00:00\",\"dateModified\":\"2017-11-06T20:10:29+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/\"},\"wordCount\":183,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2010\/01\/single-pass-sort.png\",\"articleSection\":[\"Data Transformation\",\"VLOG\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/\",\"url\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/\",\"name\":\"Single-Pass Sort, Join and Aggregation Transformations [video] - IRI\",\"isPartOf\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2010\/01\/single-pass-sort.png\",\"datePublished\":\"2010-01-01T05:00:42+00:00\",\"dateModified\":\"2017-11-06T20:10:29+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/#primaryimage\",\"url\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2010\/01\/single-pass-sort.png\",\"contentUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2010\/01\/single-pass-sort.png\",\"width\":760,\"height\":468},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.iri.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Single-Pass Sort, Join and Aggregation Transformations [video]\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.iri.com\/blog\/#website\",\"url\":\"https:\/\/www.iri.com\/blog\/\",\"name\":\"IRI\",\"description\":\"Total Data Management Blog\",\"publisher\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.iri.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\",\"name\":\"IRI\",\"url\":\"https:\/\/www.iri.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"contentUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"width\":750,\"height\":206,\"caption\":\"IRI\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/87be5da567628ab9396ca81170f36d63\",\"name\":\"Susan Gegner\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/2b1ca5592a65d44483351292cf1ae00a?s=96&d=blank&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/2b1ca5592a65d44483351292cf1ae00a?s=96&d=blank&r=g\",\"caption\":\"Susan Gegner\"},\"url\":\"https:\/\/www.iri.com\/blog\/author\/susang\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Single-Pass Sort, Join and Aggregation Transformations [video] - IRI","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/","og_locale":"en_US","og_type":"article","og_title":"Single-Pass Sort, Join and Aggregation Transformations [video]","og_description":"CoSort can sort, join, and aggregate in the same job. In fact, CoSort&#8217;s SortCL Program supports the consolidation of these three major data warehouse transformation functions in the same job script and I-O pass, along with reporting, protection, and many other data manipulations that would otherwise require separate steps, and possibly separate products. This kindRead More","og_url":"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/","og_site_name":"IRI","article_published_time":"2010-01-01T05:00:42+00:00","article_modified_time":"2017-11-06T20:10:29+00:00","og_image":[{"width":760,"height":468,"url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2010\/01\/single-pass-sort.png","type":"image\/png"}],"author":"Susan Gegner","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Susan Gegner","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/#article","isPartOf":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/"},"author":{"name":"Susan Gegner","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/87be5da567628ab9396ca81170f36d63"},"headline":"Single-Pass Sort, Join and Aggregation Transformations [video]","datePublished":"2010-01-01T05:00:42+00:00","dateModified":"2017-11-06T20:10:29+00:00","mainEntityOfPage":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/"},"wordCount":183,"commentCount":0,"publisher":{"@id":"https:\/\/www.iri.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/#primaryimage"},"thumbnailUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2010\/01\/single-pass-sort.png","articleSection":["Data Transformation","VLOG"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/","url":"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/","name":"Single-Pass Sort, Join and Aggregation Transformations [video] - IRI","isPartOf":{"@id":"https:\/\/www.iri.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/#primaryimage"},"image":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/#primaryimage"},"thumbnailUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2010\/01\/single-pass-sort.png","datePublished":"2010-01-01T05:00:42+00:00","dateModified":"2017-11-06T20:10:29+00:00","breadcrumb":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/#primaryimage","url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2010\/01\/single-pass-sort.png","contentUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2010\/01\/single-pass-sort.png","width":760,"height":468},{"@type":"BreadcrumbList","@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/single-pass-sort-join-and-aggregation-transformations\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.iri.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Single-Pass Sort, Join and Aggregation Transformations [video]"}]},{"@type":"WebSite","@id":"https:\/\/www.iri.com\/blog\/#website","url":"https:\/\/www.iri.com\/blog\/","name":"IRI","description":"Total Data Management Blog","publisher":{"@id":"https:\/\/www.iri.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.iri.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.iri.com\/blog\/#organization","name":"IRI","url":"https:\/\/www.iri.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","contentUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","width":750,"height":206,"caption":"IRI"},"image":{"@id":"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/87be5da567628ab9396ca81170f36d63","name":"Susan Gegner","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/2b1ca5592a65d44483351292cf1ae00a?s=96&d=blank&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/2b1ca5592a65d44483351292cf1ae00a?s=96&d=blank&r=g","caption":"Susan Gegner"},"url":"https:\/\/www.iri.com\/blog\/author\/susang\/"}]}},"jetpack_featured_media_url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2010\/01\/single-pass-sort.png","_links":{"self":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/9539"}],"collection":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/comments?post=9539"}],"version-history":[{"count":5,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/9539\/revisions"}],"predecessor-version":[{"id":11795,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/9539\/revisions\/11795"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/media\/11794"}],"wp:attachment":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/media?parent=9539"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/categories?post=9539"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/tags?post=9539"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}