{"id":3290,"date":"2013-01-07T19:44:56","date_gmt":"2013-01-08T00:44:56","guid":{"rendered":"http:\/\/www.iri.com\/blog\/?p=3290"},"modified":"2017-11-06T14:08:58","modified_gmt":"2017-11-06T19:08:58","slug":"informatica-pushdown-optimization-with-cosort","status":"publish","type":"post","link":"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/","title":{"rendered":"Informatica Pushdown Optimization with CoSort"},"content":{"rendered":"<p><strong><a title=\"Big Data Blog Category\" href=\"http:\/\/www.iri.com\/blog\/category\/big-data-2\/\" target=\"_blank\" rel=\"noopener noreferrer\">Big data<\/a><\/strong> integration activities can happen outside the database in an extract, transform, load (ETL)\u00a0environment, or inside the database in ELT:<\/p>\n<p><a title=\"Blog Article - ETL vs. ELT: We Posit, You Judge\" href=\"http:\/\/www.iri.com\/blog\/data-transformation2\/etl-vs-elt-we-posit-you-judge\/\" target=\"_blank\" rel=\"noopener noreferrer\">http:\/\/www.iri.com\/blog\/data-transformation2\/etl-vs-elt-we-posit-you-judge\/<\/a><\/p>\n<p>One example of an ELT operation would be Informatica\u2019s Pushdown Optimization option, in which users transform data in a relational database like Oracle, or in Teradata. According to Informatica, this approach &#8220;allows the IT systems and their operators to respond to changing demands and peak processing needs.&#8221;<\/p>\n<p>IRI and its customers believe that the work of <a href=\"http:\/\/www.iri.com\/solutions\/big-data\"><span style=\"text-decoration: underline;\">big data<\/span><\/a> transformation does not belong in databases designed for storage and retrieval. Transformations in the database layer can burden the database and the whole system, drawing resources from data storage maintenance, and slowing down queries.<\/p>\n<p>Speed in any circumstance, whether an ETL or ELT operation, is always critical to the equation. The\u00a0<a title=\"CoSort Product Page\" href=\"http:\/\/www.iri.com\/products\/cosort\" target=\"_blank\" rel=\"noopener noreferrer\">IRI CoSort<\/a>\u00a0engine, and the modern <a href=\"http:\/\/www.iri.com\/prouducts\/voracity\">IRI Voracity<\/a> platform which uses it (<a href=\"http:\/\/www.iri.com\/solutions\/big-data\/hadoop-optional\">or Hadoop<\/a>), both support\u00a0the ETL paradigm. They optimize big data integration performance through superior transformation algorithms, task consolidation, and file system resource exploitation. For more information, see:<\/p>\n<p><a title=\"Data Integration &amp; ETL Solutions Page\" href=\"http:\/\/www.iri.com\/solutions\/data-integration\/\" target=\"_blank\" rel=\"noopener noreferrer\">http:\/\/www.iri.com\/solutions\/data-integration<\/a><\/p>\n<p>Such external operations are more efficient, affordable, and remove the burdens of data integration and staging from BI, database and legacy ETL tools, freeing them to do the jobs they do best instead.\u00a0Teradata, Hadoop, in-memory and columnar databases, and ELT appliances are also much more expensive and complicated &#8216;first resort&#8217; options than CoSort!<\/p>\n<p><strong>How Does this Work?<\/strong><\/p>\n<p>IRI is frequently approached by customers using platforms like Informatica PowerCenter who need to accelerate production jobs, where jobs like sorting, joining and aggregation are staged separately:<\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/S-J-Agg-Workforce.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-3422\" title=\"S-J-Agg-Workforce\" src=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/S-J-Agg-Workforce.jpg\" alt=\"Sort, join, aggregation workflow\" width=\"557\" height=\"266\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/S-J-Agg-Workforce.jpg 928w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/S-J-Agg-Workforce-300x143.jpg 300w\" sizes=\"(max-width: 557px) 100vw, 557px\" \/><\/a><\/p>\n<p>With CoSort, all data transformation tasks run a single job script and I\/O pass. Those jobs can also include filtering and conversion, an extensive array of field-level protections, custom reformatting and reporting with math functions, special layouts, multiple targets, and so on.<\/p>\n<p>The difference CoSort data transformation makes can be seen even on a laptop. An independent consultant using PowerCenter 8.6 under Windows XP SP3 on an Intel\u00ae Core\u2122 CPU M370 @2.4GHz with 3 GB of DDR3 memory defined sort, join, and aggregate stages for 100MB and 53MB flat files, and ran them all in 94 seconds. That same job &#8216;pushed out&#8217; to a \u00a0single CoSort Sort Control Language (<a href=\"http:\/\/www.iri.com\/products\/cosort\/sortcl\">SortCL<\/a>) program in the file system produced the same results &#8220;from&#8221; Informatica in 16 seconds.<\/p>\n<p>&#8220;Push out&#8221; optimization from Informatica to CoSort is straightforward:<\/p>\n<p>1. Create a simple SortCL language program like the one below in any text editor or new job wizard in the <a href=\"http:\/\/www.iri.com\/products\/workbench\">IRI Workbench<\/a> GUI, built on Eclipse. The job runs on the command line ($sortcl \/spec=script.scl) and can be called through the same PowerCenter &#8220;Command Task&#8221; that runs any .exe file.<\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-1.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-3292 aligncenter\" title=\"Slide-1\" src=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-1.jpg\" alt=\"A SocrtCL script that can be run from numerous places for big data processing\" width=\"601\" height=\"318\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-1.jpg 927w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-1-300x158.jpg 300w\" sizes=\"(max-width: 601px) 100vw, 601px\" \/><\/a><\/p>\n<p>2. To create such a command task in PowerCenter, select &#8220;Workflow&#8221; from the upper menu and then select &#8220;create&#8221;:<\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-21.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-3294\" title=\"Slide-2\" src=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-21.jpg\" alt=\"Creating a command task in PowerCenter for CoSort to transform big data\" width=\"551\" height=\"267\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-21.jpg 918w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-21-300x145.jpg 300w\" sizes=\"(max-width: 551px) 100vw, 551px\" \/><\/a><\/p>\n<p>3. Name the workflow.\u00a0 For example, wf_sorting_joining_aggr_cmd<\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-3.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-3295\" title=\"Slide-3\" src=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-3.jpg\" alt=\"Creating a big data workflow in Informatica PowerCenter\" width=\"553\" height=\"310\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-3.jpg 921w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-3-300x168.jpg 300w\" sizes=\"(max-width: 553px) 100vw, 553px\" \/><\/a><\/p>\n<p>4. To display the command in the workflow designer, select command task from the upper tool bar and\u00a0create the actual command that will run:<\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-101.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-3348\" title=\"Slide-10\" src=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-101.jpg\" alt=\"Workflow designer showing CoSort for big data transformation\" width=\"434\" height=\"226\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-101.jpg 723w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-101-300x156.jpg 300w\" sizes=\"(max-width: 434px) 100vw, 434px\" \/><\/a><\/p>\n<p style=\"text-align: left;\"><span style=\"text-align: left;\">5. This task will call the SortCL job in the command task. The task itself can be edited to call any job; in this case the SortCL job script &#8220;sortjoinaggr.scl.&#8221;<\/span><\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-12.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-3303\" title=\"Slide-12\" src=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-12.jpg\" alt=\"PowerCenter calling a SortCL script to speed big data in Informatica\" width=\"564\" height=\"317\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-12.jpg 940w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-12-300x168.jpg 300w\" sizes=\"(max-width: 564px) 100vw, 564px\" \/><\/a><\/p>\n<p>6. The consultant&#8217;s last image shows the CoSort\/SortCL job completed from Informatica in 16 seconds; while the original job in Informatica took 94:<\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-13.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-3304\" title=\"Slide-13\" src=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-13.jpg\" alt=\"Results show that CoSort transformed big data in 16 seconds.\" width=\"551\" height=\"251\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-13.jpg 918w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/Slide-13-300x136.jpg 300w\" sizes=\"(max-width: 551px) 100vw, 551px\" \/><\/a><\/p>\n<p style=\"text-align: left;\">Contact <a href=\"mailto:info@iri.com\" target=\"_blank\" rel=\"noopener noreferrer\">info@iri.com<\/a> if you are interested in optimizing your ETL tools&#8217; transformations through a simple call to CoSort. See <a href=\"http:\/\/www.iri.com\/blog\/etl\/from-informatica-to-voracity-via-ads-catfx\/\" target=\"_blank\" rel=\"noopener noreferrer\">this article<\/a> if you&#8217;re interested in <a href=\"http:\/\/www.iri.com\/solutions\/data-integration\/replatform-etl\">re-platforming<\/a> Informatica ETL jobs to the same jobs in <a href=\"http:\/\/www.iri.com\/blog\/etl\/from-informatica-to-voracity-via-ads-catfx\/\" target=\"_blank\" rel=\"noopener noreferrer\">IRI Voracity<\/a>\u00a0jobs via AnalytiX DS technology to save time and money.<\/p>\n<p style=\"text-align: left;\">FYI, Voracity is IRI&#8217;s &#8216;total data management platform&#8217; which uses CoSort or Hadoop engines (but the same metadata either way). Voracity <a href=\"http:\/\/www.iri.com\/products\/voracity\/features-benefits\">combines<\/a>:<\/p>\n<ol>\n<li style=\"text-align: left;\">data discovery (DB &amp; flat-file profiling, classification, ERDs, dark data)<\/li>\n<li style=\"text-align: left;\">data integration (ETL, CDC, SCD, etc.),<\/li>\n<li style=\"text-align: left;\">data migration (for databases, file formats, data types, etc.) and replication<\/li>\n<li style=\"text-align: left;\">data governance (cleansing, masking, EMM, MDM, etc.)<\/li>\n<li style=\"text-align: left;\">analytics (embedded reporting and data preparation)<\/li>\n<\/ol>\n<p style=\"text-align: left;\">all in the same Eclipse <a href=\"http:\/\/www.iri.com\/products\/workbench\/voracity-gui\" target=\"_blank\" rel=\"noopener noreferrer\">GUI<\/a>. Voracity\u00a0is also compatible with AnalytiX DS Mapping Manager and CATfx templates for managing ETL metadata stratification, creation, lineage, and conversion.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Big data integration activities can happen outside the database in an extract, transform, load (ETL)\u00a0environment, or inside the database in ELT: http:\/\/www.iri.com\/blog\/data-transformation2\/etl-vs-elt-we-posit-you-judge\/ One example of an ELT operation would be Informatica\u2019s Pushdown Optimization option, in which users transform data in a relational database like Oracle, or in Teradata. According to Informatica, this approach &#8220;allows the<\/p>\n<div><a class=\"btn-filled btn\" href=\"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/\" title=\"Informatica Pushdown Optimization with CoSort\">Read More<\/a><\/div>\n","protected":false},"author":5,"featured_media":11783,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[1,776],"tags":[290,119,25,44,5,106,100,131,46,112,215,289,132,118,288,312,114,115,117,116,402,68],"class_list":["post-3290","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-transformation2","category-etl","tag-8-6","tag-aggregate","tag-big-data","tag-cosort","tag-data-transformation","tag-elt","tag-etl","tag-etl-tool","tag-etl-tools","tag-etl-vs-elt","tag-in","tag-informatica","tag-informatica-etl","tag-join","tag-optimization","tag-pushdown","tag-pushdown-optimization","tag-relational-database-optimization","tag-sort","tag-sort-join-aggregation","tag-sort-package","tag-sortcl"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v23.4 (Yoast SEO v23.4) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Informatica Pushdown Optimization with CoSort - IRI<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Informatica Pushdown Optimization with CoSort\" \/>\n<meta property=\"og:description\" content=\"Big data integration activities can happen outside the database in an extract, transform, load (ETL)\u00a0environment, or inside the database in ELT: http:\/\/www.iri.com\/blog\/data-transformation2\/etl-vs-elt-we-posit-you-judge\/ One example of an ELT operation would be Informatica\u2019s Pushdown Optimization option, in which users transform data in a relational database like Oracle, or in Teradata. According to Informatica, this approach &#8220;allows theRead More\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/\" \/>\n<meta property=\"og:site_name\" content=\"IRI\" \/>\n<meta property=\"article:published_time\" content=\"2013-01-08T00:44:56+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2017-11-06T19:08:58+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/informatica-pushdown-optimization.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"600\" \/>\n\t<meta property=\"og:image:height\" content=\"337\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Jason Koivu\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Jason Koivu\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/\"},\"author\":{\"name\":\"Jason Koivu\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/c60bc4ff5919427034376979fb2cc8df\"},\"headline\":\"Informatica Pushdown Optimization with CoSort\",\"datePublished\":\"2013-01-08T00:44:56+00:00\",\"dateModified\":\"2017-11-06T19:08:58+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/\"},\"wordCount\":700,\"commentCount\":1,\"publisher\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/informatica-pushdown-optimization.jpg\",\"keywords\":[\"8.6\",\"aggregate\",\"big data\",\"CoSort\",\"data transformation\",\"ELT\",\"ETL\",\"ETL tool\",\"ETL tools\",\"ETL vs. ELT\",\"in\",\"informatica\",\"Informatica ETL\",\"join\",\"optimization\",\"pushdown\",\"pushdown optimization\",\"relational database optimization\",\"sort\",\"sort join aggregation\",\"sort package\",\"SortCL\"],\"articleSection\":[\"Data Transformation\",\"ETL\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/\",\"url\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/\",\"name\":\"Informatica Pushdown Optimization with CoSort - IRI\",\"isPartOf\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/informatica-pushdown-optimization.jpg\",\"datePublished\":\"2013-01-08T00:44:56+00:00\",\"dateModified\":\"2017-11-06T19:08:58+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/#primaryimage\",\"url\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/informatica-pushdown-optimization.jpg\",\"contentUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/informatica-pushdown-optimization.jpg\",\"width\":600,\"height\":337},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.iri.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Informatica Pushdown Optimization with CoSort\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.iri.com\/blog\/#website\",\"url\":\"https:\/\/www.iri.com\/blog\/\",\"name\":\"IRI\",\"description\":\"Total Data Management Blog\",\"publisher\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.iri.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\",\"name\":\"IRI\",\"url\":\"https:\/\/www.iri.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"contentUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"width\":750,\"height\":206,\"caption\":\"IRI\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/c60bc4ff5919427034376979fb2cc8df\",\"name\":\"Jason Koivu\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/01e97234ff964558ca620a43a0506ef0?s=96&d=blank&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/01e97234ff964558ca620a43a0506ef0?s=96&d=blank&r=g\",\"caption\":\"Jason Koivu\"},\"url\":\"https:\/\/www.iri.com\/blog\/author\/jasonk\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Informatica Pushdown Optimization with CoSort - IRI","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/","og_locale":"en_US","og_type":"article","og_title":"Informatica Pushdown Optimization with CoSort","og_description":"Big data integration activities can happen outside the database in an extract, transform, load (ETL)\u00a0environment, or inside the database in ELT: http:\/\/www.iri.com\/blog\/data-transformation2\/etl-vs-elt-we-posit-you-judge\/ One example of an ELT operation would be Informatica\u2019s Pushdown Optimization option, in which users transform data in a relational database like Oracle, or in Teradata. According to Informatica, this approach &#8220;allows theRead More","og_url":"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/","og_site_name":"IRI","article_published_time":"2013-01-08T00:44:56+00:00","article_modified_time":"2017-11-06T19:08:58+00:00","og_image":[{"width":600,"height":337,"url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/informatica-pushdown-optimization.jpg","type":"image\/jpeg"}],"author":"Jason Koivu","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Jason Koivu","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/#article","isPartOf":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/"},"author":{"name":"Jason Koivu","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/c60bc4ff5919427034376979fb2cc8df"},"headline":"Informatica Pushdown Optimization with CoSort","datePublished":"2013-01-08T00:44:56+00:00","dateModified":"2017-11-06T19:08:58+00:00","mainEntityOfPage":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/"},"wordCount":700,"commentCount":1,"publisher":{"@id":"https:\/\/www.iri.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/#primaryimage"},"thumbnailUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/informatica-pushdown-optimization.jpg","keywords":["8.6","aggregate","big data","CoSort","data transformation","ELT","ETL","ETL tool","ETL tools","ETL vs. ELT","in","informatica","Informatica ETL","join","optimization","pushdown","pushdown optimization","relational database optimization","sort","sort join aggregation","sort package","SortCL"],"articleSection":["Data Transformation","ETL"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/","url":"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/","name":"Informatica Pushdown Optimization with CoSort - IRI","isPartOf":{"@id":"https:\/\/www.iri.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/#primaryimage"},"image":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/#primaryimage"},"thumbnailUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/informatica-pushdown-optimization.jpg","datePublished":"2013-01-08T00:44:56+00:00","dateModified":"2017-11-06T19:08:58+00:00","breadcrumb":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/#primaryimage","url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/informatica-pushdown-optimization.jpg","contentUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/informatica-pushdown-optimization.jpg","width":600,"height":337},{"@type":"BreadcrumbList","@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/informatica-pushdown-optimization-with-cosort\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.iri.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Informatica Pushdown Optimization with CoSort"}]},{"@type":"WebSite","@id":"https:\/\/www.iri.com\/blog\/#website","url":"https:\/\/www.iri.com\/blog\/","name":"IRI","description":"Total Data Management Blog","publisher":{"@id":"https:\/\/www.iri.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.iri.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.iri.com\/blog\/#organization","name":"IRI","url":"https:\/\/www.iri.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","contentUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","width":750,"height":206,"caption":"IRI"},"image":{"@id":"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/c60bc4ff5919427034376979fb2cc8df","name":"Jason Koivu","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/01e97234ff964558ca620a43a0506ef0?s=96&d=blank&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/01e97234ff964558ca620a43a0506ef0?s=96&d=blank&r=g","caption":"Jason Koivu"},"url":"https:\/\/www.iri.com\/blog\/author\/jasonk\/"}]}},"jetpack_featured_media_url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2013\/01\/informatica-pushdown-optimization.jpg","_links":{"self":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/3290"}],"collection":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/comments?post=3290"}],"version-history":[{"count":126,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/3290\/revisions"}],"predecessor-version":[{"id":11249,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/3290\/revisions\/11249"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/media\/11783"}],"wp:attachment":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/media?parent=3290"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/categories?post=3290"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/tags?post=3290"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}