{"id":9813,"date":"2016-03-20T11:46:24","date_gmt":"2016-03-20T15:46:24","guid":{"rendered":"http:\/\/www.iri.com\/blog\/?p=9813"},"modified":"2017-11-06T13:46:55","modified_gmt":"2017-11-06T18:46:55","slug":"creating-voracity-flows-using-existing-iri-scripts-part-1","status":"publish","type":"post","link":"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/","title":{"rendered":"Creating a Voracity Flow Using Existing IRI Scripts (Part 1 of 3)"},"content":{"rendered":"<p>This is first in a series of articles explaining how to create and use Flows in the IRI Workbench\u00a0<a style=\"color: #1155cc; text-decoration: underline;\" href=\"http:\/\/www.iri.com\/products\/workbench\/voracity-gui\">GUI for Voracity<\/a>. Flows contain ETL and other data processing steps, and are illustrated in flow and transform mapping diagrams in the GUI. They are saved in .flow files for use in the GUI, and can be serialized as .sh or .bat files for execution from within or outside the GUI.<\/p>\n<p>The example below assumes that data transformation and\/or reporting scripts in the <a style=\"color: #1155cc; text-decoration: underline;\" href=\"http:\/\/www.iri.com\/products\/sortcl\">SortCL<\/a> language of IRI CoSort already exist. It also shows how to modify the flow as requirements for the job change. Subsequent articles will be a continuation of the flow started here.<\/p>\n<p>The following scenario starts with a month-end job that consists of a CoSort SortCL script contained within a batch process. We now want to represent this job in a flow using simulated sales transactions.<\/p>\n<p style=\"font-size: 12pt; font-weight: bold;\"><span style=\"font-size: 12pt; font-weight: bold;\"><span style=\"font-size: 12pt; font-weight: bold;\">Process Outline<\/span><\/span><\/p>\n<p>We have a sales transaction table (TRANSACTIONS) in Oracle, where each entry is appended to the table as it occurs. The table contains records for purchases, returns, and account payments. These transaction records are extracted using a SortCL script. Then, they are sorted into three (3) separate target files. One for:<\/p>\n<ul>\n<li>credit transactions (CreditRecords.dat) \u2013 these include all 3 transaction types mentioned above<\/li>\n<li>cash transactions (CashRecords.dat)<\/li>\n<li>all transactions except for credit payments (AllSales.dat)<\/li>\n<\/ul>\n<p>Here is the script (SortTransSelect.scl) that extracts records and maps the fields to these files. Notice the \/QUERY that only allows selection of records from the table for the current month (December in this case) using an SQL statement.<\/p>\n<pre>\/STATISTICS=sorttrans.stat\r\n\/INFILE=\u201dNIGHTLY.TRANSACTIONS;DSN=Oracle_qa2;\u201d\r\n     \/PROCESS=ODBC\r\n     \/ALIAS=NIGHTLY_TRANSACTIONS\r\n     \/QUERY=\u201dSELECT * FROM NIGHTLY.TRANSACTIONS WHERE PURCHASE_DATE &gt;= \\\u2019161201\\\u2019 AND\r\nPURCHASE_DATE &lt; \\\u2019170101\\'\u201d\r\n     \/FIELD=(TRANSTYPE, TYPE=ASCII, POSITION=1, SEPARATOR=\u201d|\u201d)\r\n     \/FIELD=(ACCTNUM, TYPE=ASCII, POSITION=2, SEPARATOR=\u201d|\u201d)\r\n     \/FIELD=(QUANTITY, TYPE=NUMERIC, POSITION=3, SEPARATOR=\u201d|\u201d, PRECISION=0)\r\n     \/FIELD=(DEPT, TYPE=ASCII, POSITION=4, SEPARATOR=\u201d|\u201d)\r\n     \/FIELD=(SKU, TYPE=ASCII, POSITION=5, SEPARATOR=\u201d|\u201d)\r\n     \/FIELD=(SKU_DESCRIPTION, TYPE=ASCII, POSITION=6, SEPARATOR=\u201d|\u201d)\r\n     \/FIELD=(PRICE, TYPE=NUMERIC, POSITION=7, SEPARATOR=\u201d|\u201d, PRECISION=2)\r\n     \/FIELD=(PURCHASE_DATE, TYPE=ASCII, POSITION=8, SEPARATOR=\u201d|\u201d)\r\n     \/FIELD=(TOTAL_PRICE, TYPE=NUMERIC, POSITION=9, SEPARATOR=\u201d|\u201d, PRECISION=2)\r\n\r\n\/SORT\r\n     \/KEY=(ACCTNUM, TYPE=ASCII)\r\n\r\n\/OUTFILE=CashRecords.dat\r\n     \/PROCESS=RECORD\r\n     \/INCLUDE WHERE ACCTNUM EQ \u201c999999999999\u201d\r\n     \/FIELD=(TRANSTYPE, TYPE=ASCII, POSITION=1, SEPARATOR=\u201d|\u201d)\r\n     \/FIELD=(ACCTNUM, TYPE=ASCII, POSITION=2, SEPARATOR=\u201d|\u201d)\r\n     \/FIELD=(QUANTITY, TYPE=NUMERIC, POSITION=3, SEPARATOR=\u201d|\u201d, PRECISION=0)\r\n     \/FIELD=(DEPT, TYPE=ASCII, POSITION=4, SEPARATOR=\u201d|\u201d)\r\n     \/FIELD=(SKU, TYPE=ASCII, POSITION=5, SEPARATOR=\u201d|\u201d)\r\n     \/FIELD=(SKU_DESCRIPTION, TYPE=ASCII, POSITION=6, SEPARATOR=\u201d|\u201d)\r\n     \/FIELD=(PRICE, TYPE=NUMERIC, POSITION=7, SEPARATOR=\u201d|\u201d, PRECISION=2)\r\n     \/FIELD=(PURCHASE_DATE, TYPE=ASCII, POSITION=8, SEPARATOR=\u201d|\u201d)\r\n     \/FIELD=(TOTAL_PRICE, TYPE=NUMERIC, POSITION=9, SEPARATOR=\u201d|\u201d, PRECISION=2)\r\n\r\n\/OUTFILE=CreditRecords.dat\r\n     \/PROCESS=RECORD\r\n     \/OMIT WHERE ACCTNUM EQ \u201c999999999999\u201d\r\n     \/FIELD=(TRANSTYPE, TYPE=ASCII, POSITION=1, SEPARATOR=\u201d|\u201d)\r\n     \/FIELD=(ACCTNUM, TYPE=ASCII, POSITION=2, SEPARATOR=\u201d|\u201d)\r\n     \/FIELD=(QUANTITY, TYPE=NUMERIC, POSITION=3, SEPARATOR=\u201d|\u201d, PRECISION=0)\r\n     \/FIELD=(DEPT, TYPE=ASCII, POSITION=4, SEPARATOR=\u201d|\u201d)\r\n     \/FIELD=(SKU, TYPE=ASCII, POSITION=5, SEPARATOR=\u201d|\u201d)\r\n     \/FIELD=(SKU_DESCRIPTION, TYPE=ASCII, POSITION=6, SEPARATOR=\u201d|\u201d)\r\n     \/FIELD=(PRICE=PRICE, TYPE=NUMERIC, POSITION=7, SEPARATOR=\u201d|\u201d, PRECISION=2)\r\n     \/FIELD=(PURCHASE_DATE, TYPE=ASCII, POSITION=8, SEPARATOR=\u201d|\u201d)\r\n     \/FIELD=(TOTAL_PRICE, TYPE=NUMERIC, POSITION=9, SEPARATOR=\u201d|\u201d, PRECISION=2)\r\n\r\n\/OUTFILE=AllSales.dat\r\n     \/PROCESS=RECORD\r\n     \/FIELD=(DEPT, TYPE=ASCII, POSITION=1, SEPARATOR=\u201d|\u201d)\r\n     \/FIELD=(YEARMONTH=sub_string(PURCHASE_DATE, 1, 4), TYPE=ASCII, POSITION=2, SEPARATOR=\u201d|\u201d)\r\n     \/FIELD=(TOTAL_PRICE, TYPE=NUMERIC, POSITION=3, SEPARATOR=\u201d|\u201d, PRECISION=2)\r\n     \/INCLUDE WHERE TRANSTYPE NE \u201c0\u201d\r\n<\/pre>\n<p>Before proceeding, a project needs to be created in IRI Workbench to hold all the files used in this job. I created the project <span style=\"font-style: italic;\">MonthEnd<\/span>.<\/p>\n<p style=\"font-size: 12pt; font-weight: bold;\"><span style=\"font-size: 12pt; font-weight: bold;\"><span style=\"font-size: 12pt; font-weight: bold;\">Create a Flow<\/span><\/span><\/p>\n<p>You can create a flow from an existing script by right-clicking the SortTransSelect.scl file in the <span style=\"font-weight: bold;\">Project Explorer<\/span> and selecting <span style=\"font-weight: bold;\">IRI<\/span> &gt;<span style=\"font-weight: bold;\">Create Flow from Script<\/span>. This creates a flow file named SortTransSelect.flow, adds the script to it, and opens a diagram in one step (after saying yes to displaying the diagram in the prompt). Click Save. Rename the file to <span style=\"font-weight: bold;\">MonthEnd.flow<\/span> by right-clicking it in the <span style=\"font-weight: bold;\">Project Explorer<\/span> and selecting <span style=\"font-weight: bold;\">Rename<\/span>.<\/p>\n<p>The flow diagram that displays, contains a green <span style=\"font-weight: bold;\">Start <\/span>block connected to a brown <span style=\"font-weight: bold;\">Transform Mapping Block<\/span> contained by a grey <span style=\"font-weight: bold;\">Flowlet<\/span> block. A flowlet block and one start block are required for all flow jobs.<\/p>\n<p>Make sure you have a <span style=\"font-weight: bold;\">Properties View<\/span> open where you can edit more detailed information about the Flow file using the <span style=\"font-weight: bold;\">Base <\/span>tab. Click on the flowlet block, change the name to <span style=\"font-style: italic;\">MonthEnd<\/span> in <span style=\"font-weight: bold;\">Properties<\/span>, and click save on the navigation bar. The flowlet represents the job and since the job will be run on windows, will be used to create the batch file for the job.<\/p>\n<p>In the <span style=\"font-weight: bold;\">Properties <\/span>view, the transform mapping block has a property called <span style=\"font-weight: bold;\">Name <\/span>and a property called <span style=\"font-weight: bold;\">IRI Job,<\/span> both of which have the name of the script that is represented. The transform mapping block displays the inputs and outputs with the input using the icon for a database table and the outputs using the icon for a file.<\/p>\n<p style=\"font-size: 12pt; font-weight: bold;\"><span style=\"font-size: 12pt; font-weight: bold;\"><span style=\"font-size: 12pt; font-weight: bold;\">Create a Transform Mapping Diagram<\/span><\/span><\/p>\n<p>Double-click the transform mapping block to open a transform mapping diagram. Name the diagram <span style=\"font-style: italic;\">SortTrans Transform Mapping Diagram<\/span> and click <span style=\"font-weight: bold;\">OK<\/span>.<\/p>\n<p style=\"font-family: Calibri;\"><a title=\"Creating Voracity Flows Using Existing IRI Scripts (Part 1) - Susan Gegner 0.jpg\" href=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2016\/06\/Creating-Voracity-Flows-Using-Existing-IRI-Scripts-Part-1-Susan-Gegner-0.jpg\" rel=\"prettyPhoto\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter\" style=\"width: 624px; height: 389px;\" title=\"Create a Transform Mapping Diagram\" src=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2016\/06\/t_Creating-Voracity-Flows-Using-Existing-IRI-Scripts-Part-1-Susan-Gegner-0.jpg\" alt=\"Create a Transform Mapping Diagram\" width=\"624\" height=\"389\" \/><\/a><\/p>\n<p>A detailed diagram for the script now displays in the editor. I have moved and resized the blocks to better display those details.<\/p>\n<p style=\"font-family: Calibri;\"><a title=\"Creating Voracity Flows Using Existing IRI Scripts (Part 1) - Susan Gegner 1.jpg\" href=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2016\/06\/Creating-Voracity-Flows-Using-Existing-IRI-Scripts-Part-1-Susan-Gegner-1.jpg\" rel=\"prettyPhoto\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter\" style=\"width: 624px; height: 350px;\" title=\"A detailed diagram for the script\" src=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2016\/06\/t_Creating-Voracity-Flows-Using-Existing-IRI-Scripts-Part-1-Susan-Gegner-1.jpg\" alt=\"A detailed diagram for the script\" width=\"624\" height=\"350\" \/><\/a><\/p>\n<p>Notice that there are three main blocks:<\/p>\n<p>1. \u00a0The purple block is for <span style=\"font-weight: bold;\">Input Data. <\/span>Each data source is defined in a white block. There is a:<\/p>\n<ul>\n<li>yellow block for <span style=\"font-weight: bold;\">Section Options<\/span>. This contains the definitions that apply to the whole file or table. This is where filters can be applied. You can double-click on any one of the options to bring up a screen to edit the option. To add an option, click in the block, and go to the <span style=\"font-weight: bold;\">Section Options<\/span> tab under the <span style=\"font-weight: bold;\">Properties<\/span> view.<\/li>\n<li>blue block with all the definitions of fields that are referenced for the file or table. You can double-click on any field to bring up the <span style=\"font-weight: bold;\">Field Editor<\/span> to make changes to the definition of that field.<\/li>\n<\/ul>\n<p>2. \u00a0The dark gray block is for an <span style=\"font-weight: bold;\">Action<\/span>. Within this block, the light gray block shows the fields that can be mapped to the target. The action can:<\/p>\n<ul>\n<li><strong>Sort<\/strong> using the Keys in the Action Key block<\/li>\n<li><strong>Report<\/strong> where no keys are used and records are copied from input to output<\/li>\n<li><strong>Join<\/strong> records from 2 or more files or tables based on common fields<\/li>\n<li><strong>Merge<\/strong> pre-sorted files using the Keys in the Action Key block<\/li>\n<li><strong>Check\u00a0<\/strong>using the Keys in the<span style=\"font-weight: bold;\"> Action Key <\/span>block to find the first record that is not in order.<\/li>\n<\/ul>\n<p>3. \u00a0The green block is for <span style=\"font-weight: bold;\">Output Data. <\/span>Each target is defined in a white block. It also contains a:<\/p>\n<ul>\n<li>yellow block for <span style=\"font-weight: bold;\">Section Options<\/span> and, as in the input, contains definitions that apply to the whole file or table, including filters that are only applied to the particular output. Here is where aggregation definitions are held, but separate layout attributes for the aggregation field will be in the blue block.<\/li>\n<li>blue block with the definitions of fields that are mapped from input to output. The fields that are connected with orange arrows are derived fields where masking, math or string<br \/>\nfunctions, ad hoc pseudonymization, and IF-THEN-ELSE selection or transform logic has been applied.<\/li>\n<\/ul>\n<p>Each of the output files for our script has a filter in the <span style=\"font-weight: bold;\">Section Options<\/span>. CashRecords.dat and AllSales.dat each have a <span style=\"font-weight: bold;\">Filter Include<\/span> from the <span style=\"font-weight: bold;\">Palette<\/span>, and CreditRecords.dat has a <span style=\"font-weight: bold;\">Filter Omit<\/span>. Notice that the blue arrows show how the fields or columns map from the input, through the action, and then to the outputs. The outputs can receive all or some of the fields, and can have them in a different order than the input.<\/p>\n<p>There is one field in the target shown with an orange arrow for the connection. This indicates that a function or transformation is being applied to the data in the field, so its target value is derived from the original field value. Double-click on the field with the orange arrow to bring up the <span style=\"font-weight: bold;\">Field Editor<\/span>. This dialog shows that the sub_string function is applied to the field PURCHASE_DATE to get the value for the field YEARMONTH.<\/p>\n<p style=\"font-family: Calibri;\" align=\"center\"><a title=\"Creating Voracity Flows Using Existing IRI Scripts (Part 1) - Susan Gegner 2.jpg\" href=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2016\/06\/Creating-Voracity-Flows-Using-Existing-IRI-Scripts-Part-1-Susan-Gegner-2.jpg\" rel=\"prettyPhoto\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter\" style=\"width: 441px; height: 289px;\" title=\"Field Editor\" src=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2016\/06\/t_Creating-Voracity-Flows-Using-Existing-IRI-Scripts-Part-1-Susan-Gegner-2.jpg\" alt=\"Field Editor\" width=\"441\" height=\"289\" \/><\/a><\/p>\n<p style=\"font-size: 12pt; font-weight: bold;\"><span style=\"font-size: 12pt; font-weight: bold;\"><span style=\"font-size: 12pt; font-weight: bold;\">Deleting a Target<\/span><\/span><\/p>\n<p>After studying the Transform Mapping Diagram, I decided to make a few changes to the job. There is no longer a need for the output file CashRecords.dat. To delete, either use the delete key on the keyboard or right-click in the white block for CashRecords.dat, then <span style=\"font-weight: bold;\">Edit<\/span> &gt; <span style=\"font-weight: bold;\">Delete from model<\/span>.<\/p>\n<p style=\"font-size: 12pt; font-weight: bold;\"><span style=\"font-size: 12pt; font-weight: bold;\"><span style=\"font-size: 12pt; font-weight: bold;\">Adding a Sort Key<\/span><\/span><\/p>\n<p>Now we need to add a key to the sort. The PURCHASE_DATE needs to be sorted within each ACCTNUM. Double-click in the white <span style=\"font-weight: bold;\">Sort<\/span> block to bring up the <span style=\"font-weight: bold;\">Sort<\/span> dialog. This is where all edits to the keys must be done. Under <span style=\"font-weight: bold;\">Input Fields<\/span>, click on the field PURCHASE_DATE &gt; <span style=\"font-weight: bold;\">Add key<\/span>. PURCHASE_DATE is added to the <span style=\"font-weight: bold;\">Key Fields<\/span> column. Then click <span style=\"font-weight: bold;\">Finish<\/span>. The new key is now displayed in the <span style=\"font-weight: bold;\">Action Key<\/span> block. The following represents the script with our changes:<\/p>\n<p style=\"font-family: Calibri;\"><a title=\"Creating Voracity Flows Using Existing IRI Scripts (Part 1) - Susan Gegner 3.jpg\" href=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2016\/06\/Creating-Voracity-Flows-Using-Existing-IRI-Scripts-Part-1-Susan-Gegner-3.jpg\" rel=\"prettyPhoto\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter\" style=\"width: 624px; height: 302px;\" title=\"Adding a Sort Key\" src=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2016\/06\/t_Creating-Voracity-Flows-Using-Existing-IRI-Scripts-Part-1-Susan-Gegner-3.jpg\" alt=\"Adding a Sort Key\" width=\"624\" height=\"302\" \/><\/a><\/p>\n<p>We can now go back to the MonthEnd Flow Diagram. Notice the icon for the file Out_CashRecords.dat has been removed.<\/p>\n<p style=\"font-family: Calibri;\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter\" style=\"width: 624px; height: 412px;\" title=\"MonthEnd Flow Diagram\" src=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2016\/06\/Creating-Voracity-Flows-Using-Existing-IRI-Scripts-Part-1-Susan-Gegner-4.jpg\" alt=\"MonthEnd Flow Diagram\" width=\"624\" height=\"412\" \/><\/p>\n<p style=\"font-size: 12pt; font-weight: bold;\"><span style=\"font-size: 12pt; font-weight: bold;\"><span style=\"font-size: 12pt; font-weight: bold;\">Exporting the Flow<\/span><\/span><\/p>\n<p>Right-click in the flowlet &gt; <span style=\"font-weight: bold;\">IRI Diagram Actions<\/span> &gt; <span style=\"font-weight: bold;\">Export Flow Component<\/span>. Here is where you select the operating system type and name the batch\/shell script, in this case <span style=\"font-style: italic;\">MonthEnd.bat<\/span> because the batch script will be executed in Windows. Upon clicking <span style=\"font-weight: bold;\">Finish<\/span>, a batch file is created and the script SortTransSelect.scl is updated. The script is named the same as the <span style=\"font-weight: bold;\">IRI Job <\/span>field in the transform mapping block. If you want to keep the original job script then put a different name for the <span style=\"font-weight: bold;\">IRI Job<\/span> field in the <span style=\"font-weight: bold;\">Properties <\/span>view.<\/p>\n<p>The file MonthEnd.bat can be used to execute this flow and consists of the following lines:<\/p>\n<p style=\"font-size: 9pt; font-family: Courier New;\"><span style=\"font-size: 9pt; font-family: Courier New;\"><span style=\"font-family: Courier New; font-size: 9pt;\">@echo off<br \/>\n<\/span><\/span><span style=\"font-size: 9pt; font-family: Courier New;\"><span style=\"font-family: Courier New; font-size: 9pt;\">sortcl \/SPECIFICATION=SortTransSelect.scl<br \/>\n<\/span><\/span><\/p>\n<p>The batch flow, or individually executable job steps within it, can run from within Workbench directly, through its task scheduler, any third-party job scheduler, or from the command line.<\/p>\n<p>Continue onto part 2 in this series with <a href=\"http:\/\/www.iri.com\/blog\/iri\/iri-workbench\/creating-voracity-flows-using-existing-iri-scripts-part-2\/\">this next article<\/a>, where we will add another task (job script) to the flow. For feedback or help with your use case, please email <a style=\"color: #1155cc; text-decoration: underline;\" href=\"mailto:voracity@iri.com\">voracity@iri.com<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This is first in a series of articles explaining how to create and use Flows in the IRI Workbench\u00a0GUI for Voracity. Flows contain ETL and other data processing steps, and are illustrated in flow and transform mapping diagrams in the GUI. They are saved in .flow files for use in the GUI, and can be<\/p>\n<div><a class=\"btn-filled btn\" href=\"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/\" title=\"Creating a Voracity Flow Using Existing IRI Scripts (Part 1 of 3)\">Read More<\/a><\/div>\n","protected":false},"author":10,"featured_media":9815,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[1,776,91],"tags":[1095,71,100,1059,1093,1005,92,546,789,850,537,50,1096,68,977,1094],"class_list":["post-9813","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-transformation2","category-etl","category-iri-workbench","tag-action-key","tag-eclipse","tag-etl","tag-field-editor","tag-flow","tag-flow-diagram","tag-gui","tag-iri-cosort","tag-iri-voracity","tag-iri-workbench","tag-job-script","tag-oracle","tag-sort-key","tag-sortcl","tag-sql","tag-transform-mapping-diagram"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v23.4 (Yoast SEO v23.4) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Creating a Voracity Flow Using Existing IRI Scripts (Part 1 of 3) - IRI<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Creating a Voracity Flow Using Existing IRI Scripts (Part 1 of 3)\" \/>\n<meta property=\"og:description\" content=\"This is first in a series of articles explaining how to create and use Flows in the IRI Workbench\u00a0GUI for Voracity. Flows contain ETL and other data processing steps, and are illustrated in flow and transform mapping diagrams in the GUI. They are saved in .flow files for use in the GUI, and can beRead More\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/\" \/>\n<meta property=\"og:site_name\" content=\"IRI\" \/>\n<meta property=\"article:published_time\" content=\"2016-03-20T15:46:24+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2017-11-06T18:46:55+00:00\" \/>\n<meta name=\"author\" content=\"Susan Gegner\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Susan Gegner\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/\"},\"author\":{\"name\":\"Susan Gegner\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/87be5da567628ab9396ca81170f36d63\"},\"headline\":\"Creating a Voracity Flow Using Existing IRI Scripts (Part 1 of 3)\",\"datePublished\":\"2016-03-20T15:46:24+00:00\",\"dateModified\":\"2017-11-06T18:46:55+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/\"},\"wordCount\":1428,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2016\/06\/Creating-Voracity-Flows-Using-Existing-IRI-Scripts-Part-1-Susan-Gegner-0.jpg\",\"keywords\":[\"action key\",\"Eclipse\",\"ETL\",\"field editor\",\"flow\",\"flow diagram\",\"GUI\",\"IRI CoSort\",\"IRI Voracity\",\"IRI Workbench\",\"job script\",\"Oracle\",\"sort key\",\"SortCL\",\"SQL\",\"transform mapping diagram\"],\"articleSection\":[\"Data Transformation\",\"ETL\",\"IRI Workbench\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/\",\"url\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/\",\"name\":\"Creating a Voracity Flow Using Existing IRI Scripts (Part 1 of 3) - IRI\",\"isPartOf\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2016\/06\/Creating-Voracity-Flows-Using-Existing-IRI-Scripts-Part-1-Susan-Gegner-0.jpg\",\"datePublished\":\"2016-03-20T15:46:24+00:00\",\"dateModified\":\"2017-11-06T18:46:55+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/#primaryimage\",\"url\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2016\/06\/Creating-Voracity-Flows-Using-Existing-IRI-Scripts-Part-1-Susan-Gegner-0.jpg\",\"contentUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2016\/06\/Creating-Voracity-Flows-Using-Existing-IRI-Scripts-Part-1-Susan-Gegner-0.jpg\",\"width\":624,\"height\":390},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.iri.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Creating a Voracity Flow Using Existing IRI Scripts (Part 1 of 3)\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.iri.com\/blog\/#website\",\"url\":\"https:\/\/www.iri.com\/blog\/\",\"name\":\"IRI\",\"description\":\"Total Data Management Blog\",\"publisher\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.iri.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\",\"name\":\"IRI\",\"url\":\"https:\/\/www.iri.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"contentUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"width\":750,\"height\":206,\"caption\":\"IRI\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/87be5da567628ab9396ca81170f36d63\",\"name\":\"Susan Gegner\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/2b1ca5592a65d44483351292cf1ae00a?s=96&d=blank&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/2b1ca5592a65d44483351292cf1ae00a?s=96&d=blank&r=g\",\"caption\":\"Susan Gegner\"},\"url\":\"https:\/\/www.iri.com\/blog\/author\/susang\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Creating a Voracity Flow Using Existing IRI Scripts (Part 1 of 3) - IRI","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/","og_locale":"en_US","og_type":"article","og_title":"Creating a Voracity Flow Using Existing IRI Scripts (Part 1 of 3)","og_description":"This is first in a series of articles explaining how to create and use Flows in the IRI Workbench\u00a0GUI for Voracity. Flows contain ETL and other data processing steps, and are illustrated in flow and transform mapping diagrams in the GUI. They are saved in .flow files for use in the GUI, and can beRead More","og_url":"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/","og_site_name":"IRI","article_published_time":"2016-03-20T15:46:24+00:00","article_modified_time":"2017-11-06T18:46:55+00:00","author":"Susan Gegner","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Susan Gegner","Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/#article","isPartOf":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/"},"author":{"name":"Susan Gegner","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/87be5da567628ab9396ca81170f36d63"},"headline":"Creating a Voracity Flow Using Existing IRI Scripts (Part 1 of 3)","datePublished":"2016-03-20T15:46:24+00:00","dateModified":"2017-11-06T18:46:55+00:00","mainEntityOfPage":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/"},"wordCount":1428,"commentCount":0,"publisher":{"@id":"https:\/\/www.iri.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/#primaryimage"},"thumbnailUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2016\/06\/Creating-Voracity-Flows-Using-Existing-IRI-Scripts-Part-1-Susan-Gegner-0.jpg","keywords":["action key","Eclipse","ETL","field editor","flow","flow diagram","GUI","IRI CoSort","IRI Voracity","IRI Workbench","job script","Oracle","sort key","SortCL","SQL","transform mapping diagram"],"articleSection":["Data Transformation","ETL","IRI Workbench"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/","url":"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/","name":"Creating a Voracity Flow Using Existing IRI Scripts (Part 1 of 3) - IRI","isPartOf":{"@id":"https:\/\/www.iri.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/#primaryimage"},"image":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/#primaryimage"},"thumbnailUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2016\/06\/Creating-Voracity-Flows-Using-Existing-IRI-Scripts-Part-1-Susan-Gegner-0.jpg","datePublished":"2016-03-20T15:46:24+00:00","dateModified":"2017-11-06T18:46:55+00:00","breadcrumb":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/#primaryimage","url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2016\/06\/Creating-Voracity-Flows-Using-Existing-IRI-Scripts-Part-1-Susan-Gegner-0.jpg","contentUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2016\/06\/Creating-Voracity-Flows-Using-Existing-IRI-Scripts-Part-1-Susan-Gegner-0.jpg","width":624,"height":390},{"@type":"BreadcrumbList","@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/creating-voracity-flows-using-existing-iri-scripts-part-1\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.iri.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Creating a Voracity Flow Using Existing IRI Scripts (Part 1 of 3)"}]},{"@type":"WebSite","@id":"https:\/\/www.iri.com\/blog\/#website","url":"https:\/\/www.iri.com\/blog\/","name":"IRI","description":"Total Data Management Blog","publisher":{"@id":"https:\/\/www.iri.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.iri.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.iri.com\/blog\/#organization","name":"IRI","url":"https:\/\/www.iri.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","contentUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","width":750,"height":206,"caption":"IRI"},"image":{"@id":"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/87be5da567628ab9396ca81170f36d63","name":"Susan Gegner","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/2b1ca5592a65d44483351292cf1ae00a?s=96&d=blank&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/2b1ca5592a65d44483351292cf1ae00a?s=96&d=blank&r=g","caption":"Susan Gegner"},"url":"https:\/\/www.iri.com\/blog\/author\/susang\/"}]}},"jetpack_featured_media_url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2016\/06\/Creating-Voracity-Flows-Using-Existing-IRI-Scripts-Part-1-Susan-Gegner-0.jpg","_links":{"self":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/9813"}],"collection":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/comments?post=9813"}],"version-history":[{"count":19,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/9813\/revisions"}],"predecessor-version":[{"id":11461,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/9813\/revisions\/11461"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/media\/9815"}],"wp:attachment":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/media?parent=9813"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/categories?post=9813"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/tags?post=9813"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}