{"id":15950,"date":"2022-06-28T12:33:29","date_gmt":"2022-06-28T16:33:29","guid":{"rendered":"https:\/\/www.iri.com\/blog\/?p=15950"},"modified":"2025-01-27T15:24:53","modified_gmt":"2025-01-27T20:24:53","slug":"real-time-database-data-replication","status":"publish","type":"post","link":"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/","title":{"rendered":"Real-Time Database Data Replication"},"content":{"rendered":"<p><b><i>Abstract:\u00a0 <\/i><\/b><i><span style=\"font-weight: 400;\">The previous <\/span><\/i><a href=\"https:\/\/www.iri.com\/blog\/vldb-operations\/getting-started-with-iri-ripcurrent\"><i><span style=\"font-weight: 400;\">article<\/span><\/i><\/a><i><span style=\"font-weight: 400;\"> in this series of 4 detailed how to install, set up, and configure IRI Ripcurrent, an IRI-developed command-line Java application that utilizes the <\/span><\/i><a href=\"https:\/\/debezium.io\/documentation\/reference\/1.9\/development\/engine.html\"><i><span style=\"font-weight: 400;\">Debezium embedded engine<\/span><\/i><\/a><i><span style=\"font-weight: 400;\"> and the streaming feature of the <\/span><\/i><a href=\"https:\/\/www.iri.com\/products\/cosort\/sortcl\"><i><span style=\"font-weight: 400;\">IRI (CoSort) SortCL program<\/span><\/i><\/a><i><span style=\"font-weight: 400;\"> to react in real-time to database change events by replicating data to downstream target(s), optionally with transformation (e.g., PII masking) rules consistently applied based on classification of the data.\u00a0<\/span><\/i><\/p>\n<p><i><span style=\"font-weight: 400;\">One capability of Ripcurrent is real-time data replication. Ripcurrent can monitor and react to the insertion, deletion, or update of any rows in the source database by replicating that data to target silo(s). Real-time replication of data through Ripcurrent will be demonstrated in this article through example.<\/span><\/i><\/p>\n<h4><b>Demonstration Setup<\/b><\/h4>\n<p><span style=\"font-weight: 400;\">For this example, I am using a MySQL database as the source database being monitored; <\/span><a href=\"https:\/\/debezium.io\/\"><span style=\"font-weight: 400;\">Debezium <\/span><\/a><span style=\"font-weight: 400;\">supports the monitoring of several other different types of databases, including PostgreSQL, Oracle, SQL Server, and DB2. The MySQL database must first be set up properly for Debezium to be able to monitor it.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">See <\/span><a href=\"https:\/\/debezium.io\/documentation\/\"><span style=\"font-weight: 400;\">Debezium documentation<\/span><\/a><span style=\"font-weight: 400;\"> for detailed instructions on how to set up a database for monitoring of changes. The process varies depending on the database vendor.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In the case of MySQL, this involves enabling binary logging for MySQL replication. The binary logs record transaction updates that allow replication tools to propagate changes.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">I have already installed the Ripcurrent feature for <\/span><a href=\"https:\/\/www.iri.com\/products\/workbench\"><span style=\"font-weight: 400;\">IRI Workbench<\/span><\/a><span style=\"font-weight: 400;\"> and set up a configuration file as specified in the previous <\/span><a href=\"https:\/\/www.iri.com\/blog\/vldb-operations\/getting-started-with-iri-ripcurrent\"><span style=\"font-weight: 400;\">article<\/span><\/a><span style=\"font-weight: 400;\">. I have also set up tables in the target schema with the same structure as the tables I am monitoring in the source database.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Ripcurrent is suited for many different use cases where data is being added and updated to a database frequently (e.g. orders\/transactions, sensor data from IoT devices like weather conditions, location data from transportation vehicles, patient health data).\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In this case, I am demonstrating an example of data replication with financial transactions. The source table is being updated with new transactions as they occur. Ripcurrent can replicate these new transactions to file and table targets as they happen in near real time.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Of course, there is always some latency in copying data from source to targets, but this is usually measured in sub-seconds unless a large amount of data has been modified or added to the table at once.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The image below shows a sample of data in this table.<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-15953 aligncenter\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/transactions-300x93.png\" alt=\"\" width=\"1163\" height=\"361\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/transactions-300x93.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/transactions-768x237.png 768w\" sizes=\"(max-width: 1163px) 100vw, 1163px\" \/><\/p>\n<h4><b>Generating a Properties File<\/b><\/h4>\n<p><span style=\"font-weight: 400;\">To begin with, a properties file should be created to define <\/span><a href=\"https:\/\/docs.google.com\/document\/d\/1UkFY5pQ8UkveREeKZOcpO-bec1kA6ID6EGj21Lk_kgQ\/edit#heading=h.57a6puhe3pnn\"><span style=\"font-weight: 400;\">configurations <\/span><\/a><span style=\"font-weight: 400;\">required for Ripcurrent. After installing Ripcurrent in IRI Workbench, the Ripcurrent Properties Generator wizard is available in the IRI Voracity (orca icon) menu to assist in this task.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">On the first page of the wizard, I selected a MySQL database as the source to monitor:<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-15955 aligncenter\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/ripcurrent-article-data-replication-source-config-300x219.png\" alt=\"\" width=\"584\" height=\"426\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/ripcurrent-article-data-replication-source-config-300x219.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/ripcurrent-article-data-replication-source-config.png 606w\" sizes=\"(max-width: 584px) 100vw, 584px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">On the second page, I selected the option to replicate changes to both files in a directory and database tables in a specified schema. <\/span><i><span style=\"font-weight: 400;\">\u00a0<\/span><\/i><span style=\"font-weight: 400;\">I chose a SQL Server database as my target and delimited flat files as a secondary target. I entered <\/span><b><i>\\t<\/i><\/b><span style=\"font-weight: 400;\"> as the target separator to make the flat file output tab-separated.<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-15956 aligncenter\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-103551-300x219.png\" alt=\"\" width=\"585\" height=\"427\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-103551-300x219.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-103551.png 606w\" sizes=\"(max-width: 585px) 100vw, 585px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">In the case of direct data replication without any transformations, there is no need to select anything on the third page, which pertains to selecting IRI libraries to use with Ripcurrent that defines how to classify and transform data.<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-15957 aligncenter\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/ripcurrent-article-data-replication-iri-library-config-300x219.png\" alt=\"\" width=\"580\" height=\"424\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/ripcurrent-article-data-replication-iri-library-config-300x219.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/ripcurrent-article-data-replication-iri-library-config.png 606w\" sizes=\"(max-width: 580px) 100vw, 580px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">On the final page, I entered a name for the connector and server, and a path for database structure change events to be logged to. For the most part, the connector name and server name can be anything as long as they are unique. They may show in Ripcurrent logs, and also are used for identification.<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-15958 aligncenter\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/ripcurrent-article-data-replication-miscellaneous-config-300x219.png\" alt=\"\" width=\"586\" height=\"428\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/ripcurrent-article-data-replication-miscellaneous-config-300x219.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/ripcurrent-article-data-replication-miscellaneous-config.png 606w\" sizes=\"(max-width: 586px) 100vw, 586px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">There are a few restrictions on the format of the server name; the <\/span><a href=\"https:\/\/docs.google.com\/document\/d\/1UkFY5pQ8UkveREeKZOcpO-bec1kA6ID6EGj21Lk_kgQ\/edit?usp=sharing\"><span style=\"font-weight: 400;\">Ripcurrent documentation<\/span><\/a><span style=\"font-weight: 400;\"> discusses Ripcurrent configuration properties in detail. The server ID is not required; I did not specify a value in this case.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">After successful completion of the wizard, a properties file will be generated and opened in the editor:<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-15973 aligncenter\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/2ripcurrent-article-data-replication-properties-file-300x141.png\" alt=\"\" width=\"711\" height=\"334\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/2ripcurrent-article-data-replication-properties-file-300x141.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/2ripcurrent-article-data-replication-properties-file.png 645w\" sizes=\"(max-width: 711px) 100vw, 711px\" \/><\/p>\n<h4><b>Demonstrating Real-Time Data Replication with Ripcurrent<\/b><\/h4>\n<p><span style=\"font-weight: 400;\">Ripcurrent can be started by running the <\/span><i><span style=\"font-weight: 400;\">ripcurrent.bat<\/span><\/i><span style=\"font-weight: 400;\"> script in the <\/span><i><span style=\"font-weight: 400;\">bin<\/span><\/i><span style=\"font-weight: 400;\"> folder of the Ripcurrent distribution, since I am using a Windows operating system. This script can be triggered through a terminal or through an IRI Workbench <\/span><i><span style=\"font-weight: 400;\">external tools run configuration<\/span><\/i><span style=\"font-weight: 400;\"> (see below).<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-15960 aligncenter\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/ripcurrent-run-configuration-1-300x231.png\" alt=\"\" width=\"771\" height=\"594\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/ripcurrent-run-configuration-1-300x231.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/ripcurrent-run-configuration-1-768x592.png 768w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/ripcurrent-run-configuration-1.png 850w\" sizes=\"(max-width: 771px) 100vw, 771px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">Ripcurrent runs in the background, monitoring events from Debezium until explicitly terminated.\u00a0<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-15961 aligncenter\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-103828-300x103.png\" alt=\"\" width=\"1323\" height=\"454\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-103828-300x103.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-103828-1024x352.png 1024w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-103828-768x264.png 768w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-103828.png 1216w\" sizes=\"(max-width: 1323px) 100vw, 1323px\" \/><\/p>\n<p style=\"text-align: center;\"><i><span style=\"font-weight: 400;\">Snapshot of running Ripcurrent from the <\/span><\/i><b><i>External Tools Configuration<\/i><\/b><i><span style=\"font-weight: 400;\"> in IRI Workbench. <\/span><\/i><i><span style=\"font-weight: 400;\"><br \/>\n<\/span><\/i><i><span style=\"font-weight: 400;\">Logging output has been configured to display on the console.<\/span><\/i><\/p>\n<p><span style=\"font-weight: 400;\">Each event received from Debezium when a database table has data added or modified includes information about the new data values, the data types, and the names for each column.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The data received is used as the input to a dynamically generated SortCL job based on the information from the Debezium event. New jobs are created when there is an event from a table not seen before, or there is an event from a table seen before but there is no existing job that has the same layout of columns from the event and the same type of event (e.g. update, insert, delete).\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">With Ripcurrent running, I edited values in the source table through the <\/span><i><span style=\"font-weight: 400;\">Data Source Explorer <\/span><\/i><span style=\"font-weight: 400;\">in IRI Workbench to trigger change events.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The new data in the source table is mirrored to a corresponding table in the target SQL Server schema, as well as a file in a specified directory. Ripcurrent will replicate data to additional tables or files for each event that is received from a unique table in the source database.<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-15974 aligncenter\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/TRF-TO-Transaction-300x174.png\" alt=\"\" width=\"929\" height=\"539\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/TRF-TO-Transaction-300x174.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/TRF-TO-Transaction-768x447.png 768w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/TRF-TO-Transaction.png 827w\" sizes=\"(max-width: 929px) 100vw, 929px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">First, I inserted one row (shown highlighted) into the source transactions table in MySQL.<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-15963 aligncenter\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-104414-300x12.png\" alt=\"\" width=\"825\" height=\"33\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-104414-300x12.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-104414.png 730w\" sizes=\"(max-width: 825px) 100vw, 825px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">This was the first event received from the table, so a new replication job was started.<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-15965 aligncenter\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-112303-300x46.png\" alt=\"\" width=\"711\" height=\"109\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-112303-300x46.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-112303-768x117.png 768w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-112303.png 819w\" sizes=\"(max-width: 711px) 100vw, 711px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">As a result of the replication job, the data added to the source table was duplicated to a tab-delimited file in the target directory.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Additionally, the data was replicated to the target table.<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-15967 aligncenter\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-112457-300x42.png\" alt=\"\" width=\"664\" height=\"93\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-112457-300x42.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-112457.png 721w\" sizes=\"(max-width: 664px) 100vw, 664px\" \/><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-15966 aligncenter\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-112751-300x19.png\" alt=\"\" width=\"873\" height=\"55\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-112751-300x19.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-112751-1024x64.png 1024w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-112751-768x48.png 768w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-112751.png 1062w\" sizes=\"(max-width: 873px) 100vw, 873px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">Next, I modified the account number of the record previously inserted into the source table. Note that updates and deletes of data from the source database can only be reflected in database targets, not file targets.<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-15969 aligncenter\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-112821-300x34.png\" alt=\"\" width=\"913\" height=\"104\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-112821-300x34.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-112821-1024x117.png 1024w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-112821-768x88.png 768w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-112821.png 1077w\" sizes=\"(max-width: 913px) 100vw, 913px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">This change is also captured by Ripcurrent and reflected in the target table.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Finally, I added several new transaction records to the source table (3051-3053):<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-15970 aligncenter\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-113618-300x64.png\" alt=\"\" width=\"1069\" height=\"228\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-113618-300x64.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-113618-1024x219.png 1024w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-113618-768x165.png 768w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-113618.png 1087w\" sizes=\"(max-width: 1069px) 100vw, 1069px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">Ripcurrent captures these additional changes and duplicates them to the target table.<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-15971 aligncenter\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-114048-300x90.png\" alt=\"\" width=\"1018\" height=\"306\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-114048-300x90.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-114048-1024x307.png 1024w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-114048-768x230.png 768w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-05-26-114048.png 1183w\" sizes=\"(max-width: 1018px) 100vw, 1018px\" \/><\/p>\n<h4><b>Conclusion<\/b><\/h4>\n<p><span style=\"font-weight: 400;\">This article demonstrated one use case for Ripcurrent \u2013 the replication of changed database data directly into target files and\/or tables. The <a href=\"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/\">next (third) article<\/a> in this series will augment this demonstration with the use of data classification and masking rules to categorize and mask data on a consistent basis before it is replicated to your targets.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Abstract:\u00a0 The previous article in this series of 4 detailed how to install, set up, and configure IRI Ripcurrent, an IRI-developed command-line Java application that utilizes the Debezium embedded engine and the streaming feature of the IRI (CoSort) SortCL program to react in real-time to database change events by replicating data to downstream target(s), optionally<\/p>\n<div><a class=\"btn-filled btn\" href=\"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/\" title=\"Real-Time Database Data Replication\">Read More<\/a><\/div>\n","protected":false},"author":119,"featured_media":15978,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[108,31,1,3],"tags":[1305,77,51,2011,2012,1696,2013,1698,1697,2014],"class_list":["post-15950","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-big-data-2","category-data-migration","category-data-transformation2","category-vldb-operations","tag-data-masking-tools","tag-data-migration-2","tag-data-replication","tag-data-replication-tools","tag-database-replication","tag-iri-ripcurrent","tag-lower-environment","tag-real-time-db-data-replication","tag-ripcurrent","tag-test-data-refresh"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v23.4 (Yoast SEO v23.4) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Real-Time Database Data Replication - IRI<\/title>\n<meta name=\"description\" content=\"Refresh (and mask the PII in) new\/changed rows in your lower environment via IRI Ripcurrent, the real-time data replication tool in Voracity.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Real-Time Database Data Replication\" \/>\n<meta property=\"og:description\" content=\"Refresh (and mask the PII in) new\/changed rows in your lower environment via IRI Ripcurrent, the real-time data replication tool in Voracity.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/\" \/>\n<meta property=\"og:site_name\" content=\"IRI\" \/>\n<meta property=\"article:published_time\" content=\"2022-06-28T16:33:29+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-01-27T20:24:53+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Realtime-DB-Image-Feature.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Devon Kozenieski\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Devon Kozenieski\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"9 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/\"},\"author\":{\"name\":\"Devon Kozenieski\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/de972c035aaeecfc40a3ae2ea5ff7ba1\"},\"headline\":\"Real-Time Database Data Replication\",\"datePublished\":\"2022-06-28T16:33:29+00:00\",\"dateModified\":\"2025-01-27T20:24:53+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/\"},\"wordCount\":1122,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Realtime-DB-Image-Feature.png\",\"keywords\":[\"data masking tools\",\"data migration\",\"data replication\",\"data replication tools\",\"database replication\",\"IRI Ripcurrent\",\"lower environment\",\"Real-time db data replication\",\"Ripcurrent\",\"test data refresh\"],\"articleSection\":[\"Big Data\",\"Data Migration\",\"Data Transformation\",\"VLDB\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/\",\"url\":\"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/\",\"name\":\"Real-Time Database Data Replication - IRI\",\"isPartOf\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Realtime-DB-Image-Feature.png\",\"datePublished\":\"2022-06-28T16:33:29+00:00\",\"dateModified\":\"2025-01-27T20:24:53+00:00\",\"description\":\"Refresh (and mask the PII in) new\/changed rows in your lower environment via IRI Ripcurrent, the real-time data replication tool in Voracity.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/#primaryimage\",\"url\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Realtime-DB-Image-Feature.png\",\"contentUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Realtime-DB-Image-Feature.png\",\"width\":1200,\"height\":628},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.iri.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Real-Time Database Data Replication\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.iri.com\/blog\/#website\",\"url\":\"https:\/\/www.iri.com\/blog\/\",\"name\":\"IRI\",\"description\":\"Total Data Management Blog\",\"publisher\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.iri.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\",\"name\":\"IRI\",\"url\":\"https:\/\/www.iri.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"contentUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"width\":750,\"height\":206,\"caption\":\"IRI\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/de972c035aaeecfc40a3ae2ea5ff7ba1\",\"name\":\"Devon Kozenieski\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/e4c421588c1a85dd9a76146fe15528f7?s=96&d=blank&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/e4c421588c1a85dd9a76146fe15528f7?s=96&d=blank&r=g\",\"caption\":\"Devon Kozenieski\"},\"url\":\"https:\/\/www.iri.com\/blog\/author\/devonk\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Real-Time Database Data Replication - IRI","description":"Refresh (and mask the PII in) new\/changed rows in your lower environment via IRI Ripcurrent, the real-time data replication tool in Voracity.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/","og_locale":"en_US","og_type":"article","og_title":"Real-Time Database Data Replication","og_description":"Refresh (and mask the PII in) new\/changed rows in your lower environment via IRI Ripcurrent, the real-time data replication tool in Voracity.","og_url":"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/","og_site_name":"IRI","article_published_time":"2022-06-28T16:33:29+00:00","article_modified_time":"2025-01-27T20:24:53+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Realtime-DB-Image-Feature.png","type":"image\/png"}],"author":"Devon Kozenieski","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Devon Kozenieski","Est. reading time":"9 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/#article","isPartOf":{"@id":"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/"},"author":{"name":"Devon Kozenieski","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/de972c035aaeecfc40a3ae2ea5ff7ba1"},"headline":"Real-Time Database Data Replication","datePublished":"2022-06-28T16:33:29+00:00","dateModified":"2025-01-27T20:24:53+00:00","mainEntityOfPage":{"@id":"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/"},"wordCount":1122,"commentCount":0,"publisher":{"@id":"https:\/\/www.iri.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/#primaryimage"},"thumbnailUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Realtime-DB-Image-Feature.png","keywords":["data masking tools","data migration","data replication","data replication tools","database replication","IRI Ripcurrent","lower environment","Real-time db data replication","Ripcurrent","test data refresh"],"articleSection":["Big Data","Data Migration","Data Transformation","VLDB"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/","url":"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/","name":"Real-Time Database Data Replication - IRI","isPartOf":{"@id":"https:\/\/www.iri.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/#primaryimage"},"image":{"@id":"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/#primaryimage"},"thumbnailUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Realtime-DB-Image-Feature.png","datePublished":"2022-06-28T16:33:29+00:00","dateModified":"2025-01-27T20:24:53+00:00","description":"Refresh (and mask the PII in) new\/changed rows in your lower environment via IRI Ripcurrent, the real-time data replication tool in Voracity.","breadcrumb":{"@id":"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/#primaryimage","url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Realtime-DB-Image-Feature.png","contentUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Realtime-DB-Image-Feature.png","width":1200,"height":628},{"@type":"BreadcrumbList","@id":"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.iri.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Real-Time Database Data Replication"}]},{"@type":"WebSite","@id":"https:\/\/www.iri.com\/blog\/#website","url":"https:\/\/www.iri.com\/blog\/","name":"IRI","description":"Total Data Management Blog","publisher":{"@id":"https:\/\/www.iri.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.iri.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.iri.com\/blog\/#organization","name":"IRI","url":"https:\/\/www.iri.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","contentUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","width":750,"height":206,"caption":"IRI"},"image":{"@id":"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/de972c035aaeecfc40a3ae2ea5ff7ba1","name":"Devon Kozenieski","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/e4c421588c1a85dd9a76146fe15528f7?s=96&d=blank&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e4c421588c1a85dd9a76146fe15528f7?s=96&d=blank&r=g","caption":"Devon Kozenieski"},"url":"https:\/\/www.iri.com\/blog\/author\/devonk\/"}]}},"jetpack_featured_media_url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Realtime-DB-Image-Feature.png","_links":{"self":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/15950"}],"collection":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/users\/119"}],"replies":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/comments?post=15950"}],"version-history":[{"count":10,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/15950\/revisions"}],"predecessor-version":[{"id":18228,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/15950\/revisions\/18228"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/media\/15978"}],"wp:attachment":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/media?parent=15950"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/categories?post=15950"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/tags?post=15950"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}