{"id":15979,"date":"2022-06-28T12:33:49","date_gmt":"2022-06-28T16:33:49","guid":{"rendered":"https:\/\/www.iri.com\/blog\/?p=15979"},"modified":"2025-11-12T06:13:55","modified_gmt":"2025-11-12T11:13:55","slug":"real-time-incremental-data-masking","status":"publish","type":"post","link":"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/","title":{"rendered":"Real-Time, Incremental Data Masking"},"content":{"rendered":"<p><em><b>Abstract: <\/b><\/em><em><span style=\"font-weight: 400;\">The <a href=\"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\">previous article<\/a> in this series of 4 demonstrated real-time database replication with IRI Ripcurrent, an IRI-developed command-line Java application that utilizes the <\/span><a href=\"https:\/\/debezium.io\/documentation\/reference\/1.9\/development\/engine.html\"><span style=\"font-weight: 400;\">Debezium embedded engine<\/span><\/a><span style=\"font-weight: 400;\"> and the streaming features of the <\/span><a href=\"https:\/\/www.iri.com\/products\/cosort\/sortcl\"><span style=\"font-weight: 400;\">IRI (CoSort) SortCL program<\/span><\/a><span style=\"font-weight: 400;\">. Together they react in real-time to database change events.<\/span><\/em><\/p>\n<p><em><span style=\"font-weight: 400;\">The capability demonstrated in the <\/span><a href=\"https:\/\/www.iri.com\/blog\/migration\/data-migration\/real-time-database-data-replication\"><span style=\"font-weight: 400;\">previous article<\/span><\/a><span style=\"font-weight: 400;\"> can also be augmented with the classification and masking of sensitive data in transit to file or database targets. When data in the source table is inserted or updated, it is moved and masked in real-time to the designated target table using rules defined in the <a href=\"https:\/\/www.iri.com\/fieldshield\">IRI FieldShield<\/a> data masking tool. When data in the source table is deleted, the same row(s) will be deleted in the (different) target table.<\/span><\/em><\/p>\n<h4><b>Prerequisites<\/b><\/h4>\n<p><span style=\"font-weight: 400;\">Assuming the prerequisites detailed in the <\/span><a href=\"https:\/\/www.iri.com\/blog\/vldb-operations\/getting-started-with-iri-ripcurrent\"><span style=\"font-weight: 400;\">first article<\/span><\/a><span style=\"font-weight: 400;\"> have been met, for this case an <a href=\"https:\/\/www.iri.com\/blog\/data-protection\/iri-data-classification\/\">IRI Data Class &amp; Rule Library<\/a> are also required.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">For Ripcurrent, a default rule is required to be assigned for each data class. The default rule is used by Ripcurrent to apply a chosen <\/span><a href=\"https:\/\/www.iri.com\/solutions\/data-masking\/static-data-masking\"><span style=\"font-weight: 400;\">masking function<\/span><\/a><span style=\"font-weight: 400;\"> to data or a column name matching the specified data class search parameters.<\/span><\/p>\n<h4 style=\"text-align: left;\"><b>Generating a Ripcurrent Properties File<\/b><\/h4>\n<p><span style=\"font-weight: 400;\">Before running Ripcurrent, a property file should be created to define configuration options that are required for Ripcurrent to run.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">After installing the Ripcurrent feature in IRI Workbench, the <\/span><i><span style=\"font-weight: 400;\">Ripcurrent Properties Generator <\/span><\/i><span style=\"font-weight: 400;\">wizard is available from the Voracity menu to assist in generating a properties file for Ripcurrent.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">On the first page of the wizard, options can be specified pertaining to the database to be monitored. In this image, an Oracle connection was selected and a filter was specified to narrow down the tables to be monitored to only those in the <\/span><i><span style=\"font-weight: 400;\">SCOTT<\/span><\/i><span style=\"font-weight: 400;\"> schema.<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-15992 aligncenter\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-01-122037-300x217.png\" alt=\"\" width=\"614\" height=\"444\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-01-122037-300x217.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-01-122037.png 613w\" sizes=\"(max-width: 614px) 100vw, 614px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">In this case, the DB name (required to set up a configuration to monitor an Oracle database) was also entered, <\/span><b><i>XE<\/i><\/b><span style=\"font-weight: 400;\"> in this case.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">On the second page of the wizard, there are options to replicate to database tables, files, or <\/span><b><i>both <\/i><\/b><span style=\"font-weight: 400;\">files and database tables. The image shows the choice of replicating data changes from the source Oracle database to a MySQL database with existing tables of the same name and structure as the source database, as well as to Excel files.\u00a0<\/span><\/p>\n<p style=\"text-align: center;\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-15993\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-01-155508-300x217.png\" alt=\"\" width=\"615\" height=\"445\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-01-155508-300x217.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-01-155508.png 613w\" sizes=\"(max-width: 615px) 100vw, 615px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">On the third page, locations of an IRI data class library and rule library can be specified. The image shows the selection of the library files created earlier in the article:<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-15994 aligncenter\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-01-155346-300x217.png\" alt=\"\" width=\"617\" height=\"446\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-01-155346-300x217.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-01-155346.png 613w\" sizes=\"(max-width: 617px) 100vw, 617px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">On the final page, the image shows the selection of a log location for structural change event information from the source database. Names for the connector and server connection used by Debezium are also specified.\u00a0<\/span><\/p>\n<p style=\"text-align: center;\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-15995\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-01-155409-300x217.png\" alt=\"\" width=\"616\" height=\"445\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-01-155409-300x217.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-01-155409.png 613w\" sizes=\"(max-width: 616px) 100vw, 616px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">After the properties file was successfully generated for my configurations, I began the execution of Ripcurrent by running the <\/span><b><i>ripcurrent.bat<\/i><\/b><span style=\"font-weight: 400;\"> script. After completion of an initial snapshot of the source database (if running for the first time), Ripcurrent will be monitoring for changes from the source database.<\/span><\/p>\n<h4><b>Adding Test Data to Source Tables with IRI RowGen<\/b><\/h4>\n<p><span style=\"font-weight: 400;\">Next, I will run a batch file generated by the <\/span><a href=\"https:\/\/www.iri.com\/blog\/vldb-operations\/how-to-generate-db-test-data-2\/\"><i><span style=\"font-weight: 400;\">New Database Test Data<\/span><\/i><\/a><span style=\"font-weight: 400;\"> wizard, accessible from the <\/span><a href=\"https:\/\/www.iri.com\/products\/rowgen\"><span style=\"font-weight: 400;\">IRI RowGen<\/span><\/a><span style=\"font-weight: 400;\"> menu to synthesize and simulate new data coming into the source tables. The New Database Test Data wizard synthesizes data consistently by classifying database columns and pairing them with generation rules.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The image below shows an entity-relationship mapping diagram of the test data batch operation, produced in the <a href=\"https:\/\/www.iri.com\/products\/workbench\">IRI Workbench<\/a> GUI for all IRI software.<\/span><\/p>\n<p style=\"text-align: center;\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-15996\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/saved-example-300x172.png\" alt=\"\" width=\"924\" height=\"530\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/saved-example-300x172.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/saved-example-1024x586.png 1024w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/saved-example-768x440.png 768w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/saved-example-1536x879.png 1536w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/saved-example.png 1600w\" sizes=\"(max-width: 924px) 100vw, 924px\" \/><\/p>\n<h4><b>Replication of Data from Source Tables to Multiple Targets, with Consistent Masking Rules Applied to Classified Data<\/b><\/h4>\n<p><span style=\"font-weight: 400;\">After running the batch file to generate data for 5 tables in the source database, Ripcurrent recognized the new data inserted and triggered a replication and masking operation to the targets.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">This is a sample of the data that was generated in one of the<\/span><b> source<\/b><span style=\"font-weight: 400;\"> Oracle database tables (named DM_EMPLOYEES):<\/span><\/p>\n<p style=\"text-align: center;\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-16000\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-02-144854-300x103.png\" alt=\"\" width=\"700\" height=\"240\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-02-144854-300x103.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-02-144854.png 824w\" sizes=\"(max-width: 700px) 100vw, 700px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">Ripcurrent was silently triggered by this insertion event, and automatically masked and moved the new data into its defined targets, per this event log:<\/span><\/p>\n<p style=\"text-align: center;\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-16001\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-01-165416-300x82.png\" alt=\"\" width=\"721\" height=\"197\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-01-165416-300x82.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-01-165416-768x209.png 768w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-01-165416.png 797w\" sizes=\"(max-width: 721px) 100vw, 721px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">See that a series of Excel files have also been created in the target directory:<\/span><\/p>\n<p style=\"text-align: center;\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-16002\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-01-165444-300x69.png\" alt=\"\" width=\"912\" height=\"210\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-01-165444-300x69.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-01-165444-1024x235.png 1024w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-01-165444-768x176.png 768w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-01-165444.png 1197w\" sizes=\"(max-width: 912px) 100vw, 912px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">In the Excel file associated with the DM_EMPLOYEES table in the source database, notice that the email values have been modified from the original value.<\/span><\/p>\n<p style=\"text-align: center;\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-16004\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-02-144656-300x121.png\" alt=\"\" width=\"860\" height=\"347\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-02-144656-300x121.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-02-144656-768x311.png 768w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-02-144656.png 1282w\" sizes=\"(max-width: 860px) 100vw, 860px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">Additionally, in the target MySQL database, data was replicated to a table with the same name and structure as the source table, with the email address column encrypted to the same values as in the Excel file.<\/span><\/p>\n<p style=\"text-align: center;\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-16005\" src=\"\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-02-144510-300x65.png\" alt=\"\" width=\"862\" height=\"187\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-02-144510-300x65.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-02-144510-1024x223.png 1024w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-02-144510-768x167.png 768w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Screenshot-2022-06-02-144510.png 1073w\" sizes=\"(max-width: 862px) 100vw, 862px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">This column was classified as containing email addresses based on the regular expression pattern to match emails defined in the data class library. The original email values were encrypted with format-preserving encryption, defined by the default rule that was mapped to the EMAIL data class.<\/span><\/p>\n<h4><b>Conclusion<\/b><\/h4>\n<p><span style=\"font-weight: 400;\">This article demonstrated another use case of Ripcurrent that builds on the database data replication process demonstrated in the previous article \u2013 that is, consistent data classification, data masking and data replication as rows are inserted, updated or deleted in source tables and refresh into target tables (usually in a lower environment).\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">This approach ensures that data can be dynamically replicated to test schema, with sensitive data classified appropriately and consistently masked according to data classes and rules that can be flexibly defined.<\/span><\/p>\n<p><span style=\"font-weight: 400;\"> The <a href=\"https:\/\/www.iri.com\/blog\/vldb-operations\/using-ripcurrent-for-db-structure-change-reporting\/\">next and final article<\/a> in this series deals with real-time notifications of structural changes to the database which can impact the layout of the source or target tables. This would indicate the need to re-run data class search operations on your tables.<\/span><\/p>\n<section>\n<div class=\"container plain-width\">\n<div class=\"faq-section\">\n<h3>Frequently Asked Questions (FAQs)<\/h3>\n<div class=\"faq-item\">\n<div class=\"faq-question\">1. What is real time incremental data masking and what problem does it solve? <i class=\"faq-icon fa fa-plus\"><\/i><i class=\"faq-icon fa fa-minus\"><\/i><\/div>\n<div class=\"faq-answer\">Real time incremental data masking applies classification and masking to only the rows that change, as they change, while data is replicated from a source database to file or database targets. This protects sensitive values during continuous movement so lower environments and downstream systems are refreshed with safe, consistent data without waiting for batch jobs.<\/div>\n<\/div>\n<div class=\"faq-item\">\n<div class=\"faq-question\">2. How does Ripcurrent perform change capture and masking in this workflow? <i class=\"faq-icon fa fa-plus\"><\/i><i class=\"faq-icon fa fa-minus\"><\/i><\/div>\n<div class=\"faq-answer\">Ripcurrent embeds Debezium to listen for inserts, updates, and deletes, then streams those change events through the IRI <a href=\"https:\/\/www.iri.com\/products\/cosort\/sortcl\">SortCL<\/a> program (powering FieldShield) where <a href=\"https:\/\/www.iri.com\/solutions\/data-masking\/static-data-masking\">pre-defined data masking rules<\/a> are applied. Masked rows are written to the defined targets, and deletes in the source are propagated to the target tables.<\/div>\n<\/div>\n<div class=\"faq-item\">\n<div class=\"faq-question\">3. What are the prerequisites to enable real time masking with Ripcurrent? <i class=\"faq-icon fa fa-plus\"><\/i><i class=\"faq-icon fa fa-minus\"><\/i><\/div>\n<div class=\"faq-answer\">You need: 1) the IRI Voracity <a href=\"https:\/\/www.iri.com\/products\/ripcurrent\">Ripcurrent feature<\/a> installed in your <a href=\"https:\/\/www.iri.com\/products\/workbench\">IRI Workbench IDE<\/a>; 2) a generated Ripcurrent properties file; 3) JDBC or other connections to the source and target systems; and, 4) an <a href=\"https:\/\/www.iri.com\/blog\/data-protection\/iri-data-classification\/\">IRI Data Class and Rule Library<\/a> with data masking functions assigned as rules to each active data class.<\/div>\n<\/div>\n<div class=\"faq-item\">\n<div class=\"faq-question\">4. How are masking rules selected and applied consistently across targets? <i class=\"faq-icon fa fa-plus\"><\/i><i class=\"faq-icon fa fa-minus\"><\/i><\/div>\n<div class=\"faq-answer\">Each data class, such as Email or SSN, has a default rule like format preserving encryption or pseudonymization. When Ripcurrent detects a field that matches a class, it applies that class\u2019s default (<a href=\"https:\/\/www.iri.com\/support\/data-education-center\/what-is-deterministic-data-masking\">deterministic data masking<\/a>) rule so the same source value becomes the same masked value in every target.<\/div>\n<\/div>\n<div class=\"faq-item\">\n<div class=\"faq-question\">5. What targets can receive replicated and masked data in this setup? <i class=\"faq-icon fa fa-plus\"><\/i><i class=\"faq-icon fa fa-minus\"><\/i><\/div>\n<div class=\"faq-answer\">You can replicate to relational databases that mirror the source table structures, to files such as Excel for operational sharing, or to both at once. Target selection is configured in the Ripcurrent properties wizard.<\/div>\n<\/div>\n<div class=\"faq-item\">\n<div class=\"faq-question\">6. How are the Ripcurrent properties created and what do they control? <i class=\"faq-icon fa fa-plus\"><\/i><i class=\"faq-icon fa fa-minus\"><\/i><\/div>\n<div class=\"faq-answer\">Use the Ripcurrent Properties Generator wizard in IRI Workbench to specify the source database, table filters, destinations, locations of the data class and rule libraries, connector names, server settings, and log paths. The wizard outputs the properties file that Ripcurrent uses at runtime.<\/div>\n<\/div>\n<div class=\"faq-item\">\n<div class=\"faq-question\">7. What happens when a source row is updated or deleted? <i class=\"faq-icon fa fa-plus\"><\/i><i class=\"faq-icon fa fa-minus\"><\/i><\/div>\n<div class=\"faq-answer\">For updates, only changed columns are reprocessed and masked, then written to targets. For deletes, Ripcurrent issues corresponding deletes in the target tables so downstream environments remain synchronized.<\/div>\n<\/div>\n<div class=\"faq-item\">\n<div class=\"faq-question\">8. How can I generate realistic change events to test the pipeline? <i class=\"faq-icon fa fa-plus\"><\/i><i class=\"faq-icon fa fa-minus\"><\/i><\/div>\n<div class=\"faq-answer\">Use the <a href=\"https:\/\/www.iri.com\/products\/rowgen\">IRI RowGen<\/a> wizard in IRI Workbench called <a href=\"https:\/\/www.iri.com\/blog\/vldb-operations\/how-to-generate-db-test-data-2\/\">New Database Test Data job<\/a> to synthesize structurally and referentially correct data. Inserting this data into source tables triggers Ripcurrent, which replicates and masks those rows into targets for end to end validation.<\/div>\n<\/div>\n<div class=\"faq-item\">\n<div class=\"faq-question\">9. What masking techniques are recommended for transactional consistency? <i class=\"faq-icon fa fa-plus\"><\/i><i class=\"faq-icon fa fa-minus\"><\/i><\/div>\n<div class=\"faq-answer\">Format preserving encryption is commonly used to maintain patterns and lengths, while deterministic pseudonymization preserves value consistency across rows and systems. Choose techniques that keep schema constraints valid and support referential integrity in test environments.<\/div>\n<\/div>\n<div class=\"faq-item\">\n<div class=\"faq-question\">10. How is latency handled and what performance factors matter? <i class=\"faq-icon fa fa-plus\"><\/i><i class=\"faq-icon fa fa-minus\"><\/i><\/div>\n<div class=\"faq-answer\">Latency depends on source CDC throughput, network I\/O, and transformation complexity. Limiting active data classes to those required, optimizing rule choices, and filtering to the specific schemas and tables of interest help maintain low end to end delay.<\/div>\n<\/div>\n<div class=\"faq-item\">\n<div class=\"faq-question\">11. What logging or audit artifacts are produced during streaming operations? <i class=\"faq-icon fa fa-plus\"><\/i><i class=\"faq-icon fa fa-minus\"><\/i><\/div>\n<div class=\"faq-answer\">Ripcurrent writes change event logs and structural change notifications, while FieldShield rule usage is traceable through the data class and rule library mappings. These artifacts support verification, troubleshooting, and compliance reporting.<\/div>\n<\/div>\n<div class=\"faq-item\">\n<div class=\"faq-question\">12. How do I control which tables and columns are monitored? <i class=\"faq-icon fa fa-plus\"><\/i><i class=\"faq-icon fa fa-minus\"><\/i><\/div>\n<div class=\"faq-answer\">Apply schema and table filters in the properties wizard and scope your data classes to only the columns or patterns that represent PII. Using precise classification reduces overhead and false positives.<\/div>\n<\/div>\n<div class=\"faq-item\">\n<div class=\"faq-question\">13. What happens if a masked value would violate a target constraint? <i class=\"faq-icon fa fa-plus\"><\/i><i class=\"faq-icon fa fa-minus\"><\/i><\/div>\n<div class=\"faq-answer\">If a rule would break a constraint, such as making a number non numeric, switch to a format preserving method for that class or add validation logic. Reprocess the affected events so targets stay consistent and load compliant data.<\/div>\n<\/div>\n<div class=\"faq-item\">\n<div class=\"faq-question\">14. How can I ensure deterministic masking across multiple targets and runs? <i class=\"faq-icon fa fa-plus\"><\/i><i class=\"faq-icon fa fa-minus\"><\/i><\/div>\n<div class=\"faq-answer\">Use stable keys and consistent rule configurations for classes that require repeatable outcomes. Keep encryption keys and rule definitions secure and version controlled so the same inputs always yield the same masked outputs.<\/div>\n<\/div>\n<div class=\"faq-item\">\n<div class=\"faq-question\">15. What is the best way to manage configuration changes over time? <i class=\"faq-icon fa fa-plus\"><\/i><i class=\"faq-icon fa fa-minus\"><\/i><\/div>\n<div class=\"faq-answer\">Store the Ripcurrent properties, data class library, and rule library in source control. When schemas evolve, regenerate properties with the wizard, rerun classification, and update default rules as needed, then restart Ripcurrent with the revised configuration.<\/div>\n<\/div>\n<div class=\"faq-item\">\n<div class=\"faq-question\">16. How does this approach support data privacy compliance goals? <i class=\"faq-icon fa fa-plus\"><\/i><i class=\"faq-icon fa fa-minus\"><\/i><\/div>\n<div class=\"faq-answer\">By classifying and masking sensitive fields in motion, the pipeline prevents exposure of raw PII in non production systems and shared files. The consistent application of rules and the presence of audit logs help demonstrate appropriate safeguards during reviews.<\/div>\n<\/div>\n<div class=\"faq-item\">\n<div class=\"faq-question\">17. What monitoring steps help keep the pipeline healthy? <i class=\"faq-icon fa fa-plus\"><\/i><i class=\"faq-icon fa fa-minus\"><\/i><\/div>\n<div class=\"faq-answer\">Track Ripcurrent console output and logs for connector status, check target row counts against source change volumes, and periodically validate a sample of masked fields for rule correctness. Add alerts for connector disconnects or schema drift.<\/div>\n<\/div>\n<div class=\"faq-item\">\n<div class=\"faq-question\">18. Can I run masking only on selected columns while pass through others? <i class=\"faq-icon fa fa-plus\"><\/i><i class=\"faq-icon fa fa-minus\"><\/i><\/div>\n<div class=\"faq-answer\">Yes. Only fields that match active data classes have rules applied. Non sensitive columns are streamed unchanged, which preserves performance and data utility.<\/div>\n<\/div>\n<div class=\"faq-item\">\n<div class=\"faq-question\">19. How do I handle encryption key rotation without breaking consistency? <i class=\"faq-icon fa fa-plus\"><\/i><i class=\"faq-icon fa fa-minus\"><\/i><\/div>\n<div class=\"faq-answer\">Plan rotations during maintenance windows and, if deterministic behavior is required across time, maintain versioned keys and mappings. Decide whether post rotation data must match prior masked values or can start a new deterministic sequence.<\/div>\n<\/div>\n<div class=\"faq-item\">\n<div class=\"faq-question\">20. What is the typical rollout path from proof of concept to production? <i class=\"faq-icon fa fa-plus\"><\/i><i class=\"faq-icon fa fa-minus\"><\/i><\/div>\n<div class=\"faq-answer\">Start by classifying a subset of tables, define default rules, and validate masking outcomes in a non production target. Expand table coverage, tune performance, add monitoring, and then move the same configuration into production to protect live change streams.<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/section>\n","protected":false},"excerpt":{"rendered":"<p>Abstract: The previous article in this series of 4 demonstrated real-time database replication with IRI Ripcurrent, an IRI-developed command-line Java application that utilizes the Debezium embedded engine and the streaming features of the IRI (CoSort) SortCL program. Together they react in real-time to database change events. The capability demonstrated in the previous article can also<\/p>\n<div><a class=\"btn-filled btn\" href=\"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/\" title=\"Real-Time, Incremental Data Masking\">Read More<\/a><\/div>\n","protected":false},"author":119,"featured_media":16027,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[8,34,91,29,3],"tags":[218,14,1305,51,1796,1398,2012,1239,651,98,1486,1696,1699,2080,1698,2079,1697],"class_list":["post-15979","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-protection","category-business","category-iri-workbench","category-test-data","category-vldb-operations","tag-cdc","tag-data-masking","tag-data-masking-tools","tag-data-replication","tag-database-column-pii","tag-database-masking","tag-database-replication","tag-db-replication","tag-dynamic-data-masking","tag-format-preserving-encryption","tag-incremental-data-masking","tag-iri-ripcurrent","tag-real-time-data-masking","tag-real-time-database-replication","tag-real-time-db-data-replication","tag-real-time-replication","tag-ripcurrent"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v23.4 (Yoast SEO v23.4) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Real-Time, Incremental Data Masking - IRI<\/title>\n<meta name=\"description\" content=\"See how real-time (incremental) database data masking works during data replication to keep sensitive information secure and compliant.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Real-Time, Incremental Data Masking\" \/>\n<meta property=\"og:description\" content=\"See how real-time (incremental) database data masking works during data replication to keep sensitive information secure and compliant.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/\" \/>\n<meta property=\"og:site_name\" content=\"IRI\" \/>\n<meta property=\"article:published_time\" content=\"2022-06-28T16:33:49+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-11-12T11:13:55+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Blog-3-Image-Feature.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Devon Kozenieski\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Devon Kozenieski\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"11 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/\"},\"author\":{\"name\":\"Devon Kozenieski\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/de972c035aaeecfc40a3ae2ea5ff7ba1\"},\"headline\":\"Real-Time, Incremental Data Masking\",\"datePublished\":\"2022-06-28T16:33:49+00:00\",\"dateModified\":\"2025-11-12T11:13:55+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/\"},\"wordCount\":1890,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Blog-3-Image-Feature.png\",\"keywords\":[\"CDC\",\"data masking\",\"data masking tools\",\"data replication\",\"database column PII\",\"database masking\",\"database replication\",\"DB replication\",\"dynamic data masking\",\"format-preserving encryption\",\"incremental data masking\",\"IRI Ripcurrent\",\"real-time data masking\",\"real-time database replication\",\"Real-time db data replication\",\"real-time replication\",\"Ripcurrent\"],\"articleSection\":[\"Data Masking\/Protection\",\"IRI Business\",\"IRI Workbench\",\"Test Data\",\"VLDB\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/\",\"url\":\"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/\",\"name\":\"Real-Time, Incremental Data Masking - IRI\",\"isPartOf\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Blog-3-Image-Feature.png\",\"datePublished\":\"2022-06-28T16:33:49+00:00\",\"dateModified\":\"2025-11-12T11:13:55+00:00\",\"description\":\"See how real-time (incremental) database data masking works during data replication to keep sensitive information secure and compliant.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/#primaryimage\",\"url\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Blog-3-Image-Feature.png\",\"contentUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Blog-3-Image-Feature.png\",\"width\":1200,\"height\":628},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.iri.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Real-Time, Incremental Data Masking\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.iri.com\/blog\/#website\",\"url\":\"https:\/\/www.iri.com\/blog\/\",\"name\":\"IRI\",\"description\":\"Total Data Management Blog\",\"publisher\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.iri.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\",\"name\":\"IRI\",\"url\":\"https:\/\/www.iri.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"contentUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"width\":750,\"height\":206,\"caption\":\"IRI\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/de972c035aaeecfc40a3ae2ea5ff7ba1\",\"name\":\"Devon Kozenieski\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/e4c421588c1a85dd9a76146fe15528f7?s=96&d=blank&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/e4c421588c1a85dd9a76146fe15528f7?s=96&d=blank&r=g\",\"caption\":\"Devon Kozenieski\"},\"url\":\"https:\/\/www.iri.com\/blog\/author\/devonk\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Real-Time, Incremental Data Masking - IRI","description":"See how real-time (incremental) database data masking works during data replication to keep sensitive information secure and compliant.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/","og_locale":"en_US","og_type":"article","og_title":"Real-Time, Incremental Data Masking","og_description":"See how real-time (incremental) database data masking works during data replication to keep sensitive information secure and compliant.","og_url":"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/","og_site_name":"IRI","article_published_time":"2022-06-28T16:33:49+00:00","article_modified_time":"2025-11-12T11:13:55+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Blog-3-Image-Feature.png","type":"image\/png"}],"author":"Devon Kozenieski","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Devon Kozenieski","Est. reading time":"11 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/#article","isPartOf":{"@id":"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/"},"author":{"name":"Devon Kozenieski","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/de972c035aaeecfc40a3ae2ea5ff7ba1"},"headline":"Real-Time, Incremental Data Masking","datePublished":"2022-06-28T16:33:49+00:00","dateModified":"2025-11-12T11:13:55+00:00","mainEntityOfPage":{"@id":"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/"},"wordCount":1890,"commentCount":0,"publisher":{"@id":"https:\/\/www.iri.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/#primaryimage"},"thumbnailUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Blog-3-Image-Feature.png","keywords":["CDC","data masking","data masking tools","data replication","database column PII","database masking","database replication","DB replication","dynamic data masking","format-preserving encryption","incremental data masking","IRI Ripcurrent","real-time data masking","real-time database replication","Real-time db data replication","real-time replication","Ripcurrent"],"articleSection":["Data Masking\/Protection","IRI Business","IRI Workbench","Test Data","VLDB"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/","url":"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/","name":"Real-Time, Incremental Data Masking - IRI","isPartOf":{"@id":"https:\/\/www.iri.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/#primaryimage"},"image":{"@id":"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/#primaryimage"},"thumbnailUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Blog-3-Image-Feature.png","datePublished":"2022-06-28T16:33:49+00:00","dateModified":"2025-11-12T11:13:55+00:00","description":"See how real-time (incremental) database data masking works during data replication to keep sensitive information secure and compliant.","breadcrumb":{"@id":"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/#primaryimage","url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Blog-3-Image-Feature.png","contentUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Blog-3-Image-Feature.png","width":1200,"height":628},{"@type":"BreadcrumbList","@id":"https:\/\/www.iri.com\/blog\/data-protection\/real-time-incremental-data-masking\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.iri.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Real-Time, Incremental Data Masking"}]},{"@type":"WebSite","@id":"https:\/\/www.iri.com\/blog\/#website","url":"https:\/\/www.iri.com\/blog\/","name":"IRI","description":"Total Data Management Blog","publisher":{"@id":"https:\/\/www.iri.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.iri.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.iri.com\/blog\/#organization","name":"IRI","url":"https:\/\/www.iri.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","contentUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","width":750,"height":206,"caption":"IRI"},"image":{"@id":"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/de972c035aaeecfc40a3ae2ea5ff7ba1","name":"Devon Kozenieski","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/e4c421588c1a85dd9a76146fe15528f7?s=96&d=blank&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e4c421588c1a85dd9a76146fe15528f7?s=96&d=blank&r=g","caption":"Devon Kozenieski"},"url":"https:\/\/www.iri.com\/blog\/author\/devonk\/"}]}},"jetpack_featured_media_url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2022\/06\/Blog-3-Image-Feature.png","_links":{"self":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/15979"}],"collection":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/users\/119"}],"replies":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/comments?post=15979"}],"version-history":[{"count":17,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/15979\/revisions"}],"predecessor-version":[{"id":18756,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/15979\/revisions\/18756"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/media\/16027"}],"wp:attachment":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/media?parent=15979"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/categories?post=15979"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/tags?post=15979"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}