Data Masking/Protection
This is the third addition to the DevOps pipeline series. Here are the links to the DevOps articles related to GitLab and AWS CodePipeline.
What is Being Demonstrated?
IRI Workbench
This article demonstrates how to synthesize and write data to CSV, JSON, and XMLfiles, plus an Oracle table all at once via the IRI RowGen New Test Data Job wizard in IRI Workbench.
Business Intelligence (BI)
This article is a continuation of the previous article, which introduced IRI support for XLS and XLSX file formats in the SortCL program used for moving and manipulating spreadsheet data in both the IRI Voracity data management (ETL, data quality, etc.)
Test Data
Automating where you can, when you can, is the basis of DevOps automation. Allowing IRI job script executions which mask or synthesize test data to be handled in the CI/CD pipeline supports this pursuit.
Data Masking/Protection
One of the data masking requirements for IRI FieldShield that we see in PHI anonymization use cases involves the blurring of dates at the row level, instead of the column level while providing a configurable option to retain the interval between those dates.
Data Masking/Protection
Usually static data masking is performed on production data at rest so it is stored safely, or when replicated to non-production environments for testing or development purposes.
Big Data
An operational data store (or “ODS”) is another paradigm for integrating enterprise data that is relatively simpler than a data warehouse (DW).
Data Masking/Protection
There are times when it is necessary to test with or share data that has elements of personally identifiable information (PII). To comply with data privacy laws and prevent a data breach, you may need to provide data that reflects, and sometimes imparts, critical information, but still protects the PII.
Business Intelligence (BI)
IRI Workbench not only has several ways to create jobs, but also several ways to execute them.
This article focuses on IRI Workbench execution options for scripts based on the SortCL program language, which covers IRI Voracity ETL, CDC, SDC, pivoting and subsetting jobs, as well as its constituent product jobs; i.e.,
ETL
Just as production data processing tools like IRI CoSort must handle big data in NoSQL DB environments, so too must a big test data generation tool like IRI RowGen.