{"id":15131,"date":"2021-09-23T12:36:13","date_gmt":"2021-09-23T16:36:13","guid":{"rendered":"http:\/\/www.iri.com\/blog\/?p=15131"},"modified":"2024-09-23T13:48:03","modified_gmt":"2024-09-23T17:48:03","slug":"masked-test-data-in-an-aws-codepipeline","status":"publish","type":"post","link":"https:\/\/www.iri.com\/blog\/data-protection\/masked-test-data-in-an-aws-codepipeline\/","title":{"rendered":"Masked Test Data in an AWS CodePipeline"},"content":{"rendered":"

This article demonstrates how to automate the execution of <\/i>IRI DarkShield<\/i><\/a> data masking jobs from SSH commands run from within the <\/i>AWS CodePipeline<\/i><\/a> to produce fit-for-purpose test data in the DevOps process.<\/i><\/p>\n

\"\"<\/a><\/p>\n

In my last demonstration with the GitLab pipeline<\/a>, I ran structured IRI FieldShield<\/a> data masking and IRI RowGen<\/a> data synthesis job scripts to produce test data and consume them in API testing post-deployment. In this demonstration, the AWS CodePipeline will call the DarkShield-Files API<\/a> on a remote server to showcase another method of creating fit-for-purpose test data in MongoDB. Of course, the GitLab method with FieldShield and RowGen, is viable in Jenkins<\/a> and Azure DevOps<\/a>, too.<\/p>\n

About IRI DarkShield<\/b><\/h6>\n

DarkShield is a member data masking<\/a> product in the IRI Data Protector Suite<\/a> and included a component of the IRI Voracity<\/a> data management platform. DarkShield focuses mostly on data in semi-structured and unstructured sources like: NoSQL DBs (including DynamoDB), PDFs, MS Word, text and image files (including Parquet and DICOM). DarkShield also finds and masks PII in flat files (local or stored in buckets like S3) and RDBs (usually with C\/BLOB data).<\/p>\n

About AWS CodePipeline and Its Structure<\/b><\/h6>\n

The Amazon Web Service CodePipeline is a continuous delivery service that automates the build, test, and deployment phases of new software releases. AWS CodePipeline also supports the integration of third-party services such as GitHub.<\/p>\n

According to AWS documentation, \u201cA pipeline is a workflow construct that describes how software changes go through a release process\u201d<\/span>1<\/sup><\/a><\/span>. The pipelines are made up of units called stages<\/i> (build,test,deploy). Each stage in turn, is made up of sets of operations called actions<\/i>.<\/p>\n

The actions are configured to run at specific points in the pipeline. An example of an action would be a deployment action in a deployment stage that deploys code to a compute service.<\/p>\n

Each action can be run in serial or parallel within a stage. The following are valid action types in CodePipeline:<\/p>\n