IRI DarkShield REST API

by Adam Lewis

Version 6 of the IRI DarkShield data masking tool now features a self-hosted (on-premise) REST API for PII (Personally Identifiable Information) data masking in structured, semi-structured, and unstructured data sources.

In previous articles, we have covered the related topics of DarkShield API Text, Files, NoSql, and RDB Remote Procedure Call (RPC) endpoints. In this article, we cover a single new endpoint that enables statelessness, providing easier integration with data masking and test data pipelines, as well as load balancing and other applications.

The Previous DarkShield RPC APIs

If you review previous articles on the various DarkShield RPC APIs, you will find sections that describe the steps for creating a Search and Mask job. These steps had often required anywhere from 5-13 separate http(s) requests to DarkShield from the client to perform a single search and masking job.

As an RPC, each request to DarkShield provides individual pieces/instructions that, when aggregated over multiple requests, provide the full context necessary for a Search and Mask Job. This form of client interaction with the API stands in contrast to the tenants of statelessness.

Why Use a REST API?

With the format of an RPC API, there were some potential benefits in reusability and speed improvements in this method, thus DarkShield was initially built in this fashion. But as more use cases for DarkShield were presented to us by clients, we found the RPC format was no longer as beneficial. Furthermore, in certain scenarios, such as load balancing, which requires a stateless approach to communication, an RPC API was far more unwieldy.

What is Stateless?

According to Lokesh Gupta, “Statelessness means that every HTTP request happens in complete isolation. When the Client makes an HTTP request, it includes all information necessary for the server to fulfill the request”. In the context of a DarkShield task, this means that the client sends all relevant information in a single http request to DarkShield to perform a single search/mask job.

In this way, DarkShield can achieve statelessness to perform a Search and Mask on either files, relational databases, or NoSQL databases ,instead of previously requiring 5-13 separate HTTP/s requests!

REST vs. RPC tables courtesy of https://patrickkarsh.medium.com/

How to Configure a Request to the DarkShield REST API

The REST endpoint is defined as %baseapiurl%/api/darkshield/all/allContext.searchAndMask.

The contents of the context stored in your JSON file vary depending on whether you are searching and masking files or databases.

Summary Pieces of DarkShield JSON Request Body

searchContext
maskContext
fileSearchContext (required for files,rdb,nosql)
fileMaskContext (required for files,rdb,nosql)
rdbSearchContext (required for rdb)
rdbMaskContext (required for rdb)
nosqlSearchContext (required for nosql)
nosqlMaskContext (required for nosql)

Sample JSON Body:

For RDB:

{
            "searchContext": {
                  "name": "SearchContext",
                  "matchers": []
            },
            "maskContext": {
                  "name": "MaskContext",
                  "rules": [
                        {
                               "name": "DataPseudoRulessn2",
                               "type": "cosort",
                               "expression": "deterministic_pseudo_replace
                               (\"fname\", ${PersonID}, \"pswd\")"
                        },
                        {
                               "name": "DataPseudoRulessn222",
                               "type": "cosort",
                               "expression": "deterministic_pseudo_replace
                               (\"full_name\", ${PersonID}, \"pswd\")"
                        }
                ],
                "ruleMatchers": [
                        {
                               "name": "DataPseudoRulessn2Matcher",
                               "type": "name",
                               "rule": "DataPseudoRulessn2",
                               "pattern":"Email_Email_location_2"
                         },
                ]
           },
           "fileSearchContext": {
                 "name": "FileSearchContext",
                 "matchers": [
                        {
                               "name": "SearchContext",
                               "type": "searchContext"
                        },
                        {
                               "name": "Email_Email_location_2",
                               "dataClass": "Email",
                               "searchMatcherPriority": 2,
                               "type": "column",
                               "pattern": "(?i)\\b.*EMAIL.*\\b"
                        }
                 ],
                 "configs": {
                 }
           },
           "fileMaskContext": {
                 "name": "FileMaskContext",
                 "rules": [
                       {
                              "name": "MaskContext",
                              "type": "maskContext"
                       }                
                ],
                "configs": {

                }
          },
          "rdbSearchContext": {
                 "name": "RdbSearchContext",
                 "fileSearchContextName": "FileSearchContext",
                 "configs": {
                       "schemaName": “Test",
                       "url":"jdbc:sqlserver://****:1433;
                       databaseName=IRI_Testing;encrypt=false;",
                       "username": "****",
                       "password": "****",
                       "includePattern": "SmallTable",
                       "driverClassName": 
    "com.microsoft.sqlserver.jdbc.SQLServerDriver",
                       "driverConfigs": {
                             "encrypt": "false"
                       }
                  }
          },
          "rdbMaskContext": {
                 "name": "RdbMaskContext",
                 "fileMaskContextName": "FileMaskContext",
                 "configs": {
                       "schemaName": "Test",
                       "url": "jdbc:sqlserver://****:1433;
                       databaseName=IRI_Testing;encrypt=false;",
                       "username": "****",
                       "password": "****",
                       "driverClassName": 
     "com.microsoft.sqlserver.jdbc.SQLServerDriver",
                       "disableForeignKeys": true,
                       "disableTriggers": false,
                       "driverConfigs": {
                              "encrypt": "false"
                       }
                }
      },
      "sourceType": "rdb"
}