IRI DarkShield REST API for Stateless Data Masking

IRI DarkShield REST API

by Adam Lewis

Version 6 of the IRI DarkShield data masking tool now features a self-hosted (on-premise) REST API for PII (Personally Identifiable Information) data masking in structured, semi-structured, and unstructured data sources.

In previous articles, we have covered the related topics of DarkShield API Text, Files, NoSql, and RDB Remote Procedure Call (RPC) endpoints. In this article, we cover a single new endpoint that enables statelessness, providing easier integration with data masking and test data pipelines, as well as load balancing and other applications.

The Previous DarkShield RPC APIs

If you review previous articles on the various DarkShield RPC APIs, you will find sections that describe the steps for creating a Search and Mask job. These steps had often required anywhere from 5-13 separate http(s) requests to DarkShield from the client to perform a single search and masking job.

As an RPC, each request to DarkShield provides individual pieces/instructions that, when aggregated over multiple requests, provide the full context necessary for a Search and Mask Job. This form of client interaction with the API stands in contrast to the tenants of statelessness.

Why Use a REST API?

With the format of an RPC API, there were some potential benefits in reusability and speed improvements in this method, thus DarkShield was initially built in this fashion. But as more use cases for DarkShield were presented to us by clients, we found the RPC format was no longer as beneficial. Furthermore, in certain scenarios, such as load balancing, which requires a stateless approach to communication, an RPC API was far more unwieldy.

What is Stateless?

According to Lokesh Gupta, “Statelessness means that every HTTP request happens in complete isolation. When the Client makes an HTTP request, it includes all information necessary for the server to fulfill the request”. In the context of a DarkShield task, this means that the client sends all relevant information in a single http request to DarkShield to perform a single search/mask job.

In this way, DarkShield can achieve statelessness to perform a Search and Mask on either files, relational databases, or NoSQL databases ,instead of previously requiring 5-13 separate HTTP/s requests!

REST vs. RPC tables courtesy of https://patrickkarsh.medium.com/

How to Configure a Request to the DarkShield REST API

The REST endpoint is defined as %baseapiurl%/api/darkshield/all/allContext.searchAndMask.

The contents of the context stored in your JSON file vary depending on whether you are searching and masking files or databases.

Summary Pieces of DarkShield JSON Request Body

searchContext
maskContext
fileSearchContext (required for files,rdb,nosql)
fileMaskContext (required for files,rdb,nosql)
rdbSearchContext (required for rdb)
rdbMaskContext (required for rdb)
nosqlSearchContext (required for nosql)
nosqlMaskContext (required for nosql)

Sample JSON Body:

For RDB:

{
            "searchContext": {
                  "name": "SearchContext",
                  "matchers": []
            },
            "maskContext": {
                  "name": "MaskContext",
                  "rules": [
                        {
                               "name": "DataPseudoRulessn2",
                               "type": "cosort",
                               "expression": "deterministic_pseudo_replace
                               (\"fname\", ${PersonID}, \"pswd\")"
                        },
                        {
                               "name": "DataPseudoRulessn222",
                               "type": "cosort",
                               "expression": "deterministic_pseudo_replace
                               (\"full_name\", ${PersonID}, \"pswd\")"
                        }
                ],
                "ruleMatchers": [
                        {
                               "name": "DataPseudoRulessn2Matcher",
                               "type": "name",
                               "rule": "DataPseudoRulessn2",
                               "pattern":"Email_Email_location_2"
                         },
                ]
           },
           "fileSearchContext": {
                 "name": "FileSearchContext",
                 "matchers": [
                        {
                               "name": "SearchContext",
                               "type": "searchContext"
                        },
                        {
                               "name": "Email_Email_location_2",
                               "dataClass": "Email",
                               "searchMatcherPriority": 2,
                               "type": "column",
                               "pattern": "(?i)\\b.*EMAIL.*\\b"
                        }
                 ],
                 "configs": {
                 }
           },
           "fileMaskContext": {
                 "name": "FileMaskContext",
                 "rules": [
                       {
                              "name": "MaskContext",
                              "type": "maskContext"
                       }                
                ],
                "configs": {

                }
          },
          "rdbSearchContext": {
                 "name": "RdbSearchContext",
                 "fileSearchContextName": "FileSearchContext",
                 "configs": {
                       "schemaName": “Test",
                       "url":"jdbc:sqlserver://****:1433;
                       databaseName=IRI_Testing;encrypt=false;",
                       "username": "****",
                       "password": "****",
                       "includePattern": "SmallTable",
                       "driverClassName": 
    "com.microsoft.sqlserver.jdbc.SQLServerDriver",
                       "driverConfigs": {
                             "encrypt": "false"
                       }
                  }
          },
          "rdbMaskContext": {
                 "name": "RdbMaskContext",
                 "fileMaskContextName": "FileMaskContext",
                 "configs": {
                       "schemaName": "Test",
                       "url": "jdbc:sqlserver://****:1433;
                       databaseName=IRI_Testing;encrypt=false;",
                       "username": "****",
                       "password": "****",
                       "driverClassName": 
     "com.microsoft.sqlserver.jdbc.SQLServerDriver",
                       "disableForeignKeys": true,
                       "disableTriggers": false,
                       "driverConfigs": {
                              "encrypt": "false"
                       }
                }
      },
      "sourceType": "rdb"
}

For an actual example of calling the DarkShield Version 7 REST API from a Python program, see this article.

Frequently Asked Questions

What is the IRI DarkShield REST API?

The IRI DarkShield REST API provides a stateless way to search for and mask sensitive data through HTTP requests. It supports DarkShield operations across files, relational databases, and NoSQL data sources.

How is the DarkShield REST API different from the earlier RPC API?

The REST API follows a stateless, resource-oriented approach, while the earlier RPC model was method-oriented and relied more heavily on maintaining context between operations. The REST design simplifies request handling by passing the required information with each request.

What data sources can the DarkShield REST API work with?

The DarkShield REST API can support search and masking operations for files, relational databases, and NoSQL sources through the appropriate DarkShield endpoints and source contexts.