This article explains how to use IRI DarkShield to find and mask personally identifiable information (PII) and other sensitive data in NoSQL databases.

Introduction

IRI DarkShield provides purpose-built wizards in the IRI Workbench integrated development environment (IDE) for searching, classifying, and masking sensitive information. These capabilities apply to structured, semi-structured, and unstructured data in files, NoSQL databases, and relational database sources.

This article demonstrates how to use the DarkShield graphical interface (GUI) to find and mask sensitive data in three NoSQL database systems:

• Apache Cassandra
• Elasticsearch
• MongoDB

DarkShield Version 7 also provides GUI support for Cosmos DB, Redis, and Splunk. In addition, the DarkShield API can support other NoSQL databases through custom calling programs, as demonstrated in the GitHub repository referenced in the original article.

The following architecture diagram illustrates DarkShield’s components within the broader IRI Voracity data management platform. The NoSQL wizard described here operates within IRI Workbench.

Although this article focuses on NoSQL databases, DarkShield also provides other IRI Workbench wizards, including:

• New File Search/Masking Job, for discovering and protecting sensitive information in files.
• New Relational DB Search/Masking Job, for discovering and masking semi-structured or unstructured sensitive data stored in relational database (RDB) columns.

These wizards provide different ways to configure DarkShield jobs according to the data source being processed.

What the DarkShield NoSQL DB Wizard Does

The New NoSQL DB Search/Masking Job wizard in the IRI Workbench GUI for DarkShield helps users configure jobs to find and mask PII or other sensitive information in MongoDB, Cassandra, and Elasticsearch. The wizard produces an XML-based DarkShield job configuration file with the .dsc extension. This file contains the Search Contexts and Mask Contexts used to define the job.

• Search Contexts specify how DarkShield accesses the NoSQL source and scans its contents for sensitive information.
• Mask Contexts specify how the discovered sensitive information should be de-identified and how DarkShield accesses the target NoSQL database where the masked data will be written.

Data Classes and Search Matchers

DarkShield uses predefined Data Classes to identify sensitive information during scanning and masking operations. These Data Classes are stored in the IRI Data Class and Rule Library. Each Data Class contains one or more Search Matchers, which define the methods used to recognize particular kinds of PII.

Earlier versions of the NoSQL wizard supported searching for sensitive values through Java regular expressions (RegEx) and Set File lookups. The wizard described here supports additional search methods, along with masking operations that can run separately or together with discovery.

For more information about the available methods, read about Data Matchers and Location Matchers.

These resources explain how DarkShield identifies sensitive information using configured matching methods.

Prerequisites for Using the DarkShield NoSQL Wizard

Before creating a NoSQL search or masking job, complete the following preliminary steps.

1. Configure the DarkShield API Preferences

First, verify that the DarkShield API distribution directory is specified in the IRI Workbench preferences.

Navigate to:  IRI Workbench References > IRI > DarkShield

Here, you can configure the DarkShield GUI and API preferences, including the host, port, and location of the DarkShield API distribution.

Confirm these settings before proceeding with the wizard.

2. Create or Verify the IRI Data Class and Rule Library

All DarkShield wizards require an IRI project containing an IRI Data Class and Rule Library. The library must contain at least one Data Class and a masking Rule assigned to that Data Class.

Data Classes define the sensitive information to search for, while masking Rules specify how matching information should be transformed. To learn about the IRI Data Class and Rule Library and how to create Data Classes and Rules, read this article.

The following screenshot shows an IRI project containing its associated library.

IRI Project containing IRI Library

IRI Data Class and Rules Library Form Editor contains some Data Classes and Rules

3. Verify That the DarkShield API Server Is Running

Next, confirm that the Plankton (DarkShield API) server is running. Open the DarkShield API Status view in IRI Workbench. This panel displays information about the API server, including its current running status.

DarkShield API Status view panel

Verifying the API status helps establish that the required DarkShield service is available before the job is executed.

4. Verify NoSQL Database Connectivity

Finally, ensure that the DarkShield application can access the source and target NoSQL data silos. Different NoSQL database systems use different software development kits (SDKs) to establish connections and perform create, read, update, and delete (CRUD) operations. Consequently, Cassandra, Elasticsearch, MongoDB, and other NoSQL systems may require different connection parameters, credentials, and access specifications.

For the demonstration in this article, the source is a MongoDB database named Darkshield, containing a collection named data.

View of data collection from MongoDB Compass

This collection will be used to demonstrate the search and masking workflow.

Using the DarkShield NoSQL DB Search/Masking Wizard

The following steps demonstrate how to create a DarkShield NoSQL job through the IRI Workbench wizard.

Step 1: Launch the NoSQL Search/Masking Wizard

In IRI Workbench, open the DarkShield menu and select:
              New NoSQL DB Search/Masking Job…

Launch the wizard from the DarkShield menu by selecting New NoSQL DB Search/Masking Job. This opens the first page, where you name the new job:

Choose the destination folder and file name for the resulting .dsc configuration file. Once the location and name are specified, click Next to configure the sensitive data that DarkShield will search for and the masking operations to apply.

Step 2: Select Data Classes and Masking Rules

The next page allows you to select the IRI Data Class and Rule Library that contains the Data Classes, Data Class Groups, and masking Rules available to the job.

The wizard provides an Active column for selecting or deselecting individual Data Classes and groups. This allows you to control which types of sensitive information will be included in the search. For this example, the job uses the default Data Classes provided when the IRI project was created.

Next, open the Masking Rules tab. The example library contains two available masking functions:

• Format Preserving Encryption (FPE), which is used to encrypt matching values according to the configured rule.
• Blur Date, which is used to transform date values according to the configured rule.

The Masking Rules tab also allows users to add or remove rules as needed.

 

After confirming the available Data Classes and Rules, click Next to continue.

Step 3: Assign Masking Rules to Data Classes

The Assign Masking Rules to Data Classes page defines which masking function will apply to each active Data Class or Data Class Group. Each active Data Class or group must have an assigned masking Rule.

If you do not want to mask a particular PII data type, click Back and deselect its Active checkbox in the Data Class or Data Class Group tab. Then return to the assignment page and complete the remaining rule mappings. These assignments determine how DarkShield transforms the sensitive information identified during the search.

Once the mappings are complete, click Next to configure the NoSQL source database.

Step 4: Configure the NoSQL Source Database

The NoSQL Source page prompts you to select the database type and specify a connection registry.

The wizard version demonstrated in this article provides three database type options:

• Cassandra
• Elasticsearch
• MongoDB

A Connection Registry is a reusable connection configuration containing the information required to connect to a particular data silo.

To create a source connection, select the appropriate NoSQL database type and click New.

Because each database system has different connection requirements, the configuration fields vary by database type.

Cassandra Connection Configuration

For Cassandra, the connection dialog includes fields for the connection name, hosts, port, keyspace, table, datacenter, username, and password.

Elasticsearch Connection Configuration

For Elasticsearch, the connection dialog includes the connection name, hosts, port, cluster, index, username, and password.

MongoDB Connection Configuration

For MongoDB, the connection dialog includes the connection name, URL, database, scope of operation, and collection.

After entering the required connection parameters for your selected database, click Finish to create the Connection Registry. The wizard then returns to the NoSQL Source page, displaying the selected registry.

Click Next to proceed to the NoSQL Target configuration.

Step 5: Configure the NoSQL Target Database

The NoSQL Target page specifies where DarkShield will write the masked data. As with the source, the target uses a Connection Registry to identify the NoSQL data silo and provide the necessary connection information and credentials.

If a suitable target Connection Registry does not already exist, create one using the same process described for the source. In this example, the MongoDB target configuration identifies the target collection as masked_data.

 

Once the target Connection Registry is selected, click Finish.
The wizard generates the .dsc job configuration file that the DarkShield API will use at runtime.

The new job is now available in the IRI project.

DarkShield Job Editor

After completing the wizard, you can open the generated .dsc file in the DarkShield Job editor within IRI Workbench.

This form editor allows you to review and modify parameters initially defined through the wizard without creating a new job from the beginning.

Modifying Source and Target Connections

The editor allows you to add, edit, or remove the source and target configurations associated with the job.

To modify a connection, click Edit… beside the corresponding source or target entry. This lets you update the connection information after the original job configuration has been generated.

Modifying Data Class and Masking Rule Mappings

You can also change the Data Class Rule Mappings through the Modify button.

The editor allows you to select a different IRI Library or rearrange the masking Rules assigned to Data Classes. These modifications determine how the configured job searches for and transforms sensitive information.

Previewing Search and Masking Results

The DarkShield Job editor also includes a Preview function. The preview allows users to test their Data Class Search Matchers and masking Rules against supplied text input.

Clicking Preview shows which sensitive information was identified and how the configured masking Rules transformed it. This provides a way to inspect the behavior of the configured matchers and masking functions before executing the job against the NoSQL source.

Running Your DarkShield Search and Masking Job

DarkShield supports three ways to use the configured job.

1. Search Job: Identifies PII and records the findings in a log file.
2. Masking Job: Uses a search log to mask previously discovered PII.
3. Search and Masking Job: Searches for and masks PII in a single job.

These options allow discovery and masking to be performed separately or together, depending on the intended workflow.

Execute the Search and Masking Job

For this demonstration, the job will search for and mask sensitive information in one operation.

To run the configured job, right-click the .dsc file in IRI Workbench and select:
IRI > Run Search and Masking Job

DarkShield then executes the configured search and masking operations using the Data Classes, masking Rules, and source and target connections defined in the job.

Review the Masked MongoDB Data

After the Search and Masking Job completes, DarkShield writes the masked data to the target silo previously specified in the wizard. The following example compares the source MongoDB collection with the resulting protected collection.

Before masking: Original MongoDB collection

After masking: Protected MongoDB collection

The example demonstrates how the configured DarkShield job identifies sensitive information in a MongoDB collection and applies the assigned masking Rules before writing the protected data to the target collection.

Frequently Asked Questions (FAQs)

What NoSQL databases are supported by the DarkShield GUI?

The NoSQL wizard version demonstrated in this article provides configuration options for Cassandra, Elasticsearch, and MongoDB. DarkShield Version 6 also includes separate GUI integrations for Cosmos DB, Redis, and Splunk. The DarkShield API can support additional NoSQL sources through custom calling programs.

What is a DarkShield .dsc file?

A .dsc file is an XML-based DarkShield job configuration containing Search Contexts and Mask Contexts. These define the data source, search configuration, masking instructions, and target connection used when the job runs.

Can DarkShield search for sensitive data without masking it?

Yes. DarkShield supports a Search Job that identifies PII and logs the results without performing masking. It also supports a separate Masking Job that uses the search log and a combined Search and Masking Job.

How does DarkShield determine which PII values to mask?

DarkShield uses Data Classes containing Search Matchers to identify sensitive information. Masking Rules assigned to those Data Classes determine how the matching values are transformed.

Can I modify a DarkShield NoSQL job after creating it?

Yes. The DarkShield Job editor allows users to modify source and target configurations, change the selected IRI Library, adjust Data Class Rule Mappings, and preview search and masking behavior.

Ready to Find and Mask PII in Your NoSQL Database?

See how IRI DarkShield can help you discover and mask sensitive information in MongoDB, Cassandra, and Elasticsearch using configurable search and masking rules.

Start your free 30-day trial to evaluate DarkShield on your own data and platform, with technical support included at no cost. For questions, a demonstration, or assistance with your specific data protection requirements, contact IRI or email darkshield@iri.com