IRI DarkShield<\/span><\/a> is a powerful data masking package that can discover, delete, de-identify, and\/or deliver PII hidden in a wide range of unstructured data sources. In v3, a command line interface (CLI) was added, allowing third-party applications to embed or run remediation (masking) jobs configured for DarkShield, including Phantom.<\/span><\/p>\n It is thus now possible to automate security responses to PII vulnerabilities in dark data uncovered in DarkShield PII searches. Specifically, Phantom can automatically run DarkShield to plug those holes through playbooks that used Splunk to evaluate the data that DarkShield found. Compare this to sending an alert email, described in our <\/span>prior article<\/span><\/a> on using the Splunk Adaptive Response Framework with DarkShield. <\/span><\/p>\n Here are the underlying components used for this turnkey solution:\u00a0\u00a0<\/span><\/p>\n To start, create a Splunk Phantom account at <\/span>https:\/\/my.phantom.us\/<\/span><\/a> if you don\u2019t already have one. Once signed into the Phantom Community site, the Splunk Phantom Community Edition virtual appliance is available for download from the\u00a0 \u201cProducts\u201d section of the website. The virtual appliance can be utilized with a large variety of virtualization software.<\/span><\/p>\n In this example, I use Oracle VirtualBox. The Splunk Phantom virtual appliance needs to be running to access the Splunk Phantom server. Follow the instructions found on the Phantom Community site, and you should reach a point where you can see what IP the server has been set up on. Access this IP address from your web browser, and sign in to Splunk Phantom.\u00a0<\/span><\/p>\n Splunk Phantom is a CentOS Linux virtual machine that sets up a server to host Phantom. Because of this, SSH needs to be used to run command line actions on your host machine.\u00a0<\/span>While this necessitates more information to run the IRI DarkShield Remediation Playbook, it also makes the playbook more versatile. The host that holds the DarkShield CLI and<\/span> the DarkShield <\/span>.search<\/span><\/i> file can be any machine in the world running SSH.<\/span><\/p>\n The DarkShield CLI runs DarkShield externally to mask data; i.e., it allows the masking jobs to run outside the graphical development and execution environment of IRI Workbench. The CLI<\/span><\/p>\n Download the <\/span>DarkShield <\/span><\/i>CLI<\/span><\/a>, which requires a DarkShield or Voracity license to run.<\/span><\/p>\n Once downloaded, unzip the contents of the <\/span>darkshield.zip <\/span><\/i>archive. You should have the following structure:<\/span><\/p>\n darkshield\\<\/span><\/p>\n \u251c\u2500\u2500 darkshield<\/span><\/p>\n \u251c\u2500\u2500 darkshield.bat<\/span><\/p>\n \u251c\u2500\u2500 darkshield.jar<\/span><\/p>\n The darkshield folder should be added to the system path so that <\/span>darkshield <\/span><\/i>can run from anywhere, such as in the DarkShield Remediation Phantom Playbook described in this article.<\/span><\/p>\n Using the DarkShield CLI requires the following:<\/span><\/p>\n Like most IRI software, DarkShield relies on a core data processing program called <\/span>SortCL<\/span><\/a> to, in this case, mask data. By default, DarkShield uses the $COSORT_HOME environment variable to find its bin directory and use the <\/span>sortcl <\/span><\/i>executable within. <\/span><\/p>\n In this case, I will be transferring data from Splunk Enterprise to Splunk Phantom. I can even speed that flow through Splunk Universal Forwarder, though Splunk Phantom supports many other data sources.\u00a0<\/span><\/p>\n Note that the use of the word \u2018Search\u2019 in this article should be understood in each context in which it occurs. The Splunk \u2018Search\u2019 referred to above is performed subsequent to, and through DarkShield .search file data, which is produced by running the Dark Data Discovery Wizard in IRI Workbench.<\/span><\/i> The *.search file can be used with the DarkShield CLI to search for (and optionally remediate) instances of PII in unstructured file types.<\/span><\/p>\n You must have a Splunk account and a current instance of Splunk Enterprise or Splunk Enterprise Security in order to download\u00a0 and use the Phantom App for Splunk. Within Splunk, make sure that both the <\/span>Phantom App for Splunk<\/span><\/i> and the <\/span>Phantom Remote Search app<\/span><\/i> have been installed. These apps send Splunk search results to Phantom as an event.<\/span><\/p>\n Go to the Phantom Server configuration tab from the Phantom App for Splunk navbar to specify the server details needed to connect to your Splunk Phantom instance. This includes the IP address of the Phantom instance, and an authorization token called a ph-auth token that authorizes the connection.\u00a0<\/span><\/p>\n Create a server, entering the IP address and ph-auth token of the Splunk Phantom instance.\u00a0\u00a0<\/span>The ph-auth token can be found by clicking on the automation user from <\/span>administration > user management > users<\/span><\/i> from the Splunk Phantom menu. Copy the full Authorization Configuration for REST API from here, as shown here:<\/span><\/p>\n Once the Phantom Server is configured in the Phantom App for Splunk, it should be possible to test the connection. <\/span><\/p>\n Phantom allows Splunk searches from Splunk Enterprise and Splunk ES (as well as many other SIEMs and sources) to be exported to Phantom. A practical use of this is to set searches that yield results only for a certain parameter, value, or threshold.\u00a0<\/span><\/p>\n The DarkShield Remediation playbook is designed to act when a certain value or threshold of PII vulnerability as discovered in the DarkShield search process (the <\/span>.search<\/span><\/i> file results) is reached. The playbook will run DarkShield through its CLI to mask that PII when Splunk finds a specified number of unprotected instances in the indexed results.<\/span><\/p>\n The source of the Splunk Search information in this case are the results of a PII scan (requiring a .search file from the Dark Data Discovery WIzard) that outputs to a .txt file which contains detailed information pertaining to instances of unprotected PII in unstructured files. This<\/span> .txt <\/span><\/i>file can be indexed into Splunk either manually or through the Splunk Universal Forwarder.\u00a0<\/span><\/p>\n To get this data that has been indexed into Splunk from the DarkShield .txt file into Phantom, set up a search in Splunk that will only yield results when you want a Phantom playbook to run. In this case, I am going to search through the DarkShield .txt log file to discover when the DarkShield found more than 3 instances of unprotected PII in a document (such as a PDF).<\/span>1<\/sup><\/a><\/span><\/span><\/p>\n Save the search as a report via the \u201cSave As\u201d dropdown. Then click o<\/span>n Settings > Searches, Reports, and Alerts<\/span><\/i>. Make sure the search is stored under the Phantom App. Once the Splunk search has been saved as a report under the Phantom App, navigate to the Event Forwarding page in the Phantom App:<\/span><\/p>\n Click the green button for a \u201cNew Saved Search Export\u201d. This screen will appear:<\/span><\/p>\n Fill out the necessary details such as name of the search, specified saved search, destination (phantom server) receiving the search, and the level of significance of the alert.\u00a0<\/span><\/p>\n Clicking the green \u201cSave and Close\u201d button will automatically send the Splunk search results to Phantom. Clicking \u201cSave and Preview\u201d will allow you to audit which of those search result entries to send to Phantom.<\/span><\/p>\n From Splunk Phantom, playbooks such as the IRI DarkShield Remediation Playbook can be run to remedy the unprotected files that contain personally identifiable information.<\/span><\/p>\n The DarkShield Remediation Playbook works by using SSH to access the machine with the <\/span>.search<\/span><\/i> file produced by DarkShield.<\/span><\/p>\n When this playbook runs, it will prompt the user for the IP address (hostname) of the machine with the<\/span> .search<\/span><\/i> file on it. After that prompt is answered, other prompts will follow:<\/span><\/p>\n The next prompt is the username of the machine to connect to with SSH, followed by the password. The final prompt asks for the absolute file location of the .search file, so the DarkShield CLI will know which<\/span> .search <\/span><\/i>file to execute (lest it mask the wrong files!).<\/span><\/p>\n After all 4 prompts have been answered by the Phantom administrator, the DarkShield CLI will launch a remediation (data masking) job on the computer specified by IP address. This should mask the entries of PII which DarkShield identified and were indexed into Splunk.<\/span><\/p>\n You can use the same IRI DarkShield Remediation Playbook I did. Download the <\/span>archive here<\/span><\/a>, which includes notes and the playbook. You can modify the playbook using Phantom\u2019s Python Playbook <\/span>editor<\/span><\/a>.<\/span><\/p>\n This example presents a turnkey solution to automating the protection of sensitive information hidden in unstructured files on an event-driven basis. It takes some familiarity with the IRI DarkShield command line interface, Splunk Enterprise, and Splunk Phantom, but once built, it\u2019s something that can run for years.<\/span><\/p>\n There is potential for integrations with other IRI products and Phantom playbooks. If certain values are detected through a regular Splunk instance after Voracity produces them — for example in customer transaction or IoT sensor data — actions can be taken. These actions can be executed remotely, which allows for smooth automation procedures.\u00a0<\/span><\/p>\n See the article <\/span>IRI Voracity Data Munging and Masking App for Splunk <\/span><\/a>\u00a0for more ideas. Contact <\/span>info@iri.com<\/span><\/a> if you want to automate data discovery and manipulation jobs in Splunk Phantom.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":" Introduction Splunk Phantom is an orchestration, automation, and response technology for running \u201cPlaybooks\u201d to respond to various conditions. Phantom connects to Splunk Enterprise using the Phantom App for Splunk, so that actions can be taken on knowledge derived from data indexed in Splunk.\u00a0 IRI DarkShield is a powerful data masking package that can discover, delete,<\/p>\n![\"\"](\"\/blog\/wp-content\/uploads\/2019\/09\/splunk-phantom-graphic.png\")
<\/a><\/p>\nPrerequisites<\/b><\/h4>\n
\n
Phantom Setup<\/b><\/h4>\n
DarkShield CLI Setup<\/b><\/h4>\n
![\"\"](\"\/blog\/wp-content\/uploads\/2019\/09\/darkshield-cmd.png\")
<\/a><\/p>\n\n
Install and Configure the Phantom App for Splunk<\/b><\/h4>\n
![\"\"](\"\/blog\/wp-content\/uploads\/2019\/09\/spl-phantom-ph-auth-token.png\")
<\/a><\/p>\n![\"\"](\"\/blog\/wp-content\/uploads\/2019\/09\/spl-enterprise-phantom-app-server-config-1024x188.png\")
<\/a><\/p>\nExport DarkShield Search Results to Phantom via Splunk<\/b><\/h4>\n
![\"\"](\"\/blog\/wp-content\/uploads\/2019\/09\/event-forwarding-1024x212.png\")
<\/a><\/p>\n![\"\"](\"\/blog\/wp-content\/uploads\/2019\/09\/splunk-phantom-edit-config.png\")
<\/a><\/p>\n![\"\"](\"\/blog\/wp-content\/uploads\/2019\/09\/splunk-phantom-results-1024x500.png\")
<\/a><\/p>\nRunning the Playbook<\/b><\/h4>\n
![\"\"](\"\/blog\/wp-content\/uploads\/2019\/09\/Spl-Phantom-Playbook-steps-1024x472.png\")
<\/a><\/p>\n![\"\"](\"\/blog\/wp-content\/uploads\/2019\/09\/Spl-Phantom-IRI-Run-Playbook.png\")
<\/a><\/p>\nDownload this Playbook<\/b><\/h4>\n
Looking Forward<\/b><\/h4>\n