{"id":19126,"date":"2026-03-31T19:54:47","date_gmt":"2026-03-31T23:54:47","guid":{"rendered":"https:\/\/www.iri.com\/blog\/?p=19126"},"modified":"2026-10-01T17:40:50","modified_gmt":"2026-10-01T21:40:50","slug":"controlling-the-growth-of-new-iri-audit-logs","status":"publish","type":"post","link":"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/","title":{"rendered":"Controlling the Growth of New IRI Audit Logs"},"content":{"rendered":"<p><b><i>Abstract:<\/i><\/b><i><span style=\"font-weight: 400;\"> With the introduction of <a href=\"https:\/\/www.iri.com\/blog\/iri\/business\/whats-new-in-cosort-11\/\">CoSort v11<\/a> and its built-in Operational Governance System (<a href=\"https:\/\/www.iri.com\/blog\/iri\/business\/introducing-the-iri-ops-governance-system-ogs\/\">OGS<\/a>), a new JSON Audit Log has replaced less functional legacy log formats associated with the execution of <\/span><\/i><i><a href=\"https:\/\/www.iri.com\/products\/cosort\/sortcl\">SortCL<\/a> <\/i><i><span style=\"font-weight: 400;\">job scripts. This article highlights several methods to control the size and number of logs generated from CoSort, Voracity, FieldShield, RowGen, and NextForm jobs.\u00a0<\/span><\/i><\/p>\n<p><span style=\"font-weight: 400;\">Effective with the release of CoSort Version 11 in Q1 &#8216;2026, a new custom-populated JSON log file is generated with every <\/span><b>sortcl <\/b><span style=\"font-weight: 400;\">job executed. These logs should replace those generated in older versions and write to <\/span><b>%COSORT_HOME%\\logs<\/b><span style=\"font-weight: 400;\"> on Windows and <\/span><b>$COSORT_HOME\/logs <\/b><span style=\"font-weight: 400;\">on Linux. The log files are also created when jobs fail to run.\u00a0<\/span><\/p>\n<p>Even without setting up the OGS, these new JSON logs will accumulate by default. When many jobs run, the number of log files can grow rapidly because of their rich content. Managing these logs, which can exceed 10KB per file, helps save disk space.<\/p>\n<p style=\"text-align: center;\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-19136 size-large\" src=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2026\/03\/audit-log-generation-1024x251.png\" alt=\"Diagram showing IRI audit logs being managed, reduced, moved, or deleted through log management policies.\" width=\"1024\" height=\"251\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2026\/03\/audit-log-generation-1024x251.png 1024w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2026\/03\/audit-log-generation-300x74.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2026\/03\/audit-log-generation-768x188.png 768w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2026\/03\/audit-log-generation.png 1110w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">Of course, you can compress, move, or delete these files manually or automatically through operating system commands or schedulers. However, if you want to set policies and procedures to manage log creation, content, and retention through the OGS, you can. To do so, have the designated OGS \u201cGovernor\u201d run the menu-driven Policy File Manager (PFM) utility described below in this section.\u00a0<\/span><\/p>\n<h2><b>Audit Log File Names<\/b><\/h2>\n<p><span style=\"font-weight: 400;\">By way of background, each Audit Log filename is in this form:\u00a0<\/span><\/p>\n<pre><span style=\"font-size: 10pt;\"><span style=\"font-weight: 400;\">Audit-<\/span><b><i>timestamp<\/i><\/b><span style=\"font-weight: 400;\">-<\/span><b><i>pid<\/i><\/b><span style=\"font-weight: 400;\">-<\/span><b><i>hostname<\/i><\/b><span style=\"font-weight: 400;\">-sortcl.json<\/span><\/span><\/pre>\n<p><span style=\"font-weight: 400;\">where:<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><b>Timestamp <\/b><span style=\"font-weight: 400;\">is an ISO timestamp for the date and time of job execution\u00a0<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><b>pid <\/b><span style=\"font-weight: 400;\">is the process ID of the job<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><b>hostname <\/b><span style=\"font-weight: 400;\">is the hostname of the machine that ran the job\u00a0<\/span><\/li>\n<\/ul>\n<p><span style=\"font-weight: 400;\">An example Audit Log file name is:<\/span><\/p>\n<pre><span style=\"font-size: 10pt;\"><span style=\"font-weight: 400;\">Audit-2026-04-26T10-06-05-007736-CUMULUS-sortcl.json<\/span><b>\u00a0<\/b><\/span><\/pre>\n<h2><b>Audit Log File Content Policy<\/b><\/h2>\n<p><span style=\"font-weight: 400;\">Below is a Policy file as it exists for the initial installation. The AUDIT section of the Policy file determines what the Audit Log records.\u00a0<\/span><span style=\"font-weight: 400;\">Note the value for \u201cMODE\u201d.<\/span><\/p>\n<pre><span style=\"font-weight: 400;\">\u00a0<span style=\"font-size: 10pt;\">\"POLICY\": \u00a0 \u00a0 \u00a0 {<\/span><\/span><span style=\"font-size: 10pt;\">\r\n<span style=\"font-weight: 400;\"> \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"ERROR_LEVEL\":\u00a0 \"Verbose\",<\/span>\r\n<span style=\"font-weight: 400;\"> \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"UNGOVERNED_BEHAVIOR\":\u00a0 \"Permit\",<\/span>\r\n<span style=\"font-weight: 400;\"> \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"AUDIT\":\u00a0 \u00a0 \u00a0 \u00a0 {<\/span>\r\n<span style=\"font-weight: 400;\"> \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"ITEMS\":\u00a0 \u00a0 \u00a0 \u00a0 [],<\/span>\r\n<span style=\"font-weight: 400;\"> \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"CATEGORIES\": \u00a0 [],<\/span>\r\n<span style=\"font-weight: 400;\"> \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u201cMODE\u201d:<\/span> <span style=\"font-weight: 400;\">[<\/span><b>ALL<\/b>\r\n<span style=\"font-weight: 400;\">]\u00a0 <\/span>\r\n<span style=\"font-weight: 400;\"> \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0},<\/span>\r\n<span style=\"font-weight: 400;\"> \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"HISTORY\":\u00a0 \u00a0 \u00a0 {<\/span>\r\n<span style=\"font-weight: 400;\"> \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"VERSION\":\u00a0 \u00a0 \u00a0 \"1.0\",<\/span>\r\n<span style=\"font-weight: 400;\"> \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"AUTHOR\": \u00a0 \u00a0 \u00a0 \"Taylor\",<\/span>\r\n<span style=\"font-weight: 400;\"> \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"CREATION_DATE\":\u00a0 \u00a0 \u00a0 \u00a0 \"2026-01-19T14:47:43\",<\/span>\r\n<span style=\"font-weight: 400;\"> \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"LAST_MODIFIED\":\u00a0 \u00a0 \u00a0 \u00a0 \"2026-01-19T14:47:43\",<\/span>\r\n<span style=\"font-weight: 400;\"> \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"LAST_MODIFIED_BY\": \u00a0 \u00a0 \"Taylor\",<\/span>\r\n<span style=\"font-weight: 400;\"> \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"DESCRIPTION\":\u00a0 \"IRI\"<\/span><\/span><\/pre>\n<p><span style=\"font-weight: 400;\">When <\/span><span style=\"font-weight: 400; font-family: 'courier new', courier, monospace;\">MODE<\/span><span style=\"font-weight: 400;\"> is:<\/span><\/p>\n<ol>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400; font-family: 'courier new', courier, monospace;\">ALL <\/span><span style=\"font-weight: 400;\">&#8211; the default, all items will be recorded in the Audit Log file for each <\/span><b>sortcl <\/b><span style=\"font-weight: 400;\">run<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\"><span style=\"font-family: 'courier new', courier, monospace;\">None<\/span> <\/span><span style=\"font-weight: 400;\">&#8211; as no items are selected, there will be no Audit Log file<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400; font-family: 'courier new', courier, monospace;\">Minimal <\/span><span style=\"font-weight: 400;\">&#8211;\u00a0 only runtime stats and permissions are recorded in the Audit Log file<\/span><\/li>\n<\/ol>\n<p>You can leave Mode empty to customize the log content more granularly.<span style=\"font-weight: 400;\"> Then, log CATEGORIES, individual log ITEMS, or any combination of the two can be specified in the Policy File. The Audit Log Schema chapter of the Governance Manual for CoSort 11 lists these items.\u00a0<\/span><\/p>\n<h3><b>Method 1) Limiting Audit Log File Content Creation<\/b><\/h3>\n<p><span style=\"font-weight: 400;\">By default, the Policy File sets Audit to ALL. To change the values that the Audit Logs record, someone from the Governor Group must run the OGS Policy File Manager (PFM) utility.\u00a0<\/span><span style=\"font-weight: 400;\">PFM is a command-line, menu-driven program called <\/span><b>policy_file_manager.exe<\/b><span style=\"font-weight: 400;\"> on Windows (in <\/span><b>%COSORT_HOME%\\bin<\/b><span style=\"font-weight: 400;\">) or <\/span><b>policy_file_manager* <\/b><span style=\"font-weight: 400;\">on Linux (in <\/span><b>$COSORT_HOME\/bin<\/b><span style=\"font-weight: 400;\">).<\/span><\/p>\n<p><span style=\"font-weight: 400;\">A Governor can run the above command to launch the utility. To set up the AUDIT section of the Policy file using the<\/span><b> policy_file_manager <\/b><span style=\"font-weight: 400;\">program<\/span><b>,<\/b><span style=\"font-weight: 400;\"> select these menu options in this sequence:<\/span><\/p>\n<pre style=\"padding-left: 40px;\"><span style=\"font-size: 10pt;\"><i><span style=\"font-weight: 400;\">(2) \u00a0 \u00a0 Modify an existing Policy file<\/span><\/i>\r\n<i><span style=\"font-weight: 400;\">(1) \u00a0 \u00a0 Modify current Policy file<\/span><\/i>\r\n<i><span style=\"font-weight: 400;\">(1) \u00a0 \u00a0 Add Policy<\/span><\/i>\r\n<i><span style=\"font-weight: 400;\">(5) \u00a0 \u00a0 Audit Item<\/span><\/i><\/span><\/pre>\n<p><span style=\"font-weight: 400;\">This menu then displays:<\/span><\/p>\n<pre style=\"padding-left: 40px;\"><span style=\"font-size: 10pt;\"><i><span style=\"font-weight: 400;\">(1) \u00a0 \u00a0 List available audit categories<\/span><\/i>\r\n<i><span style=\"font-weight: 400;\">(2) \u00a0 \u00a0 List available audit items<\/span><\/i>\r\n<i><span style=\"font-weight: 400;\">(3) \u00a0 \u00a0 Add an audit category<\/span><\/i>\r\n<i><span style=\"font-weight: 400;\">(4) \u00a0 \u00a0 Add an audit item<\/span><\/i>\r\n<i><span style=\"font-weight: 400;\">(5) \u00a0 \u00a0 Specify an audit mode<\/span><\/i>\r\n<i><span style=\"font-weight: 400;\">(6) \u00a0 \u00a0 Return to prior menu<\/span><\/i>\r\n<i><span style=\"font-weight: 400;\">(7) \u00a0 \u00a0 Return to main menu<\/span><\/i>\r\n<i><span style=\"font-weight: 400;\">(8) \u00a0 \u00a0 (Save? and) exit program<\/span><\/i><\/span><\/pre>\n<p><span style=\"font-weight: 400;\">The options in this menu let the Governor review, choose, and specify the mode, categories, or individual items to include in the Audit Logs.<\/span><\/p>\n<h3><b>Method 2) Managing Audit Log File Location and Retention\u00a0<\/b><\/h3>\n<p><span style=\"font-weight: 400;\">Over time, generating many large Audit Log files can tax disk capacity. You can run the OGS <\/span><b>policy_file_manager <\/b><span style=\"font-weight: 400;\">utility to configure log retention rules saved in the Policy File and enforced with the <\/span><b>cs_logManager<\/b><span style=\"font-weight: 400;\"> utility. The <\/span><b>cs_logManager<\/b><span style=\"font-weight: 400;\"> utility will then move or delete existing Audit Log files from the directory where the logs are stored.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">First, it is necessary to define where a log is generated when each <\/span><b>sortcl <\/b><span style=\"font-weight: 400;\">job runs. Launch the <\/span><b>policy_file_manager <\/b><span style=\"font-weight: 400;\">utility and make the following choices in this order when presented to you:<\/span><\/p>\n<pre style=\"padding-left: 40px;\"><span style=\"font-size: 10pt;\"><i><span style=\"font-weight: 400;\">(2) \u00a0 \u00a0 Modify an existing Policy file<\/span><\/i>\r\n<i><span style=\"font-weight: 400;\">(1) \u00a0 \u00a0 Modify current Policy file<\/span><\/i>\r\n<i><span style=\"font-weight: 400;\">(1) \u00a0 \u00a0 Add Policy<\/span><\/i>\r\n<i><span style=\"font-weight: 400;\">(8) \u00a0 \u00a0 Log Retention Policy<\/span><\/i><\/span><\/pre>\n<pre style=\"padding-left: 40px;\"><span style=\"font-size: 10pt;\"><i><span style=\"font-weight: 400;\">Specify directory of stored logs. Press enter if it's located in the default location.\u00a0<\/span><\/i>\r\n<span style=\"font-weight: 400; font-family: georgia, palatino, serif;\">(Here I will press enter to specify the default location of COSORT_HOME\/logs.)<\/span>\r\n\r\n<i><span style=\"font-weight: 400;\">Add an audit log policy ?<\/span><\/i>\r\n<i><span style=\"font-weight: 400;\">(1) \u00a0 \u00a0 Yes<\/span><\/i>\r\n\r\n<i><span style=\"font-weight: 400;\">Enter selection criteria for audit log retention policy ?\u00a0<\/span><\/i>\r\n<i><span style=\"font-weight: 400;\">(1) \u00a0 \u00a0 MOVE<\/span><\/i>\r\n<i><span style=\"font-weight: 400;\">(2) \u00a0 \u00a0 DELETE<\/span><\/i>\r\n<i><span style=\"font-weight: 400;\">(3) \u00a0 \u00a0 Abort policy<\/span><\/i>\r\n\r\n<span style=\"font-weight: 400;\">1<\/span>\r\n<i><span style=\"font-weight: 400;\">\r\nEnter the Age in days that you want the logs to be checked against. Enter 0 for no checking<\/span><\/i><span style=\"font-weight: 400;\">.<\/span>\r\n<span style=\"font-weight: 400;\">30<span id='easy-footnote-1-19126' class='easy-footnote-margin-adjust'><\/span><span class='easy-footnote'><a href='https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/#easy-footnote-bottom-1-19126' title='&lt;i&gt;Log file creation date determines its age to the OGS, and be be seen. That date is evident in the name of the log file or from by right clicking for properties on it within a Windows folder.'><sup>1<\/sup><\/a><\/span><\/i><\/span>\r\n\r\n<i><span style=\"font-weight: 400;\">Enter the location where you want the logs to be moved to<\/span><\/i><span style=\"font-weight: 400;\">.<span id='easy-footnote-2-19126' class='easy-footnote-margin-adjust'><\/span><span class='easy-footnote'><a href='https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/#easy-footnote-bottom-2-19126' title='&lt;i&gt;If DELETE is chosen, then those logs are deleted.'><sup>2<\/sup><\/a><\/span><\/i><\/span>\r\n<span style=\"font-weight: 400;\">C:\\IRI\\cosort110\\LOG_FILES2<\/span><\/span><\/pre>\n<p><span style=\"font-weight: 400;\">We have chosen to move the files from <\/span><b>COSORT_HOME\\logs<\/b><span style=\"font-weight: 400;\"> to the directory <\/span><b>C:\\IRI\\cosort110\\LOG_FILES2<\/b><span style=\"font-weight: 400;\">. These files will be moved when the utility <\/span><b>cs_logManager.exe<\/b><span style=\"font-weight: 400;\"> (Windows) or <\/span><b>cs_logManager <\/b><span style=\"font-weight: 400;\">(Linux) is executed; i.e., <span style=\"text-decoration: underline;\">manually or as you automate in a job scheduler<\/span>.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">We can therefore choose how long to keep the files in the original directory. Then, running the <strong>cs_logManager<\/strong> utility (in <strong>COSORT_HOME\\bin<\/strong>) moves all Audit Log files 30 days or older from <strong>COSORT_HOME\\logs<\/strong> into <strong>C:\\IRI\\cosort110\\LOG_FILES2.<\/strong><\/span><\/p>\n<p>From here, users can manage the log files independently. <span style=\"font-weight: 400;\"><span id='easy-footnote-3-19126' class='easy-footnote-margin-adjust'><\/span><span class='easy-footnote'><a href='https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/#easy-footnote-bottom-3-19126' title='&lt;i&gt;Compression is not supported via OGS, but can be performed manually on logs after a move.'><sup>3<\/sup><\/a><\/span><\/i><\/span><\/p>\n<h3><b>Method 3) Using Wrangler to Save Only Desired Log Content<\/b><\/h3>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignright wp-image-19137\" src=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2026\/03\/data-querying.png\" alt=\"Diagram showing IRI Audit Log Wrangler filtering, searching, and exporting selected data from audit log files.\" width=\"487\" height=\"228\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2026\/03\/data-querying.png 1024w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2026\/03\/data-querying-300x140.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2026\/03\/data-querying-768x359.png 768w\" sizes=\"(max-width: 487px) 100vw, 487px\" \/><\/p>\n<p><span style=\"font-weight: 400;\"><a href=\"https:\/\/www.iri.com\/blog\/iri\/business\/iri-audit-log-wrangler-performance-security\/\">The Wrangler utility is used to query and export data from existing Audit Log files<\/a>. It can run in either interactive or command-line batch mode, and is documented in the OGS manual. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">Obtain OGS documentation from IRI if your installer did not download it with the software. <\/span><span style=\"font-weight: 400;\">Examples in its Audit Log Wrangler chapter show how to query and export only the information needed from existing logs for audits or archival. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">Running the Wrangler tool with the right mix of SELECT and WHERE syntax can produce delimited file extracts from the fuller JSON Audit Logs, where each row in the wrangled output has information from one Audit Log file. <em>Those source logs can then be deleted because they are no longer needed.<\/em><\/span><\/p>\n<p><span style=\"font-weight: 400;\">Identifying the Audit Log files to delete (i.e., those from which information has already been saved off) will be easier if <\/span><i><span style=\"font-weight: 400;\">processID <\/span><\/i><span style=\"font-weight: 400;\">is part of the query. This is because <\/span><i><span style=\"font-weight: 400;\">processID<\/span><\/i><span style=\"font-weight: 400;\"> is part of the Audit Log file name and can thus be used in a bulk deletion operation on the command line.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Note that this approach is predicated on having the permission of your enterprise and its regulators to delete logs that currently exist. And of course, IRI expressly disclaims all liability for any use or misuse of the Wrangling utility, or other ways in which log data may be removed or deleted.<\/span><\/p>\n<h3><b>Method 4) Future Tooling &#8211; Selective Audit Log Content Scrubbing\u00a0\u00a0<\/b><\/h3>\n<p><span style=\"font-weight: 400;\">IRI is now also considering a separate log scrubbing tool for selective log reduction and segmentation after the fact. This utility will be used in log segmentation and scrubbing based on a range of different qualitative and quantitative filters. This will support post-creation log retention and culling for both IRI and other log wrangling and analytic facilities, again, where permitted.<\/span><\/p>\n<h2><b>Conclusion<\/b><\/h2>\n<p><span style=\"font-weight: 400;\">Audit Logs can be created for every <\/span><b>sortcl <\/b><span style=\"font-weight: 400;\">job that executes. These Audit Logs are designed to hold information based on the items selected using the OGS Policy file. The Audit Logs are held in the COSORT_HOME\\logs directory by default and can be culled <\/span><i><span style=\"font-weight: 400;\">manually<\/span><\/i><span style=\"font-weight: 400;\">.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Using the Policy File Manager (PFM) utility, the OGS governor can configure the content of the logs. Those logs can be moved or deleted using the <\/span><b>cs_logManager. <\/b><span style=\"font-weight: 400;\">It acts on the log retention criteria defined through the PFM and saved in the Policy File.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In addition, the OGS Audit Log <\/span><b>Wrangler <\/b><span style=\"font-weight: 400;\">utility can be used to query and save off desirable subsets of data from old logs, allowing deletion of the original logs where permitted.\u00a0<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Abstract: With the introduction of CoSort v11 and its built-in Operational Governance System (OGS), a new JSON Audit Log has replaced less functional legacy log formats associated with the execution of SortCL job scripts. This article highlights several methods to control the size and number of logs generated from CoSort, Voracity, FieldShield, RowGen, and NextForm<\/p>\n<div><a class=\"btn-filled btn\" href=\"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/\" title=\"Controlling the Growth of New IRI Audit Logs\">Read More<\/a><\/div>\n","protected":false},"author":10,"featured_media":19135,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[108,32,8,31,1,776,30,29],"tags":[2426,1297,2472,2473,2395,2457,1844,2425,494,340,18,45,2463,15,2464,812,2432,2459,2465,546,2427,2461,2470,2468,2466,2467,2458,2462,2453,2454,2460,2452,2396,2456,2455,2417,68,2471,2469],"class_list":["post-19126","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-big-data-2","category-business-intelligence","category-data-protection","category-data-migration","category-data-transformation2","category-etl","category-sort-migration","category-test-data","tag-audit-logging","tag-audit-logs","tag-big-data-governance","tag-compliance-logging","tag-cosort-11","tag-cs_logmanager","tag-data-auditing","tag-data-compliance","tag-data-extraction","tag-data-governance","tag-data-privacy","tag-data-processing","tag-data-processing-automation","tag-data-security","tag-data-workflow-monitoring","tag-enterprise-data-management","tag-etl-governance","tag-governance-framework","tag-governance-policy","tag-iri-cosort","tag-json-audit-logs","tag-json-logs","tag-log-analysis","tag-log-archiving","tag-log-compression","tag-log-deletion","tag-log-file-management","tag-log-file-retention-policy","tag-log-management","tag-log-retention","tag-log-storage-optimization","tag-ogs","tag-operational-governance-system","tag-pfm-utility","tag-policy-file-manager","tag-runtime-governance","tag-sortcl","tag-structured-data-processing","tag-wrangler-utility"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v23.4 (Yoast SEO v23.4) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Audit Log Management: Control Growth and Retention in IRI<\/title>\n<meta name=\"description\" content=\"Explore how audit logs in CoSort v11 enhance operational governance with new JSON log formats for sortcl job scripts.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Controlling the Growth of New IRI Audit Logs\" \/>\n<meta property=\"og:description\" content=\"Explore how audit logs in CoSort v11 enhance operational governance with new JSON log formats for sortcl job scripts.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/\" \/>\n<meta property=\"og:site_name\" content=\"IRI\" \/>\n<meta property=\"article:published_time\" content=\"2026-03-31T23:54:47+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-10-01T21:40:50+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2026\/03\/Controlling-the-Growth-of-New-IRI-Audit-Logs.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1110\" \/>\n\t<meta property=\"og:image:height\" content=\"532\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Susan Gegner\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Susan Gegner\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/\"},\"author\":{\"name\":\"Susan Gegner\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/87be5da567628ab9396ca81170f36d63\"},\"headline\":\"Controlling the Growth of New IRI Audit Logs\",\"datePublished\":\"2026-03-31T23:54:47+00:00\",\"dateModified\":\"2026-10-01T21:40:50+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/\"},\"wordCount\":1185,\"publisher\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2026\/03\/Controlling-the-Growth-of-New-IRI-Audit-Logs.png\",\"keywords\":[\"Audit Logging\",\"audit logs\",\"Big Data Governance\",\"Compliance Logging\",\"CoSort 11\",\"cs_logManager\",\"Data Auditing\",\"Data Compliance\",\"data extraction\",\"data governance\",\"data privacy\",\"data processing\",\"Data Processing Automation\",\"data security\",\"Data Workflow Monitoring\",\"Enterprise Data Management\",\"ETL Governance\",\"Governance Framework\",\"Governance Policy\",\"IRI CoSort\",\"JSON Audit Logs\",\"JSON Logs\",\"Log Analysis\",\"Log Archiving\",\"Log Compression\",\"Log Deletion\",\"Log File Management\",\"Log File Retention Policy\",\"Log Management\",\"Log Retention\",\"Log Storage Optimization\",\"OGS\",\"Operational Governance System\",\"PFM utility\",\"Policy File Manager\",\"Runtime Governance\",\"SortCL\",\"Structured Data Processing\",\"Wrangler Utility\"],\"articleSection\":[\"Big Data\",\"Business Intelligence (BI&#041;\",\"Data Masking\/Protection\",\"Data Migration\",\"Data Transformation\",\"ETL\",\"Sort Migration\",\"Test Data\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/\",\"url\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/\",\"name\":\"Audit Log Management: Control Growth and Retention in IRI\",\"isPartOf\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2026\/03\/Controlling-the-Growth-of-New-IRI-Audit-Logs.png\",\"datePublished\":\"2026-03-31T23:54:47+00:00\",\"dateModified\":\"2026-10-01T21:40:50+00:00\",\"description\":\"Explore how audit logs in CoSort v11 enhance operational governance with new JSON log formats for sortcl job scripts.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/#primaryimage\",\"url\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2026\/03\/Controlling-the-Growth-of-New-IRI-Audit-Logs.png\",\"contentUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2026\/03\/Controlling-the-Growth-of-New-IRI-Audit-Logs.png\",\"width\":1110,\"height\":532,\"caption\":\"Managing IRI audit log growth with OGS policies and log retention controls\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.iri.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Controlling the Growth of New IRI Audit Logs\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.iri.com\/blog\/#website\",\"url\":\"https:\/\/www.iri.com\/blog\/\",\"name\":\"IRI\",\"description\":\"Total Data Management Blog\",\"publisher\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.iri.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\",\"name\":\"IRI\",\"url\":\"https:\/\/www.iri.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"contentUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"width\":750,\"height\":206,\"caption\":\"IRI\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/87be5da567628ab9396ca81170f36d63\",\"name\":\"Susan Gegner\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/2b1ca5592a65d44483351292cf1ae00a?s=96&d=blank&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/2b1ca5592a65d44483351292cf1ae00a?s=96&d=blank&r=g\",\"caption\":\"Susan Gegner\"},\"url\":\"https:\/\/www.iri.com\/blog\/author\/susang\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Audit Log Management: Control Growth and Retention in IRI","description":"Explore how audit logs in CoSort v11 enhance operational governance with new JSON log formats for sortcl job scripts.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/","og_locale":"en_US","og_type":"article","og_title":"Controlling the Growth of New IRI Audit Logs","og_description":"Explore how audit logs in CoSort v11 enhance operational governance with new JSON log formats for sortcl job scripts.","og_url":"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/","og_site_name":"IRI","article_published_time":"2026-03-31T23:54:47+00:00","article_modified_time":"2026-10-01T21:40:50+00:00","og_image":[{"width":1110,"height":532,"url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2026\/03\/Controlling-the-Growth-of-New-IRI-Audit-Logs.png","type":"image\/png"}],"author":"Susan Gegner","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Susan Gegner","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/#article","isPartOf":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/"},"author":{"name":"Susan Gegner","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/87be5da567628ab9396ca81170f36d63"},"headline":"Controlling the Growth of New IRI Audit Logs","datePublished":"2026-03-31T23:54:47+00:00","dateModified":"2026-10-01T21:40:50+00:00","mainEntityOfPage":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/"},"wordCount":1185,"publisher":{"@id":"https:\/\/www.iri.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/#primaryimage"},"thumbnailUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2026\/03\/Controlling-the-Growth-of-New-IRI-Audit-Logs.png","keywords":["Audit Logging","audit logs","Big Data Governance","Compliance Logging","CoSort 11","cs_logManager","Data Auditing","Data Compliance","data extraction","data governance","data privacy","data processing","Data Processing Automation","data security","Data Workflow Monitoring","Enterprise Data Management","ETL Governance","Governance Framework","Governance Policy","IRI CoSort","JSON Audit Logs","JSON Logs","Log Analysis","Log Archiving","Log Compression","Log Deletion","Log File Management","Log File Retention Policy","Log Management","Log Retention","Log Storage Optimization","OGS","Operational Governance System","PFM utility","Policy File Manager","Runtime Governance","SortCL","Structured Data Processing","Wrangler Utility"],"articleSection":["Big Data","Business Intelligence (BI&#041;","Data Masking\/Protection","Data Migration","Data Transformation","ETL","Sort Migration","Test Data"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/","url":"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/","name":"Audit Log Management: Control Growth and Retention in IRI","isPartOf":{"@id":"https:\/\/www.iri.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/#primaryimage"},"image":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/#primaryimage"},"thumbnailUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2026\/03\/Controlling-the-Growth-of-New-IRI-Audit-Logs.png","datePublished":"2026-03-31T23:54:47+00:00","dateModified":"2026-10-01T21:40:50+00:00","description":"Explore how audit logs in CoSort v11 enhance operational governance with new JSON log formats for sortcl job scripts.","breadcrumb":{"@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/#primaryimage","url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2026\/03\/Controlling-the-Growth-of-New-IRI-Audit-Logs.png","contentUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2026\/03\/Controlling-the-Growth-of-New-IRI-Audit-Logs.png","width":1110,"height":532,"caption":"Managing IRI audit log growth with OGS policies and log retention controls"},{"@type":"BreadcrumbList","@id":"https:\/\/www.iri.com\/blog\/data-transformation2\/controlling-the-growth-of-new-iri-audit-logs\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.iri.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Controlling the Growth of New IRI Audit Logs"}]},{"@type":"WebSite","@id":"https:\/\/www.iri.com\/blog\/#website","url":"https:\/\/www.iri.com\/blog\/","name":"IRI","description":"Total Data Management Blog","publisher":{"@id":"https:\/\/www.iri.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.iri.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.iri.com\/blog\/#organization","name":"IRI","url":"https:\/\/www.iri.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","contentUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","width":750,"height":206,"caption":"IRI"},"image":{"@id":"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/87be5da567628ab9396ca81170f36d63","name":"Susan Gegner","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/2b1ca5592a65d44483351292cf1ae00a?s=96&d=blank&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/2b1ca5592a65d44483351292cf1ae00a?s=96&d=blank&r=g","caption":"Susan Gegner"},"url":"https:\/\/www.iri.com\/blog\/author\/susang\/"}]}},"jetpack_featured_media_url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2026\/03\/Controlling-the-Growth-of-New-IRI-Audit-Logs.png","_links":{"self":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/19126"}],"collection":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/comments?post=19126"}],"version-history":[{"count":26,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/19126\/revisions"}],"predecessor-version":[{"id":19128,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/19126\/revisions\/19128"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/media\/19135"}],"wp:attachment":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/media?parent=19126"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/categories?post=19126"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/tags?post=19126"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}