{"id":14721,"date":"2021-08-30T13:19:47","date_gmt":"2021-08-30T17:19:47","guid":{"rendered":"http:\/\/www.iri.com\/blog\/?p=14721"},"modified":"2022-01-27T14:15:02","modified_gmt":"2022-01-27T19:15:02","slug":"iri-workbench-asn1-encoded-data","status":"publish","type":"post","link":"https:\/\/www.iri.com\/blog\/etl\/iri-workbench-asn1-encoded-data\/","title":{"rendered":"Using IRI Workbench with ASN.1-Encoded Data"},"content":{"rendered":"

In part 4 of this five-part series regarding newly added ASN.1 support in the <\/i>SortCL<\/i><\/a> program central to the IRI Voracity data management <\/i>platform<\/i><\/a> and its component structured data processing <\/i>products<\/i><\/a> — IRI CoSort, NextForm, FieldShield, and RowGen — we take a look specifically at their common graphical user interface (GUI) for job design, <\/i>IRI Workbench<\/i><\/a>.<\/i><\/p>\n

IRI Workbench is an intuitive Integrated Development Environment (IDE) for Voracity operations, built on EclipseTM. It is used to auto-create and ergonomically manage all structured data acquisition, discovery, and processing jobs that serialize in SortCL job scripts. These scripts that, when run, direct how the SortCL program manipulates and maps data in ASN.1 files<\/a> and other sources<\/a>.<\/p>\n

Workbench images<\/b><\/h6>\n

\"\"<\/a>Image displaying options available to choose from in IRI Workbench when selecting an ASN.1 encoded file as a source or target.<\/i><\/p>\n

\"\"<\/a>Image displaying preview of a BER encoded message containing a single record in IRI Workbench. Fields have been flattened and mapped to a SortCL script.<\/i><\/p>\n

ASN.1 Metadata Discovery<\/strong><\/h6>\n

\"\"<\/a><\/p>\n

This is an image demonstrating using the discover metadata wizard to generate metadata for an ASN.1 encoded file. Getting metadata is a crucial part of being able to map fields in an ASN.1 schema to fields for a SortCL script. Once the metadata has been created, it is reusable as long as the ASN.1 schema stays the same. Any number of operations can be performed based on this metadata – generating realistic ASN.1 encoded files as test data, masking personal information that may be in the data, reformatting to a more readable format, etc.<\/p>\n

To begin with, when selecting an input file that is an ASN.1 encoded file, the process type must be selected to be ASN1. The path to at least one ASN.1 schema file must be selected. If the schema file depends on another schema file, all dependent schema files must be selected. From that, a list of the PDUs contained in that schema will be available to select from. In this example with a TAP3 schema, there is only one PDU that references all other types. Its name is DataInterChange.<\/p>\n

\"\"<\/a><\/p>\n

Discover metadata shows all the fields that the PDU flattens out into. This TAP3 schema flattens out into 848 fields, too many for the wizard to display in the data preview.<\/p>\n

The final output has the entire flattened field structure of the ASN.1 schema. The separator will be generated as a comma and should not be modified for any infile specifying \/PROCESS=ASN1, due to how it is processed internally.<\/p>\n

Data Classification<\/strong><\/h6>\n

Data in ASN.1 encoded files can be classified in IRI Workbench, and then rules may be applied based on what the matching data class was. This is useful for masking sensitive data, but not limited in scope to that use case. There are quite a few other types of data manipulations or field functions available that may be performed based on a rule.<\/p>\n

 <\/p>\n

\"\"<\/p>\n

Here is a sample TAP3 file being classified with data classes. The sender and recipient are classified from a list of TADIG codes that identify the country and operator of the two parties in a call. In this case, it is a call between Paraguay and Hong Kong. There are over a hundred different types of data manipulations that can be consistently applied to each data class.<\/div>\n
<\/div>\n
<\/div>\n

 <\/p>\n

IRI Workbench offers a user-friendly and intuitive interface for discovering data, generating data manipulation job scripts, and ultimately producing the target data using the powerful SortCL data mapping engine. To wrap up this series on ASN.1 support in IRI products, a use case of creating a human-readable report filtered from an input file with many BER-encoded messages based on the TAP3 schema will be shown.<\/p>\n

Other articles in this series:<\/p>\n