{"id":14697,"date":"2021-08-26T11:57:59","date_gmt":"2021-08-26T15:57:59","guid":{"rendered":"http:\/\/www.iri.com\/blog\/?p=14697"},"modified":"2021-08-31T14:33:51","modified_gmt":"2021-08-31T18:33:51","slug":"asn1-integration-sortcl","status":"publish","type":"post","link":"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/","title":{"rendered":"ASN.1 Integration with SortCL (IRI Voracity)"},"content":{"rendered":"<p><i>This article presents a more in-depth look at ASN.1 and processing encoded data messages based on ASN.1 schemas with the <\/i><a href=\"https:\/\/www.iri.com\/products\/cosort\/sortcl\"><i>SortCL<\/i><\/a><i> program supporting <\/i><a href=\"https:\/\/www.iri.com\/products\/voracity\"><i>IRI Voracity<\/i><\/a><i> platform products. This is article 2 in the five part series discussing ASN.1 and its current support in IRI software.<\/i><\/p>\n<h6><b>The ASN.1 Schema File<\/b><\/h6>\n<p>With each ASN.1 encoded data message there is always an associated ASN.1 schema. An ASN.1 schema offers a complete definition of every field in the message.<\/p>\n<p>Each type described (such as Product, ProductColor, and ProductName) in ASN.1 must begin with an uppercase letter. Items that are components of a message (such as name, color) are called identifiers and must begin with a lowercase letter.<\/p>\n<p>There are also a number of limitations, called constraints, that may be placed when defining types. Usually constraints result in smaller memory footprint and more compact encodings. They also restrict the permissible values of a type.<\/p>\n<p>In ASN.1, all definitions are placed inside of a module which begins with the BEGIN keyword and ends with the END keyword.<\/p>\n<p>A very simple schema may look like this:<\/p>\n<pre style=\"text-align: center;\">ProductInfo DEFINITIONS XER INSTRUCTIONS ::= BEGIN\r\n\r\n    ProductName ::= IA5String (FROM (\"a\"..\"z\")) (SIZE(1..10))\r\n\r\n    ProductColor ::= ENUMERATED {red(0), green(1), blue(2), black(3)}\r\n\r\n    Availability ::= BOOLEAN\r\n    Price ::= INTEGER (10..100)\r\n    Product ::= SEQUENCE {\r\n        name ProductName,\r\n        color ProductColor,\r\n        available [ATTRIBUTE] Availability,\r\n        price Price\r\n    }\r\n\r\n    ProductList ::= SEQUENCE (SIZE(1..23)) OF Product\r\n\r\nENCODING-CONTROL XER\r\n    GLOBAL-DEFAULTS MODIFIED-ENCODINGS\r\n\r\nEND<\/pre>\n<p>A basic interpretation of the schema is that the module is called ProductInfo &#8212; it has information about products in a store.<\/p>\n<ul>\n<li aria-level=\"1\">ProductName type is a string between 1 and 10 characters long, of which the characters must be between a-z, inclusive.<\/li>\n<li aria-level=\"1\">ProductColor is an ENUMERATED type, meaning it must be one of the four described values &#8211; red, green, blue, or black.<\/li>\n<li aria-level=\"1\">Availability is a BOOLEAN type, meaning it must be either TRUE or FALSE.<\/li>\n<li aria-level=\"1\">Price is an INTEGER type that must be between (and including) the values of 10 and 100.<\/li>\n<li aria-level=\"1\">Product is a sequence, or grouped collection of types.<\/li>\n<li aria-level=\"1\">ProductList is a SEQUENCE OF Product (with a maximum size of 23 characters for each Product), meaning that the ProductList can contain multiple Products.<\/li>\n<\/ul>\n<p>This schema is saved in a file I\u2019m calling <i>products.asn<\/i>. Note that it is not a standard specification, but a custom (user-defined) sample specification for products sold in a store.<\/p>\n<h6><b>IRI Components for Processing ASN.1-Encoded Data<\/b><\/h6>\n<p>Support in the IRI Voracity platform and its <a href=\"https:\/\/www.iri.com\/products\/voracity\/technical-details#components\">included products<\/a> &#8212; CoSort, NextForm, FieldShield, and RowGen &#8212; for files encoded per ASN.1 schema relies on IRI\u2019s core program for processing structured data &#8212; called <a href=\"https:\/\/www.iri.com\/product\/cosort\/sortcl\">SortCL<\/a> &#8212; and these two components:<\/p>\n<ol>\n<li aria-level=\"1\"><i>libcsasn1<\/i>, an external module that gets called only when a SortCL job script specifies \/PROCESS=ASN1 as the file type for a given \/INFILE (data source) or \/OUTFILE (target). This module processes or outputs the data based on the type of encoding rules specified in the SortCL script. For example, PER can be specified to process PER-encoded data instead of the default BER encoding. The data must meet the constraints of the ASN.1 specification.<\/li>\n<li aria-level=\"1\"><i>asn1_2ddf<\/i>, a command-line executable (and IRI Workbench-embedding utility) that converts ASN.1 file specifications into the data definition file (DDF) format metadata that SortCL jobs require to parse and process those data layouts in a structured fashion. This tool thus eases the creation of the aforementioned types of scripts as a whole.<\/li>\n<\/ol>\n<p>The syntax in SortCL programs for defining ASN.1 sources and targets looks like this:<\/p>\n<pre>\/(IN\/OUT)FILE=\u201d{<i>path_to_asn1_spec.asn<\/i>[<i>,path_to_spec2.asn<\/i>]<i>}<\/i>,\r\nTYPE_OF_ENCODING_RULES[,PDU_TYPE_NAME];filename\u201d<\/pre>\n<p>Syntax rules for these statements are documented in applicable IRI product manuals.<\/p>\n<h6><b>The<\/b><b><i> asn1_2ddf <\/i><\/b><b>Converter<\/b><\/h6>\n<p>The <i>asn1_2ddf<\/i> command-line utility, also supported in <a href=\"https:\/\/www.iri.com\/products\/workbench\">IRI Workbench<\/a>, reads an ASN.1 specification and an input file to generate a Data Definition Format (DDF) <a href=\"https:\/\/www.iri.com\/products\/cosort\/sortcl-metadata\">metadata layout<\/a> compatible with all structured SortCL job scripts. Input and output field formats in DDF syntax are required for all IRI data processing operations run through SortCL.<\/p>\n<p>The jobs written for IRI Voracity, CoSort, FieldShield, NextForm, or RowGen operations rely on their typical data descriptions to parse and map data in structured and symbolic fashion, respectively. A DDF file representing an ASN.1 record could look like this:<\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2021\/08\/asn1_2ddf2.png\"><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-14705 aligncenter\" src=\"\/blog\/wp-content\/uploads\/2021\/08\/asn1_2ddf2-1024x292.png\" alt=\"\" width=\"748\" height=\"213\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2021\/08\/asn1_2ddf2-1024x292.png 1024w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2021\/08\/asn1_2ddf2-300x86.png 300w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2021\/08\/asn1_2ddf2-768x219.png 768w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2021\/08\/asn1_2ddf2.png 1737w\" sizes=\"(max-width: 748px) 100vw, 748px\" \/><\/a><i>Image displaying execution of asn1_2ddf from the command line to get SortCL metadata of a Basic Safety Message used in Intelligent Transportation Systems.<\/i><\/p>\n<h6><b>Configuration Options for ASN.1-Encoded Files <\/b><\/h6>\n<p>IRI support for ASN.1 encoded files is broad and dynamic, allowing for any combination of encoding rules and schema(s). Compare this to other solutions that target ASN.1 encoded messages based on a specific, fixed schema and only support one type of encoding rule.<\/p>\n<p>Many options can be specified in the asn1_2ddf utility (or in its graphical interface in IRI Workbench), to match the wide variety of ASN.1 encoded data that exists.<\/p>\n<p>The type of encoding rules can be specified optionally in asn1_2ddf. This should match the encoding rules of the data. The default encoding is assumed to be BER otherwise.<\/p>\n<p>The type name of the PDU can be specified optionally as well. As mentioned previously, there are many protocols that do not follow the best practice of a single PDU referencing all other types.<\/p>\n<p>In this case, data can only be interpreted based on the PDU rather than a uniform way. Adding the -t flag with the type name of the PDU to interpret will result in a DDF built for that particular PDU. The default behavior if there are multiple PDUs but no -t flag specified is to take the first PDU referenced in the schema(s).<\/p>\n<p>A -p flag can be added as well as a command line argument to <i>asn1_2ddf<\/i>. This flag outputs a commented process line. This commented process line is not a part of the DDF itself, but can serve as a reminder of the type of process that the ultimate SortCL script will need to include.<\/p>\n<p>To ease discovering the type names of the PDUs in those ASN.1 schemas with no single PDU that references all other types, -l can be specified as an option for <i>asn1_2ddf<\/i>. This will generate a list of all the PDU type names in the ASN.1 schema to the output file specified, rather than a DDF.<\/p>\n<p>A flag can also be set to generate a CSV file that contains the converted data from the original input file. In that case, the DDF will be for the CSV file.<\/p>\n<p>This flag will be used by IRI Workbench in the<i> Discover Metadata<\/i> wizard to generate a temporary CSV file that then can be viewed in the wizard. This method will also be used by other wizards to load and search records from encoded files.<\/p>\n<p>The reason is that encoded files cannot be just loaded as normal, since they are defined by a specification, and many are encoded into binary data.<\/p>\n<p>Import file metadata is an option available from the IRI Workbench metadata menu that can be used to get the metadata of a file with data encoded from an ASN.1 schema, and will call the<i> asn1_2ddf<\/i> utility with specified flags given in the wizard.<\/p>\n<p style=\"text-align: center;\"><a href=\"\/blog\/wp-content\/uploads\/2021\/08\/asn1_2ddf_wkbench_options.png\"><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-14704 aligncenter\" src=\"\/blog\/wp-content\/uploads\/2021\/08\/asn1_2ddf_wkbench_options.png\" alt=\"\" width=\"400\" height=\"285\" srcset=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2021\/08\/asn1_2ddf_wkbench_options.png 486w, https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2021\/08\/asn1_2ddf_wkbench_options-300x214.png 300w\" sizes=\"(max-width: 400px) 100vw, 400px\" \/><\/a><i>Image displaying the graphical interface for asn1_2ddf options available from \u201cimport file metadata\u201d in IRI Workbench<\/i><\/p>\n<p>Depending on the complexity of the ASN.1 specification, the final DDF file may contain a range of fields from several to several thousand.<\/p>\n<p>Sometimes multiple ASN.1 schemas may need to be compiled at once. In this case, provide the path to each schema following an -a flag for every schema file needed. For example, the command:<\/p>\n<pre>asn1_2ddf -eBER -p -aRAP-0102.asn -aTAP-0310.asn rap_bin.ber<\/pre>\n<p>compiles both the <b>RAP-0102.asn<\/b> and <b>TAP-0310.asn<\/b> schema files needed to decode\u00a0 <b>rap_bin.ber<\/b>.<\/p>\n<p>A snippet of the beginning of the <b>RAP-0102.asn <\/b>schema looks like this:<\/p>\n<pre>RapDataInterChange ::= CHOICE\r\n{\r\n\u00a0\u00a0\u00a0 returnBatch\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 ReturnBatch,\r\n\u00a0\u00a0\u00a0 acknowledgement\u00a0\u00a0\u00a0 Acknowledgement\r\n}\r\n\r\nReturnBatch ::= [APPLICATION 534] SEQUENCE\r\n{\r\n\u00a0\u00a0\u00a0 rapBatchControlInfo RapBatchControlInfo,\r\n\u00a0\u00a0\u00a0 returnDetails\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 ReturnDetailList,\r\n\u00a0\u00a0\u00a0 rapAuditControlInfo\u00a0\u00a0\u00a0 RapAuditControlInfo\r\n}\r\n\r\nAcknowledgement ::= [APPLICATION 535] SEQUENCE\r\n{\r\n\u00a0\u00a0\u00a0 sender\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 Sender,\r\n\u00a0\u00a0\u00a0 recipient\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 Recipient,\r\n\u00a0\u00a0\u00a0 rapFileSequenceNumber\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 RapFileSequenceNumber,\r\n\u00a0\u00a0\u00a0 ackFileCreationTimeStamp\u00a0\u00a0\u00a0 AckFileCreationTimeStamp,\r\n\u00a0\u00a0\u00a0 ackFileAvailableTimeStamp\u00a0\u00a0\u00a0 AckFileAvailableTimeStamp,\r\n\u00a0\u00a0\u00a0 fileTypeIndicator\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 FileTypeIndicator\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 OPTIONAL,\r\n\u00a0\u00a0\u00a0 operatorSpecList\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 OperatorSpecList\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 OPTIONAL\r\n}\r\n\r\nReturnDetailList ::= [APPLICATION 536] SEQUENCE OF ReturnDetail\r\n\r\nReturnDetail ::= CHOICE\r\n{\r\n\u00a0\u00a0\u00a0 missingReturn\u00a0\u00a0\u00a0 MissingReturn,\r\n\u00a0\u00a0\u00a0 fatalReturn\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 FatalReturn,\r\n\u00a0\u00a0\u00a0 severeReturn\u00a0\u00a0\u00a0 SevereReturn\r\n}\r\n\r\n--\r\n--\u00a0 Structure of the individual RAP records\r\n--\r\n\r\nRapBatchControlInfo ::= [APPLICATION 537] SEQUENCE\r\n{\r\n\u00a0\u00a0\u00a0 sender\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 Sender,\r\n\u00a0\u00a0\u00a0 recipient\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 Recipient,\r\n\u00a0\u00a0\u00a0 rapFileSequenceNumber\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 RapFileSequenceNumber,\r\n\u00a0\u00a0\u00a0 rapFileCreationTimeStamp\u00a0\u00a0\u00a0 RapFileCreationTimeStamp,\r\n\u00a0\u00a0\u00a0 rapFileAvailableTimeStamp\u00a0\u00a0\u00a0 RapFileAvailableTimeStamp,\r\n\u00a0\u00a0\u00a0 specificationVersionNumber\u00a0\u00a0\u00a0 SpecificationVersionNumber\u00a0\u00a0\u00a0 OPTIONAL,\r\n\u00a0\u00a0\u00a0 releaseVersionNumber\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 ReleaseVersionNumber\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 OPTIONAL,\r\n\u00a0\u00a0\u00a0 rapSpecificationVersionNumber\u00a0\u00a0\u00a0 RapSpecificationVersionNumber,\r\n\u00a0\u00a0\u00a0 rapReleaseVersionNumber\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 RapReleaseVersionNumber,\r\n\u00a0\u00a0\u00a0 fileTypeIndicator\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 FileTypeIndicator\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 OPTIONAL,\r\n\u00a0\u00a0\u00a0 roamingPartner\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 RoamingPartner\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 OPTIONAL,\r\n\u00a0\u00a0\u00a0 operatorSpecList\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 OperatorSpecList\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 OPTIONAL\r\n}<\/pre>\n<p>A snippet of the first 25 fields of the resulting SortCL DDF looks like this:<\/p>\n<pre>\/FIELD=(RETURNBATCH_RAPBATCHCONTROLINFO_SENDER, POSITION=1, TYPE=ASCII, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RAPBATCHCONTROLINFO_RECIPIENT, POSITION=2, TYPE=ASCII, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RAPBATCHCONTROLINFO_RAPFILESEQUENCENUMBER, POSITION=3, TYPE=ASCII, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RAPBATCHCONTROLINFO_RAPFILECREATIONTIMESTAMP_LOCALTIMESTAMP, POSITION=4, TYPE=ASCII, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RAPBATCHCONTROLINFO_RAPFILECREATIONTIMESTAMP_UTCTIMEOFFSET, POSITION=5, TYPE=ASCII, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RAPBATCHCONTROLINFO_RAPFILEAVAILABLETIMESTAMP_LOCALTIMESTAMP, POSITION=6, TYPE=ASCII, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RAPBATCHCONTROLINFO_RAPFILEAVAILABLETIMESTAMP_UTCTIMEOFFSET, POSITION=7, TYPE=ASCII, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RAPBATCHCONTROLINFO_SPECIFICATIONVERSIONNUMBER, POSITION=8, TYPE=NUMERIC, PRECISION=0, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RAPBATCHCONTROLINFO_RELEASEVERSIONNUMBER, POSITION=9, TYPE=NUMERIC, PRECISION=0, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RAPBATCHCONTROLINFO_RAPSPECIFICATIONVERSIONNUMBER, POSITION=10, TYPE=NUMERIC, PRECISION=0, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RAPBATCHCONTROLINFO_RAPRELEASEVERSIONNUMBER, POSITION=11, TYPE=NUMERIC, PRECISION=0, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RAPBATCHCONTROLINFO_FILETYPEINDICATOR, POSITION=12, TYPE=ASCII, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RAPBATCHCONTROLINFO_ROAMINGPARTNER, POSITION=13, TYPE=ASCII, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RAPBATCHCONTROLINFO_OPERATORSPECLIST, POSITION=14, TYPE=ASCII, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RETURNDETAILS_MISSINGRETURN_STARTMISSINGSEQNUMBER, POSITION=15, TYPE=ASCII, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RETURNDETAILS_MISSINGRETURN_ENDMISSINGSEQNUMBER, POSITION=16, TYPE=ASCII, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RETURNDETAILS_MISSINGRETURN_OPERATORSPECLIST, POSITION=17, TYPE=ASCII, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RETURNDETAILS_FATALRETURN_FILESEQUENCENUMBER, POSITION=18, TYPE=ASCII, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RETURNDETAILS_FATALRETURN_TRANSFERBATCHERROR_ERRORDETAIL_ERRORCONTEXT_PATHITEMID, POSITION=19, TYPE=ASCII, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RETURNDETAILS_FATALRETURN_TRANSFERBATCHERROR_ERRORDETAIL_ERRORCONTEXT_ITEMOCCURRENCE, POSITION=20, TYPE=ASCII, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RETURNDETAILS_FATALRETURN_TRANSFERBATCHERROR_ERRORDETAIL_ERRORCONTEXT_ITEMLEVEL, POSITION=21, TYPE=ASCII, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RETURNDETAILS_FATALRETURN_TRANSFERBATCHERROR_ERRORDETAIL_ITEMOFFSET, POSITION=22, TYPE=ASCII, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RETURNDETAILS_FATALRETURN_TRANSFERBATCHERROR_ERRORDETAIL_ERRORCODE, POSITION=23, TYPE=ASCII, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RETURNDETAILS_FATALRETURN_NOTIFICATIONERROR_NOTIFICATION_SENDER, POSITION=24, TYPE=ASCII, SEPARATOR=\",\")\r\n\/FIELD=(RETURNBATCH_RETURNDETAILS_FATALRETURN_NOTIFICATIONERROR_NOTIFICATION_RECIPIENT, POSITION=25, TYPE=ASCII, SEPARATOR=\",\")<\/pre>\n<p>In the next article, complete SortCL script examples for messages encoded based on an ASN.1 schema will be demonstrated. These scripts make use of the metadata that can be generated through IRI Workbench or <i>asn1_2ddf<\/i>.<\/p>\n<p>The other articles in the series are:<\/p>\n<ul>\n<li aria-level=\"1\"><a href=\"https:\/\/www.iri.com\/blog\/etl\/iri-voracity-support-for-asn1\/\">Introduction to ASN.1<\/a><\/li>\n<li aria-level=\"1\"><a href=\"http:\/\/www.iri.com\/blog\/etl\/sortcl-compatible-asn1-jobs\/\">SortCL ASN.1 Examples<\/a><\/li>\n<li aria-level=\"1\"><a href=\"http:\/\/www.iri.com\/blog\/etl\/iri-workbench-asn1-encoded-data\/\">Using IRI Workbench with ASN.1 encoded data<\/a><\/li>\n<li aria-level=\"1\"><a href=\"http:\/\/www.iri.com\/blog\/etl\/iri-voracity-cdr-data-processing\/\">Gaining insight from Call Detail Records<\/a><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This article presents a more in-depth look at ASN.1 and processing encoded data messages based on ASN.1 schemas with the SortCL program supporting IRI Voracity platform products. This is article 2 in the five part series discussing ASN.1 and its current support in IRI software. The ASN.1 Schema File With each ASN.1 encoded data message<\/p>\n<div><a class=\"btn-filled btn\" href=\"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/\" title=\"ASN.1 Integration with SortCL (IRI Voracity)\">Read More<\/a><\/div>\n","protected":false},"author":119,"featured_media":14709,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[1,776,91,233],"tags":[336,561,546,520,1204,789,850,489,1512,68],"class_list":["post-14697","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-transformation2","category-etl","category-iri-workbench","category-metadata-management","tag-asn-1","tag-ddf","tag-iri-cosort","tag-iri-fieldshield","tag-iri-sortcl","tag-iri-voracity","tag-iri-workbench","tag-metadata","tag-metadata-conversion","tag-sortcl"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v23.4 (Yoast SEO v23.4) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>ASN.1 Integration with SortCL (IRI Voracity) - IRI<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"ASN.1 Integration with SortCL (IRI Voracity)\" \/>\n<meta property=\"og:description\" content=\"This article presents a more in-depth look at ASN.1 and processing encoded data messages based on ASN.1 schemas with the SortCL program supporting IRI Voracity platform products. This is article 2 in the five part series discussing ASN.1 and its current support in IRI software. The ASN.1 Schema File With each ASN.1 encoded data messageRead More\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/\" \/>\n<meta property=\"og:site_name\" content=\"IRI\" \/>\n<meta property=\"article:published_time\" content=\"2021-08-26T15:57:59+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-08-31T18:33:51+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2021\/08\/asn1_workbench_ddf.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1076\" \/>\n\t<meta property=\"og:image:height\" content=\"304\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Devon Kozenieski\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Devon Kozenieski\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/\"},\"author\":{\"name\":\"Devon Kozenieski\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/de972c035aaeecfc40a3ae2ea5ff7ba1\"},\"headline\":\"ASN.1 Integration with SortCL (IRI Voracity)\",\"datePublished\":\"2021-08-26T15:57:59+00:00\",\"dateModified\":\"2021-08-31T18:33:51+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/\"},\"wordCount\":1302,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2021\/08\/asn1_workbench_ddf.png\",\"keywords\":[\"asn.1\",\"DDF\",\"IRI CoSort\",\"IRI FieldShield\",\"IRI SortCL\",\"IRI Voracity\",\"IRI Workbench\",\"metadata\",\"metadata conversion\",\"SortCL\"],\"articleSection\":[\"Data Transformation\",\"ETL\",\"IRI Workbench\",\"Metadata Management\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/\",\"url\":\"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/\",\"name\":\"ASN.1 Integration with SortCL (IRI Voracity) - IRI\",\"isPartOf\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2021\/08\/asn1_workbench_ddf.png\",\"datePublished\":\"2021-08-26T15:57:59+00:00\",\"dateModified\":\"2021-08-31T18:33:51+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/#primaryimage\",\"url\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2021\/08\/asn1_workbench_ddf.png\",\"contentUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2021\/08\/asn1_workbench_ddf.png\",\"width\":1076,\"height\":304},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.iri.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"ASN.1 Integration with SortCL (IRI Voracity)\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.iri.com\/blog\/#website\",\"url\":\"https:\/\/www.iri.com\/blog\/\",\"name\":\"IRI\",\"description\":\"Total Data Management Blog\",\"publisher\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.iri.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\",\"name\":\"IRI\",\"url\":\"https:\/\/www.iri.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"contentUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"width\":750,\"height\":206,\"caption\":\"IRI\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/de972c035aaeecfc40a3ae2ea5ff7ba1\",\"name\":\"Devon Kozenieski\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/e4c421588c1a85dd9a76146fe15528f7?s=96&d=blank&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/e4c421588c1a85dd9a76146fe15528f7?s=96&d=blank&r=g\",\"caption\":\"Devon Kozenieski\"},\"url\":\"https:\/\/www.iri.com\/blog\/author\/devonk\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"ASN.1 Integration with SortCL (IRI Voracity) - IRI","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/","og_locale":"en_US","og_type":"article","og_title":"ASN.1 Integration with SortCL (IRI Voracity)","og_description":"This article presents a more in-depth look at ASN.1 and processing encoded data messages based on ASN.1 schemas with the SortCL program supporting IRI Voracity platform products. This is article 2 in the five part series discussing ASN.1 and its current support in IRI software. The ASN.1 Schema File With each ASN.1 encoded data messageRead More","og_url":"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/","og_site_name":"IRI","article_published_time":"2021-08-26T15:57:59+00:00","article_modified_time":"2021-08-31T18:33:51+00:00","og_image":[{"width":1076,"height":304,"url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2021\/08\/asn1_workbench_ddf.png","type":"image\/png"}],"author":"Devon Kozenieski","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Devon Kozenieski","Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/#article","isPartOf":{"@id":"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/"},"author":{"name":"Devon Kozenieski","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/de972c035aaeecfc40a3ae2ea5ff7ba1"},"headline":"ASN.1 Integration with SortCL (IRI Voracity)","datePublished":"2021-08-26T15:57:59+00:00","dateModified":"2021-08-31T18:33:51+00:00","mainEntityOfPage":{"@id":"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/"},"wordCount":1302,"commentCount":0,"publisher":{"@id":"https:\/\/www.iri.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/#primaryimage"},"thumbnailUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2021\/08\/asn1_workbench_ddf.png","keywords":["asn.1","DDF","IRI CoSort","IRI FieldShield","IRI SortCL","IRI Voracity","IRI Workbench","metadata","metadata conversion","SortCL"],"articleSection":["Data Transformation","ETL","IRI Workbench","Metadata Management"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/","url":"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/","name":"ASN.1 Integration with SortCL (IRI Voracity) - IRI","isPartOf":{"@id":"https:\/\/www.iri.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/#primaryimage"},"image":{"@id":"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/#primaryimage"},"thumbnailUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2021\/08\/asn1_workbench_ddf.png","datePublished":"2021-08-26T15:57:59+00:00","dateModified":"2021-08-31T18:33:51+00:00","breadcrumb":{"@id":"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/#primaryimage","url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2021\/08\/asn1_workbench_ddf.png","contentUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2021\/08\/asn1_workbench_ddf.png","width":1076,"height":304},{"@type":"BreadcrumbList","@id":"https:\/\/www.iri.com\/blog\/etl\/asn1-integration-sortcl\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.iri.com\/blog\/"},{"@type":"ListItem","position":2,"name":"ASN.1 Integration with SortCL (IRI Voracity)"}]},{"@type":"WebSite","@id":"https:\/\/www.iri.com\/blog\/#website","url":"https:\/\/www.iri.com\/blog\/","name":"IRI","description":"Total Data Management Blog","publisher":{"@id":"https:\/\/www.iri.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.iri.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.iri.com\/blog\/#organization","name":"IRI","url":"https:\/\/www.iri.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","contentUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","width":750,"height":206,"caption":"IRI"},"image":{"@id":"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/de972c035aaeecfc40a3ae2ea5ff7ba1","name":"Devon Kozenieski","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/e4c421588c1a85dd9a76146fe15528f7?s=96&d=blank&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e4c421588c1a85dd9a76146fe15528f7?s=96&d=blank&r=g","caption":"Devon Kozenieski"},"url":"https:\/\/www.iri.com\/blog\/author\/devonk\/"}]}},"jetpack_featured_media_url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2021\/08\/asn1_workbench_ddf.png","_links":{"self":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/14697"}],"collection":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/users\/119"}],"replies":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/comments?post=14697"}],"version-history":[{"count":9,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/14697\/revisions"}],"predecessor-version":[{"id":14774,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/14697\/revisions\/14774"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/media\/14709"}],"wp:attachment":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/media?parent=14697"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/categories?post=14697"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/tags?post=14697"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}