Challenges
Every landline or cellular telephone call made over a Public Land Mobile Network (PLMN) creates one or more Call Detail Records (CDRs) or Usage Detail Records (UDRs) generated by the mobile switching center (MSC), typically in a binary file format using an Abstract Syntax Notation One (ASN.1)-compatible encoding scheme.
While ASN.1 is a popular language for describing the content and encoding of call- and subscriber-related message data exchanged between computers, these CDRs are not compatible with most data processing systems designed to handle structured text (flat) files or database tables. Instead, the CDRs are usually converted by a “mediation” system to "flatten" the data for downstream use.
Third-party mediation solutions are expensive and require time-consuming setup and conversion runtime. And after mediation, there are many different things that data require, including cleansing, masking, analytics, and/or re-archival -- usually in separate tools involving more expense, setup, and maintenance.
Solutions
Each file is described by an ASN.1 specification (also referred to as a schema) file that usually has an .asn extension. This human-readable metadata file defines every field in the message and can now be directly (and automatically) used in SortCL-compatible jobs in the IRI Voracity data management platform and its component products (CoSort, NextForm, FieldShield and RowGen).
In other words, it is possibly to manipulate, migrate, mask, munge, mine, and synthesize (test) ASN.1-encoded files without the need for mediation using the same IRI back-end executable and metadata language used to tdo the same on other structured data sources.
The conversion-only solution lies in a premium edition of IRI NextForm. NextForm can convert ASN.1 files into differently-encoded ASN.1 files and other flat targets, including CSV and JSON files, Excel sheets, and RDB tables.
NextForm also supports data type conversion at the field level, and the remapping of record layouts.
NextForm file definitions also work in SortCL, the popular 4GL and data transformation and reporting program in IRI CoSort and the IRI Voracity platform.
Beyond format conversion, the CoSort SortCL program can process (filter, sort, join, aggregate, scrub, reformat, etc.), protect, and report directly from ASN.1 files or wrangle that data into CSV, LDIF, XML, text, index, and other structured file formats and DB tables for consumption by third-party analytic tools.
Using a simple 4GL to define the layout and manipulation of your CDR files, you can perform and combine data:
- transformation (scrub, sort, join, group, calculate, re-map, etc.)
- conversion (data types, record layouts, files)
- masking (field-level encryption, de-ID, etc.)
- reporting (custom detail, delta and summaries)
plus validation, find/replace, custom transforms, etc. in the same job script and I/O pass.
To protect PII in CDR files, use IRI FieldShield. FieldShield offers the compliance industry's broadest array of field-level security functions for data at rest. Mask, encrypt, pseudonymize, randomize, or otherwise obfuscate and de-identify email and IP addresses, and other items subject to data privacy law protection. See Example #5 in this blog article.
Do you need test data in ASN.1 file formats? IRI RowGen supports the random generation and selection of data at the field level to help generate custom-formatted files. RowGen uses the same layout metadata as NextForm, CoSort (SortCL), and FieldShield, so you can easily move between test data generation and real data processing.
