Home » Products » CoSort Version 9 » SortCL
SortCL - Big Data Handling in One Pass 
 
High-Speed Control of High-Volume Data

What Is SortCL?

A) A parallel ETL engine for data staging and integration
B) A file manipulation, comparison and conversion tool
C) A custom detail, delta, and summary report generator
D) 'Beyond the mainframe' sort/merge/sum software
E) A protection and governance tool for flat-file data
F) All of the above

Answer: F) All of the above. See also: FAQ > ETL

The Sort Control Language (SortCL) program is the most powerful and popular tool in IRI's flagship CoSort package. SortCL uses an open, explicit metadata for data definition and manipulation. SortCL's self-documenting 4GL job scripts perform - and combine - high-speed data transformation, conversion, reporting, and protection in large file environments.

From a solution standpoint, SortCL can be used to:
• Replace slower 3GL, Perl and SQL jobs
• Feed, speed, or re-seed BI/DW tools
• Relieve application and system overhead
• Filter, integrate, and stage large files
• Generate custom reports and hand-offs
• Accelerate bulk DB reorgs and loads
• Mask and encrypt fields and records
• Convert legacy sorts and data types
• Migrate between flat and index files
• Synthesize safe test data in real formats
• Detect, capture, and audit changed data

What Can SortCL Do?
CoSort's Sort Control Language (SortCL) program can accept multiple inputs (files, pipes and procedures) in multiple formats, process them in many ways, and produce one or more custom-formatted reports and output targets, all at once.

Specifically, SortCL can, in one job script and I/O pass, rapidly perform and combine all of these data transformations:

Filter At the byte, field and record level, plus duplicate removal and saving
Segment Conditional (include/omit) selection with if-then-else, else-if logic
Sort Multiple keys, directions, sequences
Merge Two or more pre-sorted files
Join Two or more un/sorted files on many conditions for file compares and change data capture (deltas)
Re-map Resize, reposition, and realign fields
Convert Change data types (e.g. EBCDIC<>ASCII, Packed<>Numeric)
Re-format /
Interchange
Convert between file formats
(e.g. Text <>XML, VS<>RS, ISAM<>Vision, LDIF<>CSV)
Aggregate Parallel roll-up and drill-down sum, min, max, average, and count values. Accumulation. Ranking.
Calculate Math and trig functions across detail and summary rows
Sub-string Bit-level manipulations and Perl-compatible regular expression logic for pattern matching, etc.
Validate Check that character and field attributes match their specifications (i.e. "iscompares", gap analysis)
Sequence For custom indexing, reporting, and database load operations
Lookup Discrete field substitutions, pseudonymization, etc. using "set" file field dimensions
Protect Encrypt data at the field level and audit data security measures. Also: anonymization, de-identification, filtering, and pseudonymization
Prototype Create randomly-generated or set-selected (safe) test data fields
Report Custom-formatted, segmented detail and summary targets
Custom Complex field-level user functions (e.g. 3rd-party DQ libraries)

How Do I Run SortCL?
SortCL job scripts are called from the command line, in a batch script, through a job scheduler, from a program (as an executable or C API call), or via CoSort's free Java gui2scl (Graphical User Interface to SortCL) client. The GUI allows you to import, modify, create, save, print, share, validate, and execute SortCL jobs across the network.

What about Logging?
Application-level statistics can be output with each job; either to the screen or a file. In addition, the CoSort system logs runs in a self-appending file, and debugging information in a self-replacing file. On-screen monitoring options are available at various verbosity levels for runtime progress assessment. You can also enable and secure an XML audit log for validating compliance and performing forensic application and data analysis.

Where Can I Get More Information on SortCL?
SortCL plays in an integral role in almost every Solution area described on this web site. Explore this linked list for a description of SortCL's capabilities in:
Business Intelligence
Data Governance
Data Transformation
ETL/DB Acceleration
Field Protection
File Interchange
Third-party Sort Migration

See also:
Products > CoSort
     Products > CoSort > SortCL Metadata
     Products > CoSort > Metadata Converters
     Products > CoSort > SortCL GUI
     Products > CoSort > Dashboard
     Products > CoSort > COBOL Tools
     Products > CoSort > APIs
Products > FACT (Fast Extract)
Products > FieldShield (Data Privacy)
Products > NextForm (Data Migration)
Products > RowGen (Test Data)
make text smaller make text larger print this pageemail this page
» Resources

Unique Value Proposition


In a single pass through your single (file-based) source of the truth, CoSort's field-level transformation tool (SortCL) can: filter, migrate, integrate, stage, and present multiple, well-formatted views of big data; protect data on a need-to-know basis for outsourcing, compliance and test environments; and, prepare both real and test data for VLDB loads, ETL, analysis teams, and BI tools.


IRI Tools in the
Data Warehouse. CoSort SortCL actions are summarized around the CoSort logo in the center of diagram.
» Next Steps
1-800-333-SORT
1-321-777-8889
Did you find what you were looking for on this page?
YesNoUnsure

What you were looking for:

Include your email address if you would like a response.