Data Franchising

by David Friedland

Data franchising refers to the packaging of large data sets into meaningful chunks that business users can digest and use for decision-making, particularly through advanced business intelligence (BI) software.  To improve the usability and performance of BI tools, CoSort’s SortCL rapidly prepares CSV and XML “feed” files (or ODBC inserts) for their use.DIF Architecture with Processes 2013 V3 - Data Franschising

SortCL takes very large input data from mainframe data sets, very large database extracts, web logs, and other flat files, and performs one or more data integration and staging functions simultaneously that result in one or more outputs, such as:

  • select/filter
  • sort/merge
  • aggregate/calculate
  • match/join
  • encrypt/mask
  • convert/reformat
  • pivot/unpivot
  • substring/custom

By franchising large volumes of sequential data in the file system, SortCL takes the transformation overhead out of the BI layer. By combining and multi-threading the big data transformation and re-mapping work, SortCL also saves job design, computation, and I/O cycle time. Savings also manifest in the BI front-end, since query and display (responses) are faster with smaller inputs. SortCL can also prepare data for SOA, web services, data modeling, security, and advanced statistical applications like SAS and SPSS.

CoSort’s SortCL is routinely used for data franchising for BI platforms like Business Objects, Cognos, Microstrategy, and Excel spreadmarts. IRI also partners directly with best-of-breed dashboard vendors like Dimensional Insight and IVIZ Group, as SortCL can pare huge, disparate legacy and database extract files into aggregated, sorted, and filtered CSV and XML subsets designed to populate their Diver and iDashboard platforms, respectively. From there, users can run a variety of custom queries, modifications, and dynamic reports to visualize, and interact with, data at multiple levels of granularity, and cycle this data through additional, heterogeneous data integration and display processes. Finally, because the IRI Workbench GUI supporting SortCL runs on Eclipse, BIRT users can take SortCL target data conveniently and produce custom reports in the same environment!

SortCL itself also includes standard reporting functionality. This means you can actually run detail, summary, and delta reports  (usually for batch processes), and still franchise data for more sophisticated BI tools — at the same time.

Leave a Comment

Previous post:

Next post: