| CoSort SortCL Job Samples: Secure Sensitive Fields Example (Encrypt, Pseudonymize, De-Identify) Notice how the job script also simultaneously transforms the input files through * sorting and joining on the ticker symbol Secure Sensitive Fields Example (Encrypt, Pseudonymize, De-Identify) NYSE-a Input File A. O. Smith Corporation AOS 42.40 142900 0.04 0.09 A.G. Edwards Inc. AGE 52.81 251800 0.48 0.91 A.O. Tatn EFT First TNT 103.01 136000 1.01 0.99 AAG Holding Company1 GFZ 24.71 1900 0.06 0.24 AAG Holding Company2 GFW 25.05 4200 0.05 0.20 Aames Investment Corp. AIC 4.84 145500 0.04 0.82 Aaron Rents, Inc. RNT 24.05 1706300 2.53 9.51 ABB LTD. ABB 12.55 2456100 0.13 1.04 Abbey National plc SXA 25.00 24700 0.15 0.60 Abbott Laboratories ABT 47.25 4210700 0.15 0.31 Abercrombie & Fitch ANF 50.86 1973000 1.32 2.53 Abitibi-Consolidated ABY 2.61 240600 0.00 0.00 ABM Industries Inc. ABM 16.44 102600 0.36 2.14 ABN AMRO Holding N.V. ABN 27.47 195900 0.31 1.14 Buys Input File 1000 DIS Bill Gates 365-57-7505 950 EDS Ben Graham 762-25-9601 25000 WMT Warren Buffet 128-61-1131 3250 AMR Jeff Bezos 129-69-2156 775 TSG Wendi Deng 108-46-1219 400 HBC Stephen Covey 562-21-6485 2100 HIG Richard Branson 578-52-7911 950 TEM Sergey Brin 467-12-5544 1500 AGE Michael Bloomberg 617-51-7283 5000 BAC Donald Trump 583-14-2464 3333 PRU Steve Wynn 449-30-6343 2000 ABN Jack Welch 765-85-6704 8500 RNT George Soros 311-23-8269 1000 MCK Kerry Packer 533-97-5223 4300 UNH Rupert Murdoch 601-13-7610 9000 SDS Jesse Livermore 270-71-5951 3500 SNE Alan Greenspan 254-66-0751 825 ABT Lakshmi Mittal 373-37-4567 9000 ABY Robert Kiyosaki 259-21-9896 855 ADS Lisa Mangino 377-63-7162 50 IBM Rick Haines 474-82-4342 90 SUN Amrita Thakur 277-76-0741 pseudo.set File Bill Gates Isaiah Nordin Ben Graham Rosalee Torre Warren Buffet Petra Kivi Jeff Bezos Valentine Ormond Wendi Deng Suzanna Koster Stephen Covey Rey Gaffney Richard Branson Virgil Kerner Sergey Brin Bobbie Watson Michael Bloomberg Estelle Culbert Donald Trump Penny Worthley Steve Wynn Clarissa Dicus Jack Welch Moses Dinan George Soros Chuck Britton Kerry Packer Adrianna Brand Rupert Murdoch Gretchen Delima Jesse Livermore Martin Baynes Alan Greenspan Lakesha Croy Lakshmi Mittal Jonathan Lawhon Robert Kiyosaki Cathrine McDougal Lisa Mangino Eugenio Killen Rick Haines Tonya Dove Amrita Thakur Kenton Medlin /*Unknown*/
CoSort SortCL Job Script: Secures Sensitive Data Fields
# Produce trade summary results and secure sensitive data
# Load encryption library
/LIBRARY="libcscrypt.dll"
/INFILE=nyse-a # tab-delimited format
/ALIAS=nyse
/FIELD=(Issue,POS=1,SEP='\t')
/FIELD=(Symbol,POS=2,SIZE=3,SEP='\t')
/FIELD=(LastTrade,POS=3,SIZE=5.2,SEP='\t',NUMERIC)
/FIELD=(Volume,POS=4,SEP='\t',NUMERIC)
/FIELD=(Change,POS=5,SIZE=4.2,SEP='\t', NUMERIC)
/FIELD=(Percent,POS=6,SIZE=4.2,SEP='\t', NUMERIC)
/INFILE=buys # fixed-position format
/FIELD=(Shares,POS=1,SIZE=5)
/FIELD=(Symbol,POS=9,SIZE=3)
/FIELD=(Client,POS=15,SIZE=17)
/FIELD=(ssno,POS=35,SIZE=11)
/JOIN FULL_OUTER NOT_SORTED nyse NOT_SORTED buys WHERE nyse.Symbol EQ buys.Symbol
/OUTFILE=Secure_trades # summary record format
/HEADREC=" -----------\n"
/FIELD=(New_balance,POS=76,SIZE=14.2,currency)
/SUM New_balance from (nyse.LastTrade * buys.Shares) # expression
/OUTFILE=Secure_trades # detail record format
/HEADREC="Client SS# Symbol Shares LastTrade Shares*LT Ln.\n\n"
# If not empty, provide pseudonym for the Clients from SET file:
/FIELD=(pseudo_name,SET=pseudo.set[buys.Client],SIZE=17)
/FIELD=(name,POS=1,IF buys.Client EQ " " THEN {17}" " ELSE psuedo_name)
# If not empty, encrypt ssno field with AES256:
/FIELD=(encrypted_ssno,POS=20,IF ssno EQ " " THEN "" ELSE encryptAES256(ssno))
# If not empty, de-identify symbol fields:
/FIELD=(deid_symbol1,POS=46,IF buys.Symbol EQ " " THEN "" ELSE de_identify(buys.Symbol,"abc"))
/FIELD=(deid_symbol2,POS=54,IF nyse.Symbol EQ " " THEN "" ELSE de_identify(nyse.Symbol,"abc"))
/FIELD=(buys.Shares,POS=61,SIZE=5)
/FIELD=(nyse.LastTrade,POS=71,SIZE=5.2,NUMERIC)
# If records match, calculate last trade * volume:
/FIELD=(product,POS=80,NUMERIC,IF nyse.Symbol NE buys.Symbol THEN "" ELSE nyse.LastTrade * buys.Shares)
/FIELD=(Sequencer,POS=92,SIZE=4) # discussion reference nb
Secure_trades First Output File
Client SS# Symbol Shares LastTrade Shares*LT Ln.
CEE 12.55 1
CEO 16.44 2
Moses Dinan j6VyS6bbXYgatYL5JIFLZA== CEQ CEQ 2000 27.47 54940.00 3
Jonathan Lawhon wPubF1zFWJiueB4VZr6KyA== CEW CEW 825 47.25 38981.25 4
Cathrine McDougal WOl6ZTkEdDv+S6rhGkzHXw== CE[ CE[ 9000 2.61 23490.00 5
Eugenio Killen 2/cjA7wkt9/1kyWNZYJZRQ== CFU 855 6
Estelle Culbert 1OT7tR0oQwioioEC8p8GiA== CIH CIH 1500 52.81 79215.00 7
CMG 4.84 8
Valentine Ormond CQ93rhKKFRnfCvfPC1OVRQ== COV 3250 9
CQJ 50.86 10
CSU 42.40 11
Penny Worthley qc9F6oLKCTEENci3lAHyvg== ECG 5000 12
Isaiah Nordin HZxxfOODiWH7/7PkgZZjXw== FMU 1000 13
Rosalee Torre Gs4ZT/HqcFhwc2g4oEsezQ== HFU 950 14
IJZ 25.05 15
IJ] 24.71 16
Rey Gaffney zkHcJfLxfz6PTlMICrqbDg== KEG 400 17
Virgil Kerner pI2bZgAGWJlGWenfXL30Ng== KMI 2100 18
Tonya Dove 6P2018n88pvpR7N1dp9NaA== MEO 50 19
Adrianna Brand Tl/RPFMUyY26vccjossBrQ== OGN 1000 20
Clarissa Dicus oN+O2F3dsKMSxiyzuEfK+w== RVY 3333 21
Chuck Britton b+zJGvdsQR9C3u7VIil4Vg== VQW VQW 8500 24.05 204425.00 22
Martin Baynes d+SOp4f83fU+5LQqY5sC6w== UFU 9000 23
Lakesha Croy CV6RCAc/6nxRydcixw2TvA== UQH 3500 24
Kenton Medlin d/jN77NOt69V3InQS3EEJQ== UYQ 90 25
U\C 25.00 26
Bobbie Watson hKJuvbgr8OIvpQKUx7hw9w== WHO 950 27
WQW 103.0 28
Suzanna Koster a+r9AyZdz1GwaJmQ3vRvAQ== WUI 775 29
Gretchen Delima kluZQY9h5ntH7NpeDhQ+Mw== YQK 4300 30
Petra Kivi NDE7/5fbf9md3kRP3Lwt+g== ZOW 25000 31
----------
$401,051.25
pseudo_restore.set File Isaiah Nordin Bill Gates Rosalee Torre Ben Graham Petra Kivi Warren Buffet Valentine Ormond Jeff Bezos Suzanna Koster Wendi Deng Rey Gaffney Stephen Covey Virgil Kerner Richard Branson Bobbie Watson Sergey Brin Estelle Culbert Michael Bloomberg Penny Worthley Donald Trump Clarissa Dicus Steve Wynn Moses Dinan Jack Welch Chuck Britton George Soros Adrianna Brand Kerry Packer Gretchen Delima Rupert Murdoch Martin Baynes Jesse Livermore Lakesha Croy Alan Greenspan Jonathan Lawhon Lakshmi Mittal Cathrine McDougal Robert Kiyosaki Eugenio Killen Lisa Mangino Tonya Dove Rick Haines Kenton Medlin Amrita Thakur /*Unknown*/ Restore Original Fields Example (Decrypt, Restore Real Names, Re-Identify)
CoSort SortCL Job Script: Restores Original Fields
# Restore original values and produce trade summary results
# Load encryption library
/LIBRARY="libcscrypt.dll"
/INFILE=Secure_trades
/INCOLLECT=31 # omit summary fields from Secure_trades output
/INSKIP=2 # omit header from Secure_trades output
# define the layout of Secure_trades detail records:
/FIELD=(Client,POS=1,SIZE=17)
/FIELD=(Client_flag,POS=1,SIZE=1)
/FIELD=(ssno,POS=20,SIZE=24)
/FIELD=(ssno_flag,POS=20,SIZE=1)
/FIELD=(buys_Symbol,POS=46,SIZE=3)
/FIELD=(nyse_Symbol,POS=54,SIZE=3)
/FIELD=(Shares,POS=61,SIZE=5.0,NUMERIC)
/FIELD=(LastTrade,POS=71,SIZE=5.2,NUMERIC)
/FIELD=(product,POS=80,SIZE=9.2,NUMERIC)
/FIELD=(Sequencer,POS=92)
/REPORT
/OUTFILE=Restore_trades # summary record format
/HEADREC=" -----------\n"
/FIELD=(New_balance,POS=63,SIZE=14.2,currency)
/SUM New_balance from (LastTrade * Shares) # expression
/OUTFILE=Restore_trades # detail record format
/HEADREC="Client SS# Symbol Shares LastTrade Shares*LT Ln.\n\n"
# If not empty, restore original Clients from the pseudo_restore SET file
/FIELD=(T,SET=pseudo_restore.set[Client],POS=1,SIZE=17)
/FIELD=(restore_Client,POS=1,IF Client_flag EQ " " THEN {17}" " ELSE T)
# If not empty, decrypt the ssno field using AES256 decryption:
/FIELD=(restore_ssno,POS=20,SIZE=11,IF ssno_flag EQ " " THEN {11}" " ELSE decryptAES256(ssno))
# If not empty, re-identify symbol fields:
/FIELD=(restore_buys_symbol,POS=33,IF buys_symbol EQ " " THEN "" ELSE re_identify(buys_symbol,"abc"))
/FIELD=(restore_buys_symbol,POS=38,IF nyse_symbol EQ " " THEN "" ELSE re_identify(nyse_symbol,"abc"))
/FIELD=(Shares,POS=48,SIZE=5)
/FIELD=(LastTrade,POS=58,SIZE=5.2,NUMERIC)
/FIELD=(product,POS=67,SIZE=9.2,NUMERIC)
/FIELD=(Sequencer,POS=79)
Restore_trades
Client SS# Symbol Shares LastTrade Shares*LT Ln.
ABB 12.55 1
ABM 16.44 2
Jack Welch 765-85-6704 ABN ABN 2000 27.47 54940.00 3
Lakshmi Mittal 373-37-4567 ABT ABT 825 47.25 38981.25 4
Robert Kiyosaki 259-21-9896 ABY ABY 9000 2.61 23490.00 5
Lisa Mangino 377-63-7162 ADS 855 6
Michael Bloomberg 617-51-7283 AGE AGE 1500 52.81 79215.00 7
AIC 4.84 8
Jeff Bezos 129-69-2156 AMR 3250 9
ANF 50.86 10
AOS 42.40 11
Donald Trump 583-14-2464 BAC 5000 12
Bill Gates 365-57-7505 DIS 1000 13
Ben Graham 762-25-9601 EDS 950 14
GFW 25.05 15
GFZ 24.71 16
Stephen Covey 562-21-6485 HBC 400 17
Richard Branson 578-52-7911 HIG 2100 18
Rick Haines 474-82-4342 IBM 50 19
Kerry Packer 533-97-5223 MCK 1000 20
Steve Wynn 449-30-6343 PRU 3333 21
George Soros 311-23-8269 RNT RNT 8500 24.05 204425.00 22
Jesse Livermore 270-71-5951 SDS 9000 23
Alan Greenspan 254-66-0751 SNE 3500 24
Amrita Thakur 277-76-074 SUN 90 25
SXA 25.00 26
Sergey Brin 467-12-5544 TEM 950 27
TNT 103.0 28
Wendi Deng 108-46-1219 TSG 775 29
Rupert Murdoch 601-13-7610 UNH 4300 30
Warren Buffet 128-61-1131 WMT 25000 31
----------
$401,051.25
|