{"id":10448,"date":"2016-09-07T14:22:40","date_gmt":"2016-09-07T18:22:40","guid":{"rendered":"http:\/\/www.iri.com\/blog\/?p=10448"},"modified":"2026-02-23T15:21:45","modified_gmt":"2026-02-23T20:21:45","slug":"a-fresh-look-at-data-preparation","status":"publish","type":"post","link":"https:\/\/www.iri.com\/blog\/business-intelligence\/a-fresh-look-at-data-preparation\/","title":{"rendered":"A Fresh Look at Data Preparation"},"content":{"rendered":"

To analyze data successfully, it must first be prepared\u00a0successfully. Poor quality data creates poor results. Worse yet is data that takes too long to collect and clean because it is too big or too foreign. Raw data is usually unfit even for the imagination, much less making decisions.<\/span><\/p>\n

Traditional BI architects and big data scientists know the solution to this problem lies in good data preparation. Unfortunately, this is a process that rarely comes to mind when people think about data. It’s hardly seductive, or\u00a0the glamorous side of information today. But it is a necessary step\u00a0towards\u00a0the insights people expect to glean from their data. Indeed, before data can become useful, it has to be culled, churned, and cleansed, so the glamorous algorithmic and display magic <\/span>can <\/span><\/i>happen.<\/span><\/p>\n

\"Benefits<\/a><\/p>\n


\nWhat Exactly is Data Preparation?<\/b><\/p>\n

Think of the sum of data as a landfill. Data gets thrown in from every possible place. People don\u2019t care about it because it\u2019s easy to send it somewhere else and forget about it. As a result, you wind up with just a dump of information. Not only is most of it useless in its current state, i<\/span>t is also taking up space, possibly in what\u2019s now called a <\/span>data lake<\/span><\/a>.<\/span><\/p>\n

Worse still, this data often sits in a pile of various formats and frequently contains duplicates and errors. It\u2019s an unorganized and unmanageable tangle. To call it messy is putting it lightly.<\/span><\/p>\n

\"Landfill
\n<\/a>Data preparation solves these problems. It collects data and cleans it. It gets rid of duplicates and errors. It recognizes that \u201cWilliam Bartram,\u201d \u201cBill Bartram,\u201d and \u201cBill Bertram\u201d are actually the same person, and it unifies those records. Unstructured data, like phone calls or email, passes through the sieve of preparation and comes out in a database-friendly format. More importantly, though, this data is now becoming valuable.<\/span><\/p>\n

Data preparation has gone by a handful of names. IRI first used Rick Sherman\u2019s term for it, \u201c<\/span>data franchising<\/span>,<\/span><\/a>\u201d when CoSort was preparing data for BI tools in 2003. Now it also appears as data <\/span>blending, <\/span><\/i>data <\/span>munging<\/span><\/i>, and data <\/span>wrangling<\/span><\/i>, among other newer industry buzzwords. But they all mean the same thing. Whichever term you prefer, the core preparation activities should include:<\/span><\/p>\n