Big Data: Use It or Lose It?

The term Big Data has become ominous as of late, given the amount of personal data collected often without our awareness.  However, in the industrial community, the massive amount of data that can be generated from production and distribution can be a key basis of competitive leverage with data-driven decisions.

Let's start from the beginning, what is big data?  Per Wikipedia "Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications.  The challenges include capture, curation, storage, search, sharing, transfer, analysis, and visualization."  For example, your internet browsing history along with millions of people over a number of years.  Data may include the pages visited, the duration per page, the search words used, the flow from page to page, your personal statistics and far more.  This is often used to build profiles of users based on multiple demographics and broken into segments based on value and buying potential and specifically target in the future.  Just as transportation and manufacturing data can be lumped to determine biggest potential savings.

The "Big" in big data is an estimated 40 zettabytes of data on the planet by 2020, according to a 2012 study from the International Data Corporation o the Digital Universe.  To offer perspective, that's equal to 57 times the number of all the grains of sand on all the beaches on earth.  But of all this data, the same IDC study found that just half of 1% of available data was analyzed.

Accumulating, Storing, & Implementing Strategy

Where do you get this data?  Internal Process Monitoring, Event Recorders, GPS Tracking Hardware, Website Traffic Analysis, Transaction Data, Customer Demographics, can all offer insight. Keep in mind, large numbers provide enough data to predictive behavior - or the more information you have on record and the size of the database, the more useful the information.  But beware; while good data can have great results, bad data can be negative.  Make sure the data is accurate.

Storing and mining data can seem daunting, and our growing capacity and cost-efficient options only encourage the whirlwind but to harness the opportunities within the data, start with a simple strategy, solve a problem.  Choose an issue related to the data gathered and begin to assess a specific area targeted for improvement.  Example: Supply Chain Lead Time.  Review consistency, and the variables.  How many days does it take to process an order by product line?  Day of the week?  Don't purchase analysis software until you know how you are going to use it. 

Current projections speculate that big data used to the fullest extent could increase operating margin by over 60 percent by retailers alone.

Next Steps

Business intelligence uses statistics that help describe customers, orders, processing etc.  Big Data can be thought of as an extension of these statistics through inference reasoning.  Example, what was previously "28 Percent of shipments to plant XYZ are damaged" becomes with Big Data "Route to XYZ has inferior route conditions at location 62 outside of Elmwood."  And damage can be assumed in the future based on the amount of data analyzed, until route conditions are changed.

Use this simple process to form a clear path navigating data.  It can be challenging at first given the 3 dimensional and dynamic nature of the information, but start simple and statistics will become patterns.  Once the pattern is confirmed, data inferences will become clear.