December 13, 2014
If you’re a Big Data enthusiast, by now you should understand that Big Data is not about “More Data”. Here are 5 ways to understand Big Data.
By original, we don’t mean the “correct” or “authentic”. By original, we mean the first definition coined 12 years ago by Doug Laney. This definition views big data as the three Vs: Volume, Velocity and Variety. Since then, people have been adding their own Vs: Value, Veracity, Visibility and Validity.
The reason that Big Data has captured the spotlight now, 12 years later is due to the development of open source technologies like Hadoop, NoSQL etc.
The users of these new tools needed a term that differentiated them from previous technologies, and–somehow–ended up settling on the woefully inadequate term Big Data. If you go to a big data conference, you can be assured that sessions featuring relational databases–no matter how many Vs they boast–will be in the minority.
The problem with big-data-as-technology is that (a) it’s vague enough that every vendor in the industry jumped in to claim it for themselves and (b) everybody ‘knew’ that they were supposed to elevate the debate and talk about something more business-y and useful.
There was a classification made, Human generated data also called transactional data that was recorded and Machine generated data. This helped businesses understand their big data in business context.
Big data as data set types wasn’t business-y enough. By the time these transactions are recorded, it’s too late to do anything about them: companies are constantly ‘managing out of the rear-view mirror’. In the ‘new world,’ companies can instead use new ‘signal’ data to anticipate what’s going to happen, and intervene to improve the situation.
Examples include tracking brand sentiment on social media (if your ‘likes’ fall off a cliff, your sales will surely follow) and predictive maintenance (complex algorithms determine when you need to replace an aircraft part, before the plane gets expensively stuck on the runway).
This is the laziest and most cynical use of the term, where projects that were possible using previous technology, and would have been called BI or analytics in the past have suddenly been rebaptized in a fairly blatant attempt to jump on the big data bandwagon.