Big Data Believer Or Big Data Realist?

So what is Big Data? It’s the current drive to generate useful and relevant insights from huge (indeed, previously intractable) data sets. The phenomenon is increasingly being touted in business circles as the best route to generating greater knowledge and insight into customer and business needs.

If you’re not familiar with Big Data, all you need to know can be broken down into two projects:

* How to best store massive volumes of data (large amounts of money are being spent on the necessary storage and infrastructure in businesses right now)

* How to best access, visualise and analyse that data in a timely fashion (again, the software and consultancy support required to service these requirements is huge and growing).

Before you embark on this expensive exercise, let’s take a step back and think about where Big Data comes from. The concept originated in the scientific arena, where it was about gathering and organising huge volumes of data and analysing it according to proper mathematical and heuristic methods to test a scientific hypothesis. In other words, it had a very definite point and rationale.

In contrast, business seems to have gone on a slightly indiscriminate Big Data collecting spree, becoming a compulsive hoarder of data.  You never know when you might need it, after all!

However, the idea that collating and analysing vast and disparate amounts of data will yield automatic business insight is plainly naïve. It will yield insight of course, but it wrong to see such collation as an end in itself, business-wise.

For instance, your Big data exercise might reveal the fact that a customer has recently travelled to France on a family holiday, bought a new car last year, has a friend who has a cat for sale and is a big One Direction fan. However, despite having all this rich profiling information about your customer at your disposal, you still can’t offer your customer a precise delivery time, follow her delivery instructions – and you still get her postcode wrong.

We are trying to solve the difficult, intangible problems before addressing the simple and more basic ones it seems.

We shouldn’t attempt any ambitious Big Data projects until we have got the basics of data and analytics right – and are answering and acting on those findings first.

Or have you fallen prey to current industry hyperbole… and become a Big Data Believer, thinking Big Data is The Answer to Everything?

The author is Graham Brierton, CTO at VoiceSage (www.voicesage.com) – a world-leading British company that provides cloud-based interactive voice messaging (IVM) and SMS solutions to businesses