Forget About Big Data – It’s All About New Data

IBM has established that 90% of the world’s data is less than two years old – we create 2.5 quintillion bytes of the stuff each and every day. Let’s put that another way; in the last 24 months, the existing infrastructure for creating, storing and interrogating data has had to absorb an enormous volume of raw data. That infrastructure costs a lot, even before a single byte is analysed.

Some of this data is incredibly valuable if processed in the correct fashion. Some simply has to be retained. For both, however, there is a cost. While the emphasis these days is quite rightly on how to analyse enormous data sets and generate usable information from them, we’re also looking at significant impacts on other areas of technology. It’s no secret that the biggest driver of data centre growth is the creation, storage and processing of data – but this tends to get forgotten in the rush to real time analytics and Hadoop clusters. Aside from the cost implications of extracting valuable information from data is the baseline concern – how much does it cost to collect and store this stuff?

Let’s look at a few numbers: DataCenter Dynamics estimates the investment in data centre technology in 2012-2013 equals the Gross Domestic Product of Kenya, Costa Rica or Lithuania – $35 billion. In 2011-2012, that number was $30 billion. Investment in data centre infrastructure is increasing with the growth of new data, beating the recession and growing at a rate far above most advanced nations’ inflation.

Aside from the cost of building out a data centre, though, there are also the running costs. According to Datacenter Dynamics, in 2012 there were 7.6 million square metres of data centre space in the United Kingdom, consuming enough energy to power six million homes – 6.4 gigawatts, to be exact.  The cost of this energy is also beating inflation. In France, Iliad, provider of telecommunications services, reports its power costs increasing by 7% a year. The UK is in a similar spot. Between the fourth quarter of 2011 and the same quarter of 2012, the Department for Energy and Climate Change estimates industrial electricity bills have risen by three per cent. The UK has the fourth highest industrial electricity costs in the EU 15 group of countries.

What does all of this mean? Well, if you are in the market for a data centre to handle your Big Data, you will want to make sure your operating costs don’t eat into any investment you might want to make into Big Data tools. This applies as much to your third party capacity as it does to a server farm on your premises; those high energy costs will only be passed onto you in the form of your bill.

Doing more with the same amount of power will be important – and upgrading your data centre’s power chain to make sure as much energy as possible is available to the switches, routers, storage and servers in your data centre is vital. Big Data may be a wonderful abstract idea, but it has its grounding in very real things, not least the power supply coming into your ever-growing data centre.