Unified Data Architecture: A Joined-up Approach To Big Data

Kevin Long, Business Development Director at Teradata UK

The most common challenges faced by European firms using big data are inadequate knowledge of both technical and business issues, a new Business Application Research Centre (BARC) study has shown. Having no clear business case, technical problems and cost pressures also emerged as problems.

One of the main reasons that big data remains a practical challenge is the diversity of modern big data analytics, which frequently involves combining growing volumes of traditional data with complex structures of ‘new’ data. As there is no ‘silver bullet’ for big data, achieving the best results is likely to involve drawing on a range of different services, platforms, applications and tools, such as those found within a unified data architecture (UDA) environment.

Core to any best-practice UDA approach should be a traditional relational data warehouse. This repository will essentially act as the foundation for any analytical environment, supporting the growing volumes of data whilst also reducing the latency of business intelligence and analytics.

Additionally, many organisations are increasingly recognising the requirement for a discovery environment to facilitate data exploration. The platform they opt for to undertake this discovery depends on the nature of data being explored. For relational or well-structured sources, a segregated area of the data warehouse will suffice. Yet, as the data becomes more complex, a separate platform is often required that has the capacity to go beyond relational database constraints.

From a technology perspective, the MapReduce framework provides an efficient approach for discovery. One of the major limitations of this approach is that IT teams typically lack the skills to program in MapReduce. For this reason, a tool such as Teradata Aster that parameterises MapReduce so that queries can be executed using the familiarity of SQL is extremely appealing and can make projects much more cost effective.

A further key element of a UDA is a data staging area capable of loading, storing and refining data, as well as acting as a long term repository. Though not designed for complex analytics, Hadoop has become very popular in this area and offers the advantage of being a low-cost technology. An additional benefit of this approach is that Hadoop and the discovery environment can co-exist on the same physical platform, minimising costs.

Depending on the nature of the business and the project, one or more of the three core elements of UDA can be applied. For this reason, an organisation should evaluate all options before identifying which of the three best meets its analytical needs and maturity.

Ultimately, this unified data architecture approach is all about applying the right technology to the appropriate analytical problem. Done well, this should remove the boundaries that limit innovation in big data analytics and help the organisation as a whole to pursue new opportunities. The insights and value that they find should in turn form the basis for improved performance and actionable decisions.

Kevin Long is Business Development Director at Teradata, the world’s leading analytic data solutions company. He possesses an unusual combination of business, financial and IT skills, which he puts to good use advising how businesses can ‘do more with their data’, through actionable insight that drives competitive advantage: www.teradata.com