Interview with Neil Avery from Excelian

This article takes part from the interview series with our speakers for this year edition of Big Data Week.

Enjoy Neil’s thoughts on big data.

While Big Data is already a buzz term, what does it actually mean for you? (an old question with a fresh answer)

Big Data is a Phenomenon. Since the ‘hype cycle’ has subsided it is now leaking its way into every nook and cranny of our daily lives. The world has changed because of Big Data; it will continue to change in even more radical ways. Spark and SparkSQL are the ‘ODBC of Big Data which really opens out most platforms to become much more heterogeneous without the integration pain. Graph is the next frontier that is ‘here and now’ it shows so much promise and right at the bleeding edge of next-generation of data analysis.

Is it important to be data driven nowadays? If yes, why (please use examples from your industry if possible)?

Yes absolutely, we see many client projects where Big Data is pivotal in their function. It could be Business analysis, reporting or strategy. Data driven is changing business functions to the point that the platform and people are critical and those companies would cease to function without it.

What are the main challenges a company encounters when starting to look at their data?

Knowing where to start! Using or finding the right platform is now even more difficult than ever. Then when looking at your data, understanding the platform limits and (re)modelling will become the main challenges. The notion of Hadoop and many bolts ons has now subsided which is leads to simpler infrastructure – however you still need to embrace spark and the ability to use it properly.

What do you expect from your participation/talk at Big Data Week?

Probably a lot of interesting things I won’t find on the internet. I like conferences because of their informal aspect, yes there will be slides to share but its more about war stories and meeting the people out there who are pushing forwards innovation.

Who should come and listen your talk in Big Data Week?

Anyone that wants to be relevant and has the time. I’m expecting most people are either, ‘doing it’ or at the ‘going to do it’ stages…

Do we still need data scientists with all the tools existing today for data cleaning, analysis, machine learning?

Tough question, the notion of a data scientist seems to change as the tooling evolves. Yes, there is always going to be the concept of a datascientist, but it is likely to become even more tool dependent and less framework, getting your hands type of work (scripting etc). SparkSQL continues to evolve; as they say – it helps to stand of the shoulders of giants.

BIO:

Neil has a long history in large scale, distributed computing, ranging from architecting large scale risk-analytic systems, running startups, to data-heavy weather analysis using data-grids. He is a certified Datastax Solution Architect with Cassandra, Spark and Solr and was previously a Principle Architect at Thoughtworks where he lead several large scale projects in Retail, Logistic and other interesting fields.