How data pros are trying to achieve ROI with big data

Many of the attendees at the recent Strata & Hadoop World conference in San Jose have now completed initial big data initiatives, and are looking for return on those investments this year, says Eric Sammer, co-founder and CTO at Rocana.

In an interview with David Weldon of Information Management, Sammer provided his observations on conference and what attendees said are the top data management challenges they are facing.

What were the most common themes that you heard among conference attendees and how do they align with what you expected?
The most common theme heard was the replacement of legacy systems. This differs from previous years where there were a lot of pilot and test projects. This year it seemed like those pilots had established enough ROI that customers are looking to mainstream big data solutions, especially in the data warehousing and IT operations use cases.

What were the most common data challenges voiced by attendees?
The two most common challenges we heard were, how do I determine which data I should be keeping and for how long should I be keeping it; and a need for purpose-built analytics for specific use cases, versus generic analytics and machine-learning libraries.

The first is a learning issue, as historically the cost of storage has mandated operational processes that eradicate "low-value" data and quickly move most other data to off-line backups. With big data and the low-cost of storage, the data management drivers have changed to compliance and governance issues.

With big data analytics, data formerly considered low-value data can now be used to generate critical insights, like customer sentiment or customer experience measurements. So, IT teams are struggling with the question "should I really be throwing this away just because that's what I've been doing for years?"

The second is a skillset issue. Business can't afford to hire data scientists for the myriad of places big data analytics can be applied, leaving existing staff with the challenge of implementing new analytics. The non-data scientists are not looking for toolkits, they need pre-packaged, purpose-built analytics they can install and manage just like any other business application.

What were the most surprising things that you heard regarding data management initiatives?
The most surprising thing we heard is what customers are paying for their legacy data management products. Ten years ago there was a fundamentally different perspective about data management and the volumes of data to be managed. Those legacy pricing structures are handcuffing CIOs, who are frantically looking for lower total cost of ownership and higher return on investment big data solutions that are designed for the modern era of data management.

What do you see as the top data management and data analytics issues in 2016?
The top issue for customers really is sorting through all the noise in the market to find a solution that will work for them. The explosion in analytics options in the past couple years has been incredible, with a huge emphasis on toolkits.

Fortune 2000 businesses, let alone smaller companies, don't have access to and can't afford data scientists to choose the correct analytic techniques for the majority of business functions. But the marketing hype makes the toolkits sound like packaged solutions. This combination of the number of vendors and difficulty differentiating is a real challenge for customer’s evaluation processes.