Sergey Nivens - Fotolia

Can a big data project help repair broken HCM insights?

The time is now for HR leaders to initiate big data projects that can provide better insight into a company's human capital management processes - while maintaining data privacy and protection.

Fifty years ago human resources departments didn't worry about integration problems; they had only a payroll system. If they had an integration problem, maybe it involved the enterprise's bookkeeping system. Today HR professionals face integration problems not just between payroll and core HR, but often with multiple waves of talent management systems, a mix of time capture and workforce management systems and, of course, the whole benefits enrollment system.

The best antidote for that problem used to be data warehousing -- just extract data from these systems, do a little transform and the promise was that you could answer any question the data warehouse had been built for. It turns out that data warehouses struggle mightily with change -- both in their ability to process a variety of data sources and in their speed of ingestion and ability to answer new questions. Every change required a certain amount of redesign work and with that a cost, both for budgets and timelines.

But a solution was being created by businesses facing a similar, but even more drastic problem: Companies selling ads on the Internet faced the problem that virtually every person interested in an ad had a different segmentation and audience. Because the ad vendors did not want to charge too much or too little, they would respond to queries by simulating the ad. The technology needed to address this problem had to cost-effectively save a lot of fast-moving, diverse data, and it had to be able to answer any query. Hadoop/MapReduce was born, the Adam of big data technology.

Big data projects have high success rate

The first systems took a few minutes to respond to queries. In 2011 we saw sub-minute response times; today the average response time for big data queries is under 30 seconds. This isn't the sub-second response expected from most systems in the era of consumerized IT, but it's a very good response time for being able to answer any question.

Current big data projects have a very high success rate; more than 90% deliver the insights enterprises were hoping for. And it's easy to convince a CEO to start a big data project. The pitch goes as follows: If we could bring together all the company's electronically stored information for analysis, would we not find new and long-awaited insights? And all of that at a very small fraction of the cost that it took us to implement these systems in the first place? We know anecdotally that the CEO response to this question is along the lines of, "Why are you even asking me and have not gone ahead already?"

We often hear from HR professionals that they want to wait for their incumbent vendor to take them to the next level of technology. After all, they pay for maintenance. Although a valid perspective, and even though human capital management (HCM) vendors are making early plans and progress with big data capabilities, don't expect them to have product releases before 2016. For enterprises, that's a lot of time and potential insights lost.

Given the maturity of big data technology today, HR professionals should familiarize themselves with big data technology and become comfortable enough to push for pilots and proofs of concept in the near future. Otherwise, someone else may be in HR. Or worse, the project might be driven by another function in the enterprise that would see people data only as one "data source." But with people expense being the largest cost of most enterprises -- apart from asset-intensive and manufacturing industries where people are the second largest cost -- and with HR being the experts on data privacy and protection, big data projects should initiate from HR departments and their innovative leaders. You have no time to lose.

About the author:
Holger Mueller is vice president and principal analyst of cloud technologies at Constellation Research, providing strategy and counsel related to IaaS and PaaS, and occasionally big data, HR technology, and SaaS. Visit his blog for his latest thoughts on enterprise software.

Next Steps

Talent management innovation could transform HR

New approaches required to tap the value of big data in HR

The forecast for HR and CPM software in 2015

Dig Deeper on HR BI and data visualization software