Business News

Big data's big stumbling block

Photographer's Choice RF | Getty Images

The world is in dire need of data science professionals as experts ring alarm bells over the shortage of talent in a field that has become crucial to global business.

A recent note from McKinsey showed only 18 percent of companies believe they have the skills to gather and use data effectively while 19 percent are confident that their data-gathering processes contribute to sales effectiveness.

International Data Corporation (IDC) believes the shortage of skilled staff will persist next year.

"In the U.S. alone, there will be 181,000 deep analytics roles in 2018 and five times that many positions requiring related skills in data management and interpretation," the firm said in a report on its 'Big Data and Analytics Predictions for 2015' conference earlier this month.

Read MoreWhy your kids will want to be data scientists

The phenomenon of big data has taken the corporate world by storm in recent years, driving up the need for data professionals. Over 70 percent of Asia-Pacific senior executives view data & analytics (D&A) as a top priority, a December KPMG survey showed. Meanwhile, statistical analysis and data mining was ranked as LinkedIn's top professional skill in 2014.

Despite the overwhelming demand, the sector has experienced a dearth of talent. For example, recruitment agencies in the United Kingdom said that 77 percent of data positions were difficult to fill, according to research by business analytics firm SAS in November.

Managing FX risk with the help of big data
VIDEO4:2804:28
Managing FX risk with the help of big data

Types of professionals wanted

"Data scientists," or engineers able to manipulate data analytics for business purposes, have experienced the most demand, as seen by the number of data science programs offered by global universities. However, McKinsey warns that an increasingly multi-disciplinary approach is needed.

"What if number crunchers aren't enough? After all, if a great insight derived from advanced analytics is too complicated to understand, business managers just won't use it. That's why companies need to recruit and cultivate translators-specialists capable of bridging different functions within the organization and effectively communicating between them," the firm said in a note.

McKinsey advises companies to look for translators with complementary skills sets, such as computer programming and finance, statistics and marketing, or psychology and economics. "In all but the rarest of cases, you'll need at least two translators to bridge each pair of functions," they added.

Read MoreBig data could help your health

Data forensics is another equally important field, said Richard Brown, big data analytics global program director at Capgemini, in a note earlier this year, as the world increasingly uses data from disparate sources, which increases the need for accuracy.

"The biggest challenge is often the provenance of the source data," Brown said. "We need to have the skills to understand the origins and validity of our source data. But this is complex subject. Data sources need to be assessed in three dimensions: provenance, legality and sensitivity," he added.

"We need to establish what the level of quality actually is and adjust how we treat that data in our analytical models accordingly," he said.