The Big Data Gap: Beyond Supply & Demand, Into Open Innovation

The promise of Big Data is staggering, yet there is a growing challenge afoot. There is a sincere shortage of quality algorithmists and data scientists to create value from all of this data. Furthermore, a 20th Century approach to this challenge simply won’t make the grade any longer. A rising alternative, Enterprise Open Innovation, is showing us the promise for Big Data through Open Innovation is the likely vessel to help bridge this talent gap.

The Big Data Challenge

Luckily, the challenge is rather recognizable; there aren’t enough hyper-skilled people with the expertise needed to create value from all of this data. From a recent Gartner article analyzing the emerging role of Data Scientist, Gartner, points to the looming talent scarcity.

“With the need for data scientists growing at about 3x those for statisticians and BI analysts…. and an anticipated 100,000+ person analytic talent shortage through 2020… “

The same article defines Data Scientists as having:

“… three core data science skills: data management, analytics modeling and business analysis. But beyond these, there’s an art to data science. We detail several soft skills that our research showed are also critical to success, i.e., communication, collaboration, leadership, creativity, discipline and passion (for information and truth).”

In short, there is a legitimate supply issue of talent at stake and these sought after individuals are emerging as more and more impactful to the enterprise. Want an easy way to visualize the exploding need for data “talent” – see below:

A traditional enterprise and economic approach would dictate that with this explosion of data creation (demand), the supply (data talent) will innately increase to meet the demands of the market. There is no doubt whatsoever that universities across the globe are ramping up their efforts to cater to this new demand and will begin producing more talent. However, the Big Data phenomenon is different than any other in any time in our history. There are 3 reasons for this:

• New data is being produced at a pace not yet experienced. The size of existing data is said to double every 2 years and recent estimates say it’s actually closer to a single year. This data creation – the ability to create value from it all – is keeping demand far, far ahead of any possible traditional supply (talent). Supply (talent) is increasing, but the demand (driven by data) is increasing at a much faster pace – the Big Data gap is actually widening.

• The global economy has never been as strong in the history of the world. Emerging markets in Africa, maturation of markets in India, China, Brazil, Columbia and Eastern Europe all feed into this global demand. Sure, this maturation means a great deal more educated individuals also entering the market, but again, the demand for data talent will greatly outpace the supply.

• Big Data – and value creation from it – touches every single industry, period.

Extreme Value Outcomes through Enterprise Open Innovation