The Big Data Gap: Beyond Supply & Demand, Into Open Innovation
The promise of Big Data is staggering, yet there is a growing challenge afoot. There is a sincere shortage of quality algorithmists and data scientists to create value from all of this data. Furthermore, a 20th Century approach to this challenge simply won’t make the grade any longer. A rising alternative, Enterprise Open Innovation, is showing us the promise for Big Data through Open Innovation is the likely vessel to help bridge this talent gap.
The Big Data Challenge
Luckily, the challenge is rather recognizable; there aren’t enough hyper-skilled people with the expertise needed to create value from all of this data. From a recent Gartner article analyzing the emerging role of Data Scientist, Gartner, points to the looming talent scarcity.
“With the need for data scientists growing at about 3x those for statisticians and BI analysts…. and an anticipated 100,000+ person analytic talent shortage through 2020… “
The same article defines Data Scientists as having:
“… three core data science skills: data management, analytics modeling and business analysis. But beyond these, there’s an art to data science. We detail several soft skills that our research showed are also critical to success, i.e., communication, collaboration, leadership, creativity, discipline and passion (for information and truth).”
In short, there is a legitimate supply issue of talent at stake and these sought after individuals are emerging as more and more impactful to the enterprise. Want an easy way to visualize the exploding need for data “talent” – see below:
A traditional enterprise and economic approach would dictate that with this explosion of data creation (demand), the supply (data talent) will innately increase to meet the demands of the market. There is no doubt whatsoever that universities across the globe are ramping up their efforts to cater to this new demand and will begin producing more talent. However, the Big Data phenomenon is different than any other in any time in our history. There are 3 reasons for this:
New data is being produced at a pace not yet experienced. The size of existing data is said to double every 2 years and recent estimates say it’s actually closer to a single year. This data creation – the ability to create value from it all – is keeping demand far, far ahead of any possible traditional supply (talent). Supply (talent) is increasing, but the demand (driven by data) is increasing at a much faster pace – the Big Data gap is actually widening.
The global economy has never been as strong in the history of the world. Emerging markets in Africa, maturation of markets in India, China, Brazil, Columbia and Eastern Europe all feed into this global demand. Sure, this maturation means a great deal more educated individuals also entering the market, but again, the demand for data talent will greatly outpace the supply.
Big Data – and value creation from it – touches every single industry, period.
This gap in data emergence versus value creation from Big Data is set to only widen over the coming years. A 20th Century, traditional approach to this challenge will fall woefully short. The answer to this challenge lies in Enterprise Open Innovation and its ability to consistently breed extreme value outcomes.
Extreme Value Outcomes through Enterprise Open Innovation
What is an extreme value outcome? These are your outliers, solutions that are by a long stretch far more powerful, faster, smarter than the average solution or an existing “gold standard”. They have been described in simplistic terms as “the needle in the haystack”. Through Enterprise Open Innovation practices the ability to consistently draw out extreme value outcomes is now a reality. This is a legitimate innovation “flattener” because it eliminates the need to source, court and retain top global talent in order to produce the extreme outcome(s). This is true for all the ways Open Innovation can be utilized and is especially realized in this Big Data arena where talent scarcity is evident and the outcomes can be extraordinarily impactful to the enterprise.
- Demand will continue to greatly outpace supply of talent in Big Data
- Through extreme value outcomes, you can effectively bridge this “gap” and focus on valued outputs
- They – extreme value outcomes – are the consistent consequence of EOI (Enterprise Open Innovation) practices
As a leader, you can choose to compete for talent as you did in the 20th Century or you can shift, concentrate on value delivery and embrace Enterprise Open Innovation. A genuine shift is required.
Technology, Open Innovation, the Future: Subscribe to the TopCoder Blog (top left)
image credit: indeed.com