Nature of the Opportunity/Problem

Data Science and the Business Are Not on the Same Page

A lack of alignment exacerbates challenges

Data teams often struggle due to poor alignment within the organization. Only 40% of companies surveyed indicated a highly mature organizational alignment.^v

Figure 4 Source: Algorithmia 2021 Enterprise Machine Learning Trends

The C-Suite and other upper-level management often initiate data science projects in large enterprises. Top-down directives get things done, but often lack the nuance required to ensure things get done the right way.^vi

All too often, data teams end up operating far from the business problems, so they can’t develop the solutions that meet business needs.^vii

Figure 5 Providing the business with effective data product^viii

The lack of alignment creates technical challenges. Top-down efforts often result in data silos and technical debt.^ixData teams cannot operate effectively in chaotic environments, and 49% of organizations struggle to integrate ML into existing technologies.^x Complexity plagues companies as they adopt data science, resulting in higher deployment times year-over-year.^xi

Figure 6 Time to deploy models is increasing.^xii

Staffing Crisis

Building effective data teams requires the right balance

Companies struggle to staff data science teams with the right mix of talent. On average, organizations fill open data science positions in 62 days (70.2 days for senior data scientists)—more than double the industry average for all IT jobs in the U.S.^xii

In a 2019 enterprise AI adoption survey, over half of the respondents indicated a need for data scientists and machine learning experts, and 47% indicated the need for more employees able to identify use cases that lend themselves to AI application.^xiii

Figure 7 Source AIHR^xiv

Companies hired 76% more data scientists year-over year, tightening conditions in an already competitive labor market.^xv

Figure 8 Source: Algorithmia 2021 Enterprise Machine Learning Trends

Misguided management compounds staffing shortages and leads to inefficient teams of data scientists. Enterprises starting out often make the mistake of staffing their data team entirely with data scientists. “This mistake is like trying to get a band together with just a lead singer. That might work if you’re going to sing a cappella. If you’re going to want any musical accompaniment, you’ll need the rest of the band.” (Anderson)

With little to no technical support, data scientists in these scenarios get stuck in a quagmire of technical debt, data cleaning, and deployment issues. At many enterprises, data scientists spend more than half their time on deployment. Unfortunately, success only begets more problems. As the team deploys more models, they spend more time on deployment and less on data science.

Figure 9 Source: Algorithmia 2021 Enterprise Machine Learning Trends

Figure 10 Source: Algorithmia 2021 Enterprise Machine Learning Trends

The narrowing labor market and increasing number of use cases create a crisis for management. Hiring larger teams of data scientists will not deliver the value large enterprises expect. Executives must structure their data teams in a way that captures value more efficiently.

Figure 11: Biggest skill gaps in enterprise Machine Learning.^xvii