Friction Points in Organizational Data and AI Projects

Altair Engineering Inc, a global leader in computational science and artificial intelligence (AI), released results from an international survey which revealed high rates of adoption and implementation of organizational data and AI strategies globally. The survey also revealed that project successes suffer due to three main types of friction: organizational, technological, and financial.

“Organizations today recognize the imperative of using their data as a strategic asset to create competitive advantages,” said James R Scapa, Founder and Chief Executive Officer, Altair. “But friction points clearly exist around people, technology, and investment, preventing organizations from gaining the data-driven insights needed to deliver results. To achieve what we call ‘Frictionless AI’, businesses must make the shift to self-service data analytics tools that empower non-technical users to work easily and cost-effectively across complex technology systems and avoid the friction inhibiting them from moving forward.”

The independent survey of more than 2,000 professionals in 10 countries and multiple industries showed a high failure rate of AI and data analytics projects (between 36 percent and 56 percent) where friction between organizational departments exists.

The Three Main Areas of Friction

Overall, the survey identified organizational, technological, and financial friction as the main culprits hindering data and AI project success.

Organizational Friction: The survey found organizations are struggling to fill data science roles, which is a significant cause of friction. 75 percent of respondents say they struggle to find enough data science talent; 35 percent say AI literacy is low among the majority of their workforce; and 58 percent say the shortage of talent and the time it takes to upskill current employees is the most prevalent problem in their AI strategy adoption.

Technological Friction: More than half of respondents say their organization often faces technical limitations that are slowing down data and AI initiatives. Overall, respondents struggle most with data processing speed, making informed decisions quickly, and experiencing data quality issues; Almost two-thirds of respondents (63 percent) said their organization tends to make working with AI-driven data tools more complicated than it needs to be; and 33 percent cited legacy systems’ inability to develop advanced AI and machine learning initiatives as a recurring technology-related issue that causes friction.

Financial friction: Despite organizations’ desire to scale their data and AI strategies, teams and individuals keep hitting financial obstacles. 25 percent of respondents cited financial constraints as a point of friction that negatively affects AI initiatives within their organization; 28 percent said leadership is too focused on the strategies’ upfront costs to understand how investing in AI and machine learning would benefit their organization; and 33 percent said the “high cost of implementation”—whether real or perceived—is one of their organization’s shortfalls when relying on AI tools to complete projects.

Project Failure is Common, but Optimism Reigns

Organizations across industries and geographic regions using AI persist despite high project failure rates.

One in four respondents said more than 50 percent of their projects fail

42 percent of respondents admit they experienced AI failure within the past two years; among those respondents, the average failure rate was 36 percent at their organization
Despite experiencing AI project failures, organizations continue to use AI because they believe there is still an opportunity to level up capabilities or services in the long run (78 percent), and its minor successes have shown potential for long-term breakthroughs (54 percent)

Many organizations struggle to complete their data science projects as well. 33 percent of respondents said more than half of their data science projects never made it to production in the last two years. Moreover, 55 percent said more than a third of their data science projects never made it to production within the past two years, and a staggering 67 percent said more than a quarter of projects never made it to production.

Friction Exists Around the World

Globally, the survey revealed that both technology and talent are pain points for organizations when deploying organizational data and AI strategies.

Respondents in the Asia-Pacific (APAC) and Europe-Middle East (EMEA) regions reported experiencing more AI failure in the last two years (54 percent and 35 percent) compared to the North-South America (AMER) region (29 percent)
65 percent of APAC respondents and 61 percent of EMEA respondents agreed their organization makes working with AI tools more complicated than needed
78 percent of APAC respondents and 75 percent of EMEA respondents said they struggle to find enough data science talent

What is Frictionless AI?

When organizations achieve ‘Frictionless AI’, data analytics becomes an easy, natural part of their business with projects that are quick, repeatable, and scalable. There is no technical friction between them and their data; no organizational friction between data experts and domain experts; no workflow friction between data application design and production deployment for effective decision making; and no migration friction when infrastructure or tools change.

The global survey was commissioned by Altair and conducted by Atomik Research between March 14-31, 2023. 2,037 professionals responded across several target industries with job functions related to data and data analytics. The sample consisted of participants from 10 different countries across the globe, including the United States, China, France, Germany, India, Italy, Japan, South Korea, Spain, and the United Kingdom.

Image Source: Magic Wand Media