The Surprising Reason Most AI Projects Fail – And How to Avoid It at Your Enterprise
Last Published: Feb 19, 2025 |
Table Of Contents
AI and generative AI (GenAI) is proving to be revolutionary for organizations across all verticals, from healthcare to retail. The McKinsey Global Survey on AI (2024) revealed that 65% of respondents expect GenAI to lead to significant or disruptive change in their industries and 72% of organizations are now using AI in at least one business function, up from 50% in previous years.1
AI spending has surged to $13.8 billion in 2024,2 a six-fold increase from $2.3 billion in 2023,3 with, having adopted or plan to adopt GenAI, intending to increase their AI investments in 2025," according to Informatica’s CDO Insights 2025 survey report.4
The reason for this surge in interest and investments is, of course, the concrete business results AI delivers in the context of efficiency and productivity.
A recent study of 35,000 workers in 27 leading economies found that employees using GenAI for administrative and routine tasks save an average of 1 hour a day,5 and a fifth said it was saving them as many as 2 hours a day. The responses offer evidence that not only does AI free workers up to focus on more strategic work, but also improves their productivity.
All of this confirms that GenAI and AI is here to stay. But as much as investments in AI are growing, we are also witnessing high failure rates of AI projects, which are either abandoned due to unreliable outcomes or an inability to deliver results at scale.
As per some estimates, over 80% of AI projects fail — twice the rate of failure for information technology projects that do not involve AI.6 According to Gartner®, “The survey found that, on average, only 48% of AI projects make it into production, and it takes 8 months to go from AI prototype to production.”7 And “At least 30% of generative AI (GenAI) projects will be abandoned after proof of concept by the end of 2025, due to poor data quality, inadequate risk controls, escalating costs or unclear business value, according to Gartner, Inc.”8
Understanding how to translate AI's enormous potential into concrete results remains an urgent challenge. But first, data leaders must address the root cause of AI failure.
A lack of data.
A Lack of Data in a Time of Data Overload?
In the age of unparalleled data volume, velocity and variety, it seems unlikely that a “lack of data” would be the reason for AI projects to fail. But it’s true.
GenAI being the dominant type of AI solution deployed in organizations today. According to Gartner, one of the Top Barriers to Implement AI Techniques is “Lack of data” for “39%” of survey participants.9
But reading between the lines reveals what they really mean is not ‘a lack of data’, but a lack of ‘AI-ready data’.
The global CDO Insights 2025 survey offers insights on specific factors leading to these failures, citing the top obstacles as data quality and readiness (43%), the lack of technical maturity (43%) and the shortage of skills and data literacy (35%).10
While most organizations have invested over the years in traditional data management architectures and practices, many fail to realize that AI-ready data management is a fundamentally distinct practice.
Recognizing and addressing these distinct requirements will help you make the leap to AI success more efficiently and effectively.
3 Critical Insights About AI-Ready Data Management Every CDO Must Know
The gap between traditional data management and AI-ready data management is causing the rampant failure and low scalability of AI projects across industries.
“Over 75% of organizations state that AI-ready data remains one of their top five investment areas in the next two to three years, according to Gartner’s 2024 Evolution of Data Management as a Dedicated Function Survey.”11
However, forging the distance between traditional data to AI-ready data is not as straightforward as upgrading current data management practices.
Data leaders must first understand and address 3 fundamental distinctions of AI-ready data management.
1. AI-ready data management is a dynamic and contextual practice.
There is no one-size-fits-all formula.
Unlike the business-ready data management frameworks where pipelines could be created and forgotten, ensuring AI-ready data is not a one-time ‘fit and forget’ operation. It is an always changing, always evolving kind of data that needs to be constantly managed.
According to Informatica’s CDO Insights survey, data leaders emphasize the need for a dynamic approach to data management, with a strong focus on training and data literacy upgrades. It is not something you can build once and for all, nor something that you can build ahead of time for all your data. Its definition and function will vary because AI use-case development is iterative and dynamic.
Just having access to business-ready data is no longer enough. It must be ‘AI-ready’ data. But what makes this challenging is that there is no one definition of AI-ready data. AI is contextual to your organization. It means different things to different enterprises based on their data maturity, their skill levels and known and anticipated use-cases.
AI-ready data has to be understood, enabled, operationalized at scale and governed in the context of AI use-cases and not traditional data management practices.
These 5 key questions could help you understand your unique context for AI-ready data:
- What are my current data data management practices and how mature are they?
- How should I evolve my data management practice to support AI-ready data?
- What technologies, platforms, architectures and skills are needed to support AI-ready data?
- What will be the impact of these GenAI and AI use case demands on the current data management tools, practices and skills?
- How do I govern and scale AI ready data and mitigate risks in my industry and market?
2. AI-ready data is more than ‘high-quality’ data as we know it.
Traditional parameters of data quality are necessary but not sufficient.
Poor quality data is becoming the biggest roadblock for AI success in terms of projects going into production or scaling the project. But this is not quality as we know it.
AI is a double-edged sword that brings a host of new regulations and compliances in how data is applied. Enterprise context and metadata intelligence create the foundation for success.
Quality in the context of AI-ready data means the data must be:
Fit for purpose: Each kind of AI use case requires a specific set of structured and unstructured data. GenAI and LLMs also have different needs. AI-ready data means use-case specific data. Only robust metadata helps organize the data and make it easily available to all users without straining the system at any scale.
Representative: Traditional notions of high quality refer only to accurate and reliable data for analytics use cases. However, AI-ready data may also include outliers and poor-quality data to train the model. Metadata tracks the origin, history and transformations of data, and along with data governance, helps ensure clarity and traceability on all the kinds and versions of data being used to train models.
Open-ended, dynamic and iterative: While standard analytics use cases can have high-quality data flows, based on predefined definitions and frameworks for predictable outcomes, AI-ready data means you need a way to change the data at any point, based on the outcomes.
Able to handle new governance, privacy and compliance standards: Privacy and security protocols for AI use cases are still evolving and what is high-quality data today may not even be usable tomorrow.
Metadata management and governance at the core of AI-ready data management not only ensures data quality in terms of consistency, accuracy and compliance, but also ensures context and meaning, lineage and provenance, and efficient data discovery and access for AI use-cases.
3. AI is revolutionary, but the path to AI-ready data must be evolutionary.
There is no fit-and-forget solution.
There is no magic bullet to deliver AI-ready data. It will not come from flipping any single switch. It is the result of a rock-solid data foundation that can handle current and future unknown workloads, use-cases and capabilities with ease and efficiency.
With a bulk of effort going into exploratory data analysis and data preparation including RAG, feature selection, prompt engineering and governance, the actual data modeling is just a small portion of the AI effort. The real supercharger for AI is data management.
Fortunately, the easy availability of pre-built GenAI models and LLMs means companies can shift their focus to build an AI-ready data foundation rather than on the models themselves.
Building a strong AI-ready data management foundation ensures you can build GenAI applications which are grounded, contextualized, reliable, secure and easy to develop and deploy.
Key Elements of a Strong AI-Ready Data Foundation
What Got You Here Won’t Get You There
Traditional data management frameworks worked well for predictable analytics use cases, but when it comes to training AI models, the risk of unreliable data is clearly creating a confidence issue, slowing down or ending AI projects, especially at scale.
When still-evolving AI models are connected to traditional data management frameworks, it can lead to unpredictable outcomes ranging from hilarious12 to downright dangerous or libelous, such as patient misdiagnosis or attorney sanctions.
An AI-ready data management foundation includes key elements such as:
- Data integration for data access and delivery
- Data quality for reliable and accurate data
- Data observability and metadata management for data monitoring, lineage, impact analysis and semantic support
- Data governance to ensure trust and reduce bias
- Data infrastructure to store, process, and manage large volumes of data.
The right data management platform not only helps build an AI-ready data foundation but also ensures enterprise-scale access to AI-ready data, which is at once relevant, responsible and reliable.
Relevant
AI-ready data is accurate, transparent and contextual, leveraging a universal metadata foundation to help deliver AI answers tailored to your unique business.
- Leverage one metadata system of record, cataloging all data and metadata from various sources, including traditional DBs, cloud apps, data pipelines and all major public cloud platforms.
- Enable data reliability and a common language for data that understands the business’ unique terms, policies and processes shared across the enterprise.
- Discover, resolve and publish trusted data assets, and map their relationships (i.e. customers to purchases) to enhance AI insights.
- Provide self-service access to data products, increasing collaboration among data teams supporting AI models.
Responsible
AI-ready data is governed, democratized and secure, aligning to set standards, helping you deliver AI that is compliant, private and unbiased.
- Build stakeholder trust, allowing appropriate data access to the right roles at the right time while reducing compliance risk following regulations (EU AI Act), standards and policies.
- Help ensure AI operations are explainable and accountable, tagging structured/unstructured data to protect sensitive information.
- Drive observability across data pipelines to detect anomalies and take action.
- Uphold enterprise-level security with data management services that comply with the highest industry standards.
Reliable
AI-ready data is complete, resilient, enterprise-scale and consistent, making AI more powerful and reliable.
- Efficiently prepare and integrate structured/unstructured data from various sources for AI models using codeless data and application integration.
- Improve prediction accuracy through effortless data cleansing across countries, regions or industries via data quality accelerators.
- Drive accuracy and consistency by providing a golden data record across customer, product and supplier data to ground AI applications.
- Reduce noisy data with prebuilt rules for data profiling, ensuring validity and completeness through observation monitoring.
The Bonus Secret to GenAI Success
AI-Powered Data Management for AI-Ready Data
The key to GenAI success is not just AI-ready data but also leveraging AI to improve data management itself. Applying AI to data management capabilities will improve productivity through automation, efficiency and accuracy — ensuring that your data is fit for AI.
GenAI-powered data management can help scale GenAI projects, reducing months of work down to instantaneous data access and accelerating time to value.
Informatica Intelligent Data Management Cloud (IDMC), powered by CLAIRE® Copilot is your path to relevant, responsible and robust AI-ready data.
CLAIRE GPT — a new conversational interface to data management within IDMC — simplifies data management through GenAI-powered natural language, enabling technical and non-technical staff to perform advanced data management tasks and automate complex workflows.
IDMC is the industry’s only comprehensive and integrated cloud data management platform that is built on a common metadata foundation, driving the most advanced AI-powered capabilities to automate thousands of tasks.
In an unpredictable and still-evolving AI future, IDMC works seamlessly with all types of multi-vendor, multi-cloud, and hybrid environments, freeing you from vendor lock-ins and allowing you to adapt to any new technologies that emerge.
Companies such as Paycor, Citizens and Holiday Inn Club Vacation, which use IDMC powered by CLAIRE to automate data management tasks, are already making smarter decisions on data and AI projects, democratizing data access and ensuring best-in-class security and compliance protocols.
Want more insight into how Informatica can help your data become AI-ready? Visit us at www.informatica.com/claire.
1https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai?form=MG0AV3
2https://menlovc.com/2024-the-state-of-generative-ai-in-the-enterprise/?form=MG0AV3
3https://kpmg.com/kpmg-us/content/dam/kpmg/corporate-communications/pdf/2024/kpmg-genai-survey-august-2024.pdf?form=MG0AV3
4https://www.informatica.com/lp/cdo-insights-2025_5039.html
5https://www.adeccogroup.com/our-group/media/press-releases/ai-saves-workers-an-average-of-one-hour-each-day
6https://www.rand.org/pubs/research_reports/RRA2680-1.html
7Gartner Press Release, “Gartner Survey Finds Generative AI Is Now the Most Frequently Deployed AI Solution in Organizations,” May 7, 2024, https://www.gartner.com/en/newsroom/press-releases/2024-05-07-gartner-survey-finds-generative-ai-is-now-the-most-frequently-deployed-ai-solution-in-organizations. GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.
8Gartner Press Release, “Gartner Predicts 30% of Generative AI Projects Will be Abandoned After Proof of Concept by End of 2025,” July 29, 2024, https://www.gartner.com/en/newsroom/press-releases/2024-07-29-gartner-predicts-30-percent-of-generative-ai-projects-will-be-abandoned-after-proof-of-concept-by-end-of-2025.
9Gartner Press Release, “Gartner Survey Finds Generative AI Is Now the Most Frequently Deployed AI Solution in Organizations,” May 7, 2024, https://www.gartner.com/en/newsroom/press-releases/2024-05-07-gartner-survey-finds-generative-ai-is-now-the-most-frequently-deployed-ai-solution-in-organizations.
10https://www.informatica.com/lp/cdo-insights-2025_5039.html
11Gartner®, “A Journey Guide to Delivering AI Success Through ’AI-Ready’ Data,” Ehtisham Zaidi, Roxane Edjlali, October 18, 2024
12https://hothardware.com/news/car-dealerships-chatgpt-goes-awry-when-internet-gets-to-it