The Evolution of AI for Data Operations

ISG recently published the 2025 ISG Buyers Guides for DataOps, providing an assessment of 51 software providers offering products used by data engineers, data scientists, and data and AI professionals to facilitate the use of data for analytics and AI needs. The DataOps Buyers Guide research generated three reports and five quadrants assessing providers in relation to overall DataOps, Data Observability, Data Orchestration, Data Pipelines and Data Products. By providing an assessment of all software providers with tools in the portfolio of DataOps, the research offers a unique perspective on the extent to which emerging capabilities are being adopted by software providers. Given the amount of noise being made by providers about AI, it’s easy to assume that all providers have already delivered AI-driven capabilities that automate and accelerate DataOps use-cases. However, the DataOps Buyers Guide research illustrates that, for many providers, support for AI functionality remains a work in progress.

ISG Research defines data operations (DataOps) as the application of agile development, DevOps and lean manufacturing by data engineering professionals in support of data production. It encompasses the development, testing, deployment and orchestration of data integration and processing pipelines, along with improved data quality and validity via data monitoring and observability and the delivery of data products. While the concept of DataOps was initially embraced by emerging startups, many longstanding providers of traditional data management tools have adopted DataOps capabilities and methodologies in recent years. Enterprises have also embraced DataOps concepts and products. I assert that through 2027 more than one-half of enterprises will have adopted agile and collaborative data operations practices to facilitate responsiveness, avoid repetitive tasks and accelerate AI initiatives. The 2025 Buyers Guide for DataOps evaluated 14 software providers and products on their ability to address data pipeline development, testing and deployment, data pipeline orchestration, data pipeline observability and data products. The research found Pentaho atop the list for overall DataOps, followed by Databricks and Informatica. Meanwhile Databricks, Informatica and IBM were found to be Leaders for Data Pipelines while Databricks, Alteryx and Boomi were found to be Leaders for Data Orchestration.

I previously wrote about the role that DataOps plays in improving trust in the data used to fuel analytics and support the development and deployment of applications driven by AI. Additionally, AI functionality is also critical to DataOps: by automating and accelerating mundane but necessary tasks, AI can enable DataOps teams to improve productivity and focus on higher-value tasks. More than one-half (54%) of participants in ISG’s 2025 Market Lens Data and AI Program Study cited the usability of data for AI applications as a significant data challenge. DataOps is a critical component of delivering AI-ready data, but AI is also required to efficiently orchestrate, manage and monitor the number of data pipelines and volume of data required to support enterprise AI initiatives.

Acceleration and automation have always been core components of the DataOps approach. They are two of the primary categories of general-purpose capabilities by which we assess all providers of DataOps software, alongside collaboration and ecosystem integration. It is no surprise, therefore, to see that 96% of the 51 providers assessed in the combined DataOps Buyers Guide research were graded A- or above for avoidance of repetitive tasks, while 92% were graded at the same level for simplification of processes. These are both potential outcomes of the use of AI, but can also be achieved through other means, such as a well-designed user experience. As such, to ascertain the level of AI-based automation and acceleration, it is better to evaluate DataOps providers for the use of AI to automate and accelerate specific DataOps tasks.

The results of the DataOps Buyers Guide research indicate that there are some key tasks where the use of AI has almost become table-stakes, while other tasks illustrate opportunities for software providers to differentiate. For example, more than three-quarters (77%) of the 26 providers assessed as part of our Data Pipelines Buyers Guide research were graded A- or above for the use of AI to automate and accelerate the development of data pipelines, while the same proportion (77%) of the 26 providers assessed as part of our Data Orchestration Buyers Guide research were graded A- or above for the use of AI to automate and accelerate the management of data pipelines.

These are both use-cases that can be addressed with general-purpose AI assistants or copilots, providing guidance based on documentation or best practices. The use of AI is less widespread for narrower task-specific functionality. For example, fewer than two-fifths (38%) of providers assessed as part of our Data Pipelines Buyers Guide research were graded A- or above for the use of AI to automate and accelerate the documentation of data pipelines, despite it being perhaps one of the most time-consuming tasks required of data professionals.

AI has long been a key component of data observability software, which emerged in response to the growing complexity of enterprise data ecosystems and involves the application of machine learning and statistical modeling to automate anomaly detection and root cause analysis and the generation of alerts, explanations and recommendations to help data engineers and architects address issues quickly or prevent them from recurring. The ISG Buyers Guide for Data Observability evaluated 19 providers, with Monte Carlo, Pentaho and Acceldata found to be Leaders. More than two-thirds (68%) of the providers assessed were graded A- or above for the use of AI to automate and enhance data reliability issue detection, while 58% were graded at the same level for the use of AI to automate and enhance data reliability issue resolution and the same percentage for data reliability issue prevention.

AI is also a key enabler of data products, which ISG Research defines as the outcome of data initiatives developed with product thinking and delivered as reusable assets that can be discovered and consumed on a self-service basis. The ISG Buyers Guide for Data Products evaluated 26 software providers and products in relation to the development, classification, consumption, discovery and management of data products with Databricks, Domo and Pentaho found to be Leaders. Almost two-thirds (62%) of providers assessed were graded A- or above for the use of AI to enhance data product consumption, most often via the use of natural language search interfaces. Additionally, more than half of providers assessed were graded A- or above for the use of AI to enhance data product classification.

As always, however, software products are only one aspect of delivering on the promise of DataOps. New approaches to people, processes and information are also required to deliver agile and collaborative development, testing and deployment of data and analytics workloads, as well as data operations. To improve the value generated from analytics and data initiatives, I recommend enterprises evaluating DataOps products also adopt processes and methodologies that support rapid innovation and experimentation, as well as automation, collaboration, measurement and monitoring.

Regards,

Matt Aslett