ISG Buyers Guide for Data Quality Classifies and Rates Software Providers

ISG Research is happy to share insights gleaned from our latest Buyers Guide, an assessment of how well software providers’ offerings meet buyers’ requirements. The Data Quality: ISG Research Buyers Guide is the distillation of a year of market and product research by ISG Research.

Maintaining data quality and trust is a perennial data-management challenge, often preventing enterprises from operating at the speed of business. As organizations aspire to be more data-driven, trust in the data used to make decisions becomes more critical. Without data quality processes and tools, enterprises may make decisions based on old, incomplete, incorrect or poorly organized data. Assessing the quality of data used to make business decisions is not only more important than ever but also increasingly difficult, given the growing range of data sources and the volume of data that needs to be evaluated.

ISG Research defines data quality as both a discipline and a product category. As a discipline, data quality refers to the processes, methods and tools used to measure the suitability of a dataset for a specific purpose. The precise measure of suitability will depend on the individual use case, but important characteristics include accuracy, completeness, consistency, timeliness and validity. The data quality product category is comprised of the tools used to evaluate data in relation to these characteristics. The potential value of data quality products is clear: Poor data quality processes can result in security and privacy risks as well as unnecessary data storage and processing costs due to data duplication. Additionally, assessing the quality of data is one of the most time-consuming aspects of analytics initiatives. Almost two-thirds of enterprises participating in our Analytics and Data Benchmark Research cite reviewing data for quality and consistency issues as the most time-consuming task of analyzing data, second only to preparing data for analysis.

Traditionally, the data quality product category has been dominated by standalone products specifically focused on the requirements for assessing data quality. However, data quality functionality is also an essential component of data intelligence platforms that provide a holistic view of data production and consumption, as well as products that address other aspects of data intelligence, including data governance and master data management.

In recent years, we have seen the emergence of data observability focused on monitoring the quality and reliability of data used for analytics and governance projects and associated data pipelines. While we consider data observability to be a subset of Data Operations, as covered in our DataOps Buyers Guides, there is a clear overlap with data quality. Although data quality is more established as a discipline and product category for improving trust in data, enterprises that have invested in data quality might reasonably ask whether data observability is necessary. Businesses that have invested in data observability, however, might wonder whether to eschew traditional data quality tools.

Data quality software is designed to help users identify and resolve data quality problems, typically related to a given task. For example, data quality software will be applied to assess the validity of data being used to serve a business intelligence report or dashboard to ensure the data is valid.

In comparison, data observability software focuses on automating the monitoring of data to assess its health based on key attributes, including freshness, distribution, volume, schema and lineage. It is concerned with the reliability and health of the overall data environment. Data observability tools monitor the data in an individual environment for a specific purpose at a given point in time but also monitor the associated upstream and downstream data pipelines. In doing so, data observability software ensures that data is available and up-to-date, avoiding downtime caused by lost or inaccurate data due to schema changes, system failures or broken data pipelines.

While data quality software is designed to help users identify and resolve specific data quality problems, data observability software automates the detection and identification of the causes of data quality problems, enabling users to prevent data quality issues before they occur. For example, as long as the data being assessed remains consistent, data quality tools might not detect a failed pipeline until the data has become out-of-date. Data observability tools could detect the failure long before the data quality issue arose. Conversely, a change in a customer’s address might not be identified by data observability tools if the new information adhered to the correct schema. It could be detected—and remediated—using data quality tools.

Data quality and data observability software products are, therefore, largely complementary. Some providers offer separate products in both categories, while others provide individual products that could be said to include functionality associated with both data observability and data quality. Potential customers are advised to pay close attention and evaluate purchases carefully. Some data observability products offer quality resolution and remediation functionality traditionally associated with data quality software, albeit not to the same depth and breadth. Additionally, some providers previously associated with data quality have adopted the term data observability but may lack the depth and breadth of pipeline monitoring and error detection capabilities.

Automation is often cited as a distinction between data observability and data quality software. This, however, relies on an outdated view of data quality software. Although data quality software has historically provided users with an environment to check and correct data quality issues manually, the use of machine learning (ML) to automate the monitoring of data is being integrated into data quality tools and platforms to ensure that data is complete, valid and consistent as well as relevant and free from duplication.

In addition to data observability tools, potential customers should pay close attention to the data quality functionality offered by data intelligence, data governance and master data management platforms. Data intelligence platforms are likely to provide a superset of functionality addressing data quality, master data management and data governance as well as application and data integration. In comparison, while dedicated data governance and master data management products may offer some capabilities for assessing data quality, they may also be used alongside standalone data quality tools. Through 2026, three-quarters of enterprises will be engaged in data integrity initiatives to increase trust in data processes using data quality and master data management tools.

Our Data Quality Buyers Guide is designed to provide a holistic view of a software provider’s ability to deliver the combination of functionality that provides a complete view of data quality with either a single product or suite of products. As such, the Data Quality Buyers Guide includes the full breadth of data quality functionality. Our assessment also considered whether the functionality in question was available from a software provider in a single offering or as a suite of products or cloud services.

The ISG Buyers Guide™ for Data Quality evaluates products based on data profiling, data quality rules and data quality insights. To be included in this Buyers Guide, products must have capabilities that address the configuration of data quality software. The evaluation also assessed the use of artificial intelligence to automate and enhance data quality.

This research evaluates the following software providers that offer products that address key elements of data quality as we define it: Actian, Alation, Alibaba Cloud, AWS, Ataccama, Cloud Software Group, Collibra, Experian, Google Cloud, Huawei Cloud, IBM, Informatica, Melissa Data, Oracle, Precisely, Qlik, Quest Software, Reltio, SAS Institute and Syniti.

This research-based index evaluates the full business and information technology value of data quality software offerings. We encourage you to learn more about our Buyers Guide and its effectiveness as a provider selection and RFI/RFP tool.

We urge organizations to do a thorough job of evaluating data quality offerings in this Buyers Guide as both the results of our in-depth analysis of these software providers and as an evaluation methodology. The Buyers Guide can be used to evaluate existing suppliers, plus provides evaluation criteria for new projects. Using it can shorten the cycle time for an RFP and the definition of an RFI.

The Buyers Guide for Data Quality in 2024 finds Informatica first on the list, followed by IBM and Oracle.

Software providers that rated in the top three of any category ﹘ including the product and customer experience dimensions ﹘ earn the designation of Leader.

The Leaders in Product Experience are:

• Informatica
• Oracle
• IBM

The Leaders in Customer Experience are:

• Oracle
• AWS
• IBM

The Leaders across any of the seven categories are:

• Oracle and Informatica, which have achieved this rating in five of the seven categories.
• IBM in three categories.
• Actian and Google Cloud in two categories.
• Alation, AWS, Experian and Qlik in one category.

The overall performance chart provides a visual representation of how providers rate across product and customer experience. Software providers with products scoring higher in a weighted rating of the five product experience categories place farther to the right. The combination of ratings for the two customer experience categories determines their placement on the vertical axis. As a result, providers that place closer to the upper-right are “exemplary” and rated higher than those closer to the lower-left and identified as providers of “merit.” Software providers that excelled at customer experience over product experience have an “assurance” rating, and those excelling instead in product experience have an “innovative” rating.

Note that close provider scores should not be taken to imply that the packages evaluated are functionally identical or equally well-suited for use by every enterprise or process. Although there is a high degree of commonality in how organizations handle data quality, there are many idiosyncrasies and differences that can make one provider’s offering a better fit than another.

ISG Research has made every effort to encompass in this Buyers Guide the overall product and customer experience from our data quality blueprint, which we believe reflects what a well-crafted RFP should contain. Even so, there may be additional areas that affect which software provider and products best fit an enterprise’s particular requirements. Therefore, while this research is complete as it stands, utilizing it in your own organizational context is critical to ensure that products deliver the highest level of support for your projects.

You can find more details on our community as well as on our expertise in the research for this Buyers Guide.