Analyst Perspectives

I recently described how business data catalogs are evolving into data intelligence catalogs. These catalogs combine technical and business metadata and data governance capabilities with knowledge graph functionality to deliver a holistic, business-level view of data production and consumption. The concept of the knowledge graph has been part of the data sector for decades, but adoption has typically been limited to industries and enterprises focused on the Semantic Web, such as media,...

Domo is best known as a business intelligence (BI) and analytics software provider, thanks to its functionality for visualization, reporting, data science and embedded analytics. Additionally, as I recently explained, the company’s platform addresses a broad range of capabilities that includes data governance and security, data integration and application development, as well as the automation and incorporation of artificial intelligence (AI) and machine learning (ML) models into BI and...

Data governance has always been a critical part of the data and analytics landscape. However, for many years, it was seen as a preventive function to limit access to data and ensure compliance with security and data privacy requirements. To fulfill today’s data-driven agendas, many enterprises need an evolved perspective on data governance. The development of new applications driven by artificial intelligence requires a more agile and collaborative approach to data governance—one that automates...

It has been a little over a decade since the term data operations entered the analytics and data lexicon. It describes the application of agile development, DevOps and lean manufacturing by data engineering professionals in support of data production. DataOps was initially seen as antithetical to traditional data management approaches, which typically included batch-based and manual tools and practices. The term was embraced by emerging software providers as a means of differentiating from...

Natural language interfaces for business intelligence products existed long before the emergence of generative artificial intelligence. Large language models have allowed BI providers to accelerate the delivery of functionality to convert natural language questions into analytic queries and generate summarizations and recommendations from data and charts. Features that enable natural language query and natural language generation are now ubiquitous.

In an earlier Analyst Perspective, I discussed data democratization’s role in creating a data-driven enterprise agenda. Building a foundation of self-service data discovery, data-driven organizations provide more workers with the ability to analyze and use data. I’ve also examined how generative artificial intelligence (GenAI) could revolutionize business intelligence software by using natural language interfaces to lower the barriers to working with analytics software. Today, however, data...

As enterprises embrace the potential opportunities presented by artificial intelligence (AI), they are quickly finding that good data management is a prerequisite. As was explained in ISG’s State of Generative AI Market Report, AI requires data that is clean, well-organized and compliant with regulatory standards. There are multiple challenges to delivering AI-ready data, including combining structured and unstructured data, ensuring that the combined data can be trusted, and validating that...

Increased enterprise focus on artificial intelligence (AI) and generative AI (GenAI) has served to sharpen the focus on the need for trusted data and reliable analytics and data operations. The ISG State of Generative AI Market Report highlighted that elevated expectations and demands associated with AI are a forcing function for enterprises to take long-overdue steps to improve data and analytics processes to ensure that data that is clean, well-organized and compliant with regulatory...

Data catalogs provide an inventory of data assets that surface metadata from data platforms, analytics tools and applications that can be used to facilitate data discovery and data usage across an enterprise. As I recently explained, however, there are actually multiple types of data catalogs that offer functionality to address specific use cases and user roles, including data inventory, data discovery and data governance. The data intelligence catalog is an emerging category that combines...

As enterprises seek to expand and accelerate the adoption of artificial intelligence (AI) many are finding that longstanding analytics and data challenges are a barrier to success. As was explained in ISG’s State of Generative AI Market Report, AI requires data that is clean, well-organized and compliant with regulatory standards. The need for good data management is by no means new, but the expectations and demands associated with AI are a forcing function for enterprises to take long-overdue...

Late 2024 saw the publication of the 2024 ISG Buyers Guides for DataOps, providing an assessment of 49 software providers offering products used by data engineers, data scientists, and data and AI professionals to facilitate the use of data for analytics and AI needs. The DataOps Buyers Guide research includes five reports which are focused on overall DataOps, Data Observability, Data Orchestration, Data Pipelines and Data Products. This is the first time in the industry when all software...

It is now more than two years since the launch of ChatGPT introduced the world to generative AI (GenAI) and large language models (LLMs). GenAI-based assistants and co-pilots are now widely adopted, with individuals and enterprises adopting GenAI models to automate the generation of text, digital images, audio, video and code, amongst other things.

I recently wrote about the need for enterprises to harness events to process and act upon data at the speed of business. The core technologiesthat enable enterprises to process and analyze data in real time have been in existence for many years and are widely adopted. However, streaming and events technologies are also commonly seen as a niche requirement, separate from an enterprise’s primary focus on batch processing of data at rest. One of the reasons for this is an entrenched reliance on...

Metadata management has played a role in data governance and analytics for many years. It wasn’t until the emergence of the data catalog as a product category just over a decade ago that enterprises had a platform for metadata-driven data management that could span multiple departments and use cases across an entire enterprise.

Despite all the interest in artificial intelligence (AI) and generative AI (GenAI), ISG’s Buyers Guide for Data Platforms serves as a reminder of the ongoing importance of product experience functionality to address adaptability, manageability, reliability and usability. While new and emerging capabilities might catch the eye, features that address data platform security, performance and availability remain some of the most significant deal-breakers when enterprises are considering potential...

Too often, enterprises find that data is distributed across multiple silos on-premises and in the cloud. More than two-thirds of participants in ISG’s Market Lens Cloud Study are using a hybrid architecture involving both on-premises and cloud infrastructure for analytics and artificial intelligence deployments. Unifying data to achieve operational and analytic objectives requires complex data integration and management processes. Fulfilling these processes requires a smorgasbord of tools aimed...

I previously explained that data observability software has become a critical component of data-driven decision-making. Data observability addresses one of the most significant impediments to generating value from data by providing an environment for monitoring the quality and reliability of data on a continual basis. Maintaining quality and trust is a perennial data management challenge, the importance of which has come into sharper focus in recent years thanks to the rise of artificial...

The adoption of cloud environments for analytic workloads has been a key feature of the data platforms sector in recent years. For two-thirds (66%) of participants in ISG’s Data Lake Dynamic Insights Research, the primary data platform used for analytics is cloud based. Many enterprises adopted cloud-based analytic data platforms with a view to improving operational efficiencies by reducing the need for upfront investment in physical infrastructure as well as the ability to scale cloud services...

I previously wrote about the importance of open table formats to the evolution of data lakes into data lakehouses. The concept of the data lake was initially proposed as a single environment where data could be combined from multiple sources to be stored and processed to enable analysis by multiple users for multiple purposes.

Although the terms data fabric and data mesh are often used interchangeably, I previously explained that they are distinct but complementary. Data fabric refers to technology products that can be used to integrate, manage and govern data across distributed environments, supporting the cultural and organizational data ownership and access goals of data mesh. Data fabric and data mesh are also both related to logical data management, which is the approach of providing virtualized access to data...

As I noted in the 2024 Buyers Guide for Operational Data Platforms, intelligent applications powered by artificial intelligence have impacted the requirements for operational data platforms. These applications, infused with contextually relevant recommendations, predictions and forecasting, are driven by machine learning and generative AI.

As I recently noted, the term “data intelligence” has been used by multiple providers across analytics and data for several years and is becoming more widespread as software providers respond to the need to provide enterprises with a holistic view of data production and consumption. I assert that through 2027, three-quarters of enterprises will be engaged in data intelligence initiatives to increase trust in their data by leveraging metadata to understand how, when and where data is used in...

I previously wrote about data mesh as a cultural and organizational approach to distributed data processing. Data mesh has four key principles—domain-oriented ownership, data as a product, self-serve data infrastructure and federated governance—each of which is being widely adopted. I assert that by 2027, more than 6 in 10 enterprises will adopt technologies to facilitate the delivery of data as a product as they adapt their cultural and organizational approaches to data ownership in the...

As I explained in our recent Buyers Guide for Data Platforms, the popularization of generative artificial intelligence (GenAI) has had a significant impact on the requirements for data platforms in the last 18 months. While there is an ongoing need for data platforms to support data warehousing workloads involving analytic reports and dashboards, there is increasing demand for analytic data platform providers to add dedicated functionality for data engineering, including the development,...

The final of the men’s 100 meters at the Paris Olympics this summer was a reminder that being successful requires not just being fast but performing at the right time. Being fast is obviously a prerequisite for participating in an Olympic 100-meter final, and all the competitors finished the race in under 10 seconds, with just 0.12 seconds separating the first man from the last. While all the athletes were fast, what separated the winner of the gold medal—USA’s Noah Lyles—was execution. He was ...

Enterprises face a bewildering level of choice in relation to data platforms, as evidenced by the number of software providers and products assessed in our recent Data Platforms Buyers Guide. There are numerous data platform providers and products to choose from, but also a diverse array of functional and architectural options. Is the workload primarily operational or analytic? Will it be deployed on-premises or in the cloud? Should it be distributed or centralized? Data warehouse or data...

I have written on multiple occasions about the increasing proportion of enterprises embracing the processing of streaming data and events alongside traditional batch-based data processing. I assert that, by 2026, more than three-quarters of enterprises’ standard information architectures will include streaming data and event processing, allowing enterprises to be more responsive and provide better customer experiences.

The artificial intelligence and machine learning landscape was profoundly altered by the emergence of generative AI into the mainstream consciousness during 2023. The widespread availability of GenAI models and cloud services has lowered the barriers to individuals and enterprises engaging with AI for various use cases, including generating content, querying data, writing code, preparing data for analysis, documenting data pipelines and using software products more effectively. The impact that...

I am happy to share insights gleaned from our latest Buyers Guide, an assessment of how well software providers’ offerings meet buyers’ requirements. The Data Platforms Ventana Research Buyers Guide is the distillation of a year of market and product research by ISG and Ventana Research.

I previously wrote about the ongoing importance of event brokers and event management in enabling enterprises to adopt event-driven architecture and event stream processing. Many enterprises adopt EDA as the design pattern for maximizing events to deliver real-time business processes. There are many advantages to using EDA, including a cultural shift away from batch processing towards real-time analysis and decision-making.

I previously wrote about the potential for generative artificial intelligence technology to enhance the integration sector by facilitating outcome-driven approaches for automatically generating integration pipelines in response to declared business requirements. The use of GenAI in data and application integration remains nascent, but multiple software providers are embracing the potential for GenAI to improve the productivity of integration experts and facilitate self-service integration by...

I recently wrote about the role data observability plays in generating value from data by providing an environment for monitoring its quality and reliability. Data observability is a critical functional aspect of Data Operations, alongside the development, testing and deployment of data pipelines and data orchestration, as I explained in our Data Observability Buyers Guide. Maintaining data quality and trust is a perennial data management challenge, often preventing organizations from operating...

Enterprises are embracing the potential for artificial intelligence (AI) to deliver improvements in productivity and efficiency. As they move from initial pilots and trial projects to deployment into production at scale, many are realizing the importance of agile and responsive data processes, as well as tools and platforms that facilitate data management, with the goal of improving trust in the data used to fuel analytics and AI. This has led to increased attention on the role of data...

The emergence of generative artificial intelligence (GenAI) has significant implications at all levels of the technology stack, not least analytics and data products, which serve to support the development, training and deployment of GenAI models, and also stand to benefit from the advances in automation enabled by GenAI. The intersection of analytics and data and GenAI was a significant focus of the recent Google Cloud Next ’24 event. My colleague David Menninger has already outlined the key...

I recently wrote about the development, testing and deployment of data pipelines as a fundamental accelerator of data-driven strategies as well as the importance of data orchestration to accelerate analytics and artificial intelligence. As I explained in the recent Data Observability Buyers Guide, data observability software is also a critical aspect of data-driven decision-making. Data observability addresses one of the most significant impediments to generating value from data by providing an...

I previously wrote about the potential for rapid adoption of the data lakehouse concept as enterprises combined the benefits of data lakes based on low-cost cloud object storage with the structured data processing functionality normally associated with data warehousing. By layering support for table formats, metadata management and transactional updates and deletes as well as query engine and data orchestration functionality on top of low-cost storage of both structured and unstructured data,...

Many organizations have adopted DataOps to apply agile development, DevOps and lean manufacturing processes to the development, testing, deployment and orchestration of data integration and processing pipelines. The most likely ultimate outcome of these pipelines is the analytics reports and dashboards enterprises rely on to make business decisions.

Analytics software is used by business analysts and decision-makers to facilitate the generation of insights from data. It encompasses business intelligence and decision intelligence software, including reports and dashboards as well as embedded analytics and the development of intelligent applications infused with the results of analytic processes. Analytics software enables enterprises to improve business outcomes by operating more efficiently, accelerating product development and enhancing...

I recently wrote about the development, testing and deployment of data pipelines as a fundamental accelerator of data-driven strategies. As I explained in the 2023 Data Orchestration Buyers Guide, today’s analytics environments require agile data pipelines that can traverse multiple data-processing locations and evolve with business needs.

I previously explained how master data management helps provide trust in data, making it one of the most significant aspects of an enterprise’s strategic approach to data management. More recently, I discussed how it has a role to play in accelerating data democratization as part of data intelligence initiatives. Along with data quality, MDM enables organizations to ensure data is accurate, complete and consistent to fulfill operational business objectives. While it is an established and mature...

I wrote recently about the role that data intelligence has in enabling enterprises to facilitate data democratization and the delivery of data as a product. Data intelligence provides a holistic view of how, when, and why data is produced and consumed across an enterprise, and by whom. This information can be used by data teams toensure business users and data analysts are provided with self-service access to data that is pertinent to their roles and requirements. Delivering data as a product...

The development, testing and deployment of data pipelines is a fundamental accelerator of data-driven strategies, enabling enterprises to extract data from the operational applications and data platforms designed to run the business and load, integrate and transform it into the analytic data platforms and tools used to analyze the business. As I explained in our recent Data Pipelines Buyers Guide, data pipelines are essential to generating intelligence from data. Healthy data pipelines are...

As enterprises seek to increase data-driven decision-making, many are investing in strategic data democratization initiatives to provide business users and data analysts with self-service access to data across the enterprise. Such access has long been a goal of many enterprises, but few have achieved it. Only 15% of participants in Ventana Research’s Analytics and Data Benchmark Research say their organization is very comfortable allowing business users to work with data that has not been...

Cloud computing has had an enormous impact on the analytics and data industry in recent decades, with the on-demand provisioning of computational resources providing new opportunities for enterprises to lower costs and increase efficiency. Two-thirds of participants in Ventana Research’s Data Lakes Dynamic Insightsresearch are using a cloud-based environment as the primary data platform for analytics. 

It is well known that data integration, transformation and preparation represent a significant proportion of the time and effort required in any analytics project. Traditionally, operational data platforms are designed to store, manage, and process data to support worker-, customer- and partner-facing operational applications, and data is then extracted, transformed, and loaded (or “ETLed”) into a separate analytic data platform, which is designed to store, manage, process, and analyze data....

I have previously written about the impact of intelligent operational applications on the requirements for data platforms. Intelligent applications are used to run the business but also deliver personalization, recommendations and other features generated by machine learning and artificial intelligence. As such, they require a combination of operational and analytic processing functionality. The emergence of these intelligent applications does not eradicate the need for separate analysis of...

The increasing importance of intelligent operational applications driven by artificial intelligence (AI) is blurring the lines that have traditionally divided the requirements between operational and analytic data platforms. Operational data platforms have traditionally been deployed to support applications targeted at business users and decision-makers to run the business, with analytic data platforms typically supporting applications used by data and business analysts to analyze the business.

I have previously written about the functional evolution and emerging use cases for NoSQL databases, a category of non-relational databases that first emerged 15 or so years ago and are now well established as potential alternatives to relational databases. NoSQL is a term used to describe a variety of databases that fall into four primary functional categories: key-value stores, wide-column stores, document-oriented databases and graph databases. Each is worthy of further exploration, which is...

In recent years, many enterprises have migrated data platform workloads from on-premises infrastructure to cloud environments, attracted by the promised benefits of greater agility and lower costs. The scale of cloud data platform adoption is illustrated by Ventana Research’s Data Lakes Dynamic Insights research: For two-thirds (66%) of participants, the primary data platform used for analytics is cloud based. As the quantity and importance of the data platform workloads deployed in the cloud...

Ventana Research recently announced its 2024 Market Agenda for Analytics and Data, continuing the guidance we have offered for two decades to help enterprises derive optimal value and improve business outcomes.

Discussion about potential deployment locations for analytics and data workloads is often based on the assumption that, for enterprise workloads, there is a binary choice between on-premises data centers and public cloud. However, the low-latency performance or sovereignty characteristics of a significant and growing proportion of workloads make them better suited to data and analytics processing where data is generated rather than a centralized on-premises or public cloud environment. ...

As articulated in Ventana Research’s Data Platforms Buyer’s Guide and DataOps Buyer’s Guide research, the combination of cloud computing and advanced analytics has lowered the cost of storing and processing large volumes of data, accelerating the emergence of new data platform and data operations products that enable organizations to gain operational efficiency and competitive advantage. The right combination of data platform and data management products is essential to ensure that the right...

The phrase ‘big data’ may have largely gone out of fashion, but the concept of storing and processing all relevant data continues to be important for enterprises seeking to be more data-driven. Doing so requires analytic data platforms capable of storing and processing data in multiple formats and data models. This will be an important focus for the forthcoming Data Platforms Buyer’s Guide 2024. 

I recently discussed how fashion has a surprisingly significant role to play in the data market as various architectural approaches to data storage and processing take turns enjoying a phase in the limelight. Pendulum swing is a theory of fashion that describes the periodic movement of trends between two extremes, such as short and long hemlines or skinny and baggy/flared trousers. Pendulum swing theory is similarly a factor in data technology trends, with an example being the oscillation...

I previously described how Oracle had positioned its database portfolio to address any and all data platform requirements. The caveat to that statement at the time was that any organization wanting to take advantage of the company’s flagship Oracle Autonomous Database could only do so using Oracle Cloud Infrastructure (OCI) cloud computing service, their own datacenter or a hybrid cloud environment. The widespread popularity of Oracle Database and the advanced automation capabilities delivered...

I recently articulated some of the reasons why IT teams can fail to deliver on the business requirements for data and analytics projects. This is an age-old and multifaceted problem that is not easily solved. Organizations have a role to play in alleviating the issue by ensuring that their business processes and project planning support a collaborative approach in which business and IT professionals work together. Data and analytics product vendors can also help by delivering products that are...

I previously wrote about how document-database providers have added support for ACID transactions and the SQL query language, making their products increasingly suitable for use as replacements for applications that previously depended on relational databases. Adoption of non-relational NoSQL databases is by no means reliant on displacing incumbent relational databases, and initial adoption is often driven by differentiating capabilities, such as developer agility and application flexibility....

I have previously written about the functional evolution and emerging use cases for NoSQL databases, a category of non-relational databases that first emerged 15 or so years ago and are now well established as potential alternatives to relational databases. NoSQL is a term used to describe a variety of databases that fall into four primary functional categories: key-value stores, wide-column stores, document-oriented databases and graph databases. Each is worthy of further exploration, which is...

I previously wrote about the challenge facing distributed SQL database providers to avoid becoming pigeonholed as only being suitable for a niche set of requirements. Factors including performance, reliability, security and scalability provide a focal point for new vendors to differentiate from established providers and get a foot in the door with customer accounts. Expanding and retaining those accounts is not necessarily easy, however, especially as general-purpose data platform providers...

I previously discussed the trust and accuracy limitations of large language models, suggesting that data and analytics vendors provide guidance about potentially inaccurate results and the risks of creating a misplaced level of trust. In the months that have followed, we are seeing some clarity from these vendors about the approaches organizations can take to increase trust and accuracy when developing applications that incorporate generative AI, including fine-tuning and prompt engineering. It...

I am happy to share insights gleaned from our latest Buyers Guide, an assessment of how well vendors’ offerings meet buyers’ requirements. TheVentana Research 2023 Data Observability Buyers Guide is the distillation of a year of market and product research by Ventana Research. Drawing on our Benchmark Research, we apply a structured methodology built on evaluation categories that reflect the real-world criteria incorporated in a request for proposal to data vendors supporting the spectrum of...

The 2023 Ventana Research Buyers Guide for Data Observability research enables me to provide observations about how the market has advanced. 

I am happy to share insights gleaned from our latest Buyers Guide, an assessment of how well vendors’ offerings meet buyers’ requirements. The Ventana Research 2023 Data Orchestration Buyers Guide is the distillation of a year of market and product research by Ventana Research. Drawing on our Benchmark Research, we apply a structured methodology built on evaluation categories that reflect the real-world criteria incorporated in a request for proposal to data vendors supporting the spectrum of...

The 2023 Ventana Research Buyers Guide for Data Orchestration research enables me to provide observations about how the market has advanced. 

I am happy to share insights gleaned from our latest Buyers Guide, an assessment of how well vendors’ offerings meet buyers’ requirements. The Ventana Research 2023 Data Pipelines Buyers Guide is the distillation of a year of market and product research by Ventana Research. Drawing on our Benchmark Research, we apply a structured methodology built on evaluation categories that reflect the real-world criteria incorporated in a request for proposal to Data vendors supporting the spectrum of Data...

The 2023 Ventana Research Buyers Guide for Data Pipelines research enables me to provide observations about how the market has advanced. 

I am happy to share insights gleaned from our latest Buyers Guide, an assessment of how well vendors’ offerings meet buyers’ requirements. The Ventana Research 2023 DataOps Buyers Guide is the distillation of a year of market and product research by Ventana Research. Drawing on our Benchmark Research, we apply a structured methodology built on evaluation categories that reflect the real-world criteria incorporated in a request for proposal to Data vendors supporting the spectrum of DataOps....

I previously described how Databricks had positioned its Lakehouse Platform as the basis for data engineering, data science and data warehousing. The lakehouse design pattern provides a flexible environment for storing and processing data from multiple enterprise applications and workloads for multiple use cases. I assert that by 2025, 8 in 10 current data lake adopters will invest in data lakehouse architecture to improve the business value generated from the accumulated data.

The 2023 Ventana Research Buyers Guide for DataOps research enables me to provide observations about how the market has advanced. 

The data platforms market may appear to have little or nothing to do with haute couture, but it is one of the data sectors most strongly influenced by the fickle finger of fashion. In recent years, various architectural approaches to data storage and processing have enjoyed a phase in the limelight, including data warehouse, data mart, data hub, data lake, cloud data warehouse, object storage, data lakehouse, data fabric and data mesh. These approaches are often heralded as the next big thing,...

Despite a focus on being data-driven, many organizations find that data and analytics projects fail to deliver on expectations. These initiatives can underwhelm for many reasons, because success requires a delicate balance of people, processes, information and technology. Small deviations from perfection in any of those factors can send projects off the rails.

I have written before about the rising popularity of the data fabric approach for managing and governing data spread across distributed environments comprised of multiple data centers, systems and applications. I assert that by 2025, more than 6 in 10 organizations will adopt data fabric technologies to facilitate the management and processing of data across multiple data platforms and cloud environments. The data fabric approach is also proving attractive to vendors, including Microsoft, as a...

Organizations today have an ever-increasing appetite for platforms that improve the speed and efficiency of data analytics and business intelligence (BI). The ability to quickly process data enables organizations to make well-informed decisions in real time. This agile approach to data processing is crucial for staying ahead in today's competitive landscape. With the rising need for data-driven insights, organizations face the difficulty of dealing with massive volumes of distributed business...

I recently wrote about the various technologies used by organizations to process and analyze data in real time. I explained that while the terms streaming data and events and streaming analytics are often used interchangeably, they are separate disciplines that make use of common underlying concepts and technologies such as events, event brokers and event-driven architecture. Confluent’s acquisition of Immerok earlier this year provided a reminder of this fact. Confluent is one of the most...

Real-time business is a modern phenomenon, and business transformation has accelerated many business events in recent years. However, the execution of business events has always occurred in real time. Rather, it is the processing of the data related to business events that has accelerated instead of the event itself.

It is a mark of the rapid, current pace of development in artificial intelligence (AI) that machine learning (ML) models, until recently considered state of the art, are now routinely being referred to by developers and vendors as “traditional.” Generative AI, and large language models (LLMs) in particular, have taken the AI world by storm in the past year, automating and accelerating the development of content, including text, digital images, audio and video, as well as computer programs and...

As I have previously explained, we expect an increased demand for intelligent operational applications infused with the results of analytic processes, such as personalization and artificial intelligence-driven recommendations. These systems rely on the analysis of data in the operational data platform to accelerate worker decision-making or improve customer experience.

I have previously written about the functional evolution and emerging use cases for NoSQL databases, a category of non-relational databases that first emerged 15 or so years ago and are now well established as potential alternatives to relational databases. NoSQL is a term used to describe a variety of databases that fall into four primary functional categories: key-value stores, wide-column stores, document-oriented databases and graph databases. Each is worthy of further exploration, which is...

The publication of Ventana Research’s 2023 Operational Data Platforms Value Index earlier this year highlighted the importance of incorporating analytic processing into operational applications to deliver personalization and recommendations for workers, partners and customers. This importance is being accelerated by interest in generative AI, especially large language models. The emergence of intelligent applications has impacted the requirements for operational data platforms with the need to...

Despite best intentions, many organizations still struggle with some fundamental aspects of data processing and analytics. Taking data from operational applications and making it available for analysis is a first step, but data management remains a perennial challenge. Data movement and transformation difficulties can lead to delays and data quality problems that prevent organizations from generating value from data. The inability to govern and integrate data from multiple data sources prevents...

Maintaining data quality and trust is a perennial data management challenge, often preventing organizations from operating at the speed of business. Recent years have seen the emergence of data observability as a category of DataOps focused on monitoring the quality and reliability of data used for analytics and governance projects and associated data pipelines. There is clear overlap with data quality, which is more established as both a discipline and product category for improving trust in...

Organizations increasingly rely on real-time analytics to make informed decisions and stay competitive in today’s data-driven business landscape. As the complexity of data grows with the continuous addition of diverse sources, customers and workers alike expect real-time responsiveness. Accelerated query performance is crucial to process and extract valuable insights from data in a timely manner. Traditional analytics applications are often insufficient for managing the scale, velocity and...

I previously explained the arguments in favor of adoption of distributed SQL databases, the new generation of operational data platforms designed to combine the benefits of the relational database model and native support for distributed cloud architecture. It is critical for distributed SQL vendors to engage with developers to ensure they are considering the importance of resilience that spans multiple data centers and/or cloud regions as they choose the databases that will underpin...

Data fabric has grown in popularity as organizations struggle to manage data spread across multiple data centers, systems and applications. By providing a technology-driven approach to automating data management and governance across distributed environments, data fabric is attractive to organizations seeking to simplify and standardize data management. I assert that by 2025, more than 6 in 10 organizations will adopt data fabric technologies to facilitate the management and processing of data...

The data and analytics sector rightly places great importance on data quality: Almost two-thirds (64%) of participants in Ventana Research’s Analytics and Data Benchmark Research cite reviewing data for quality and consistency issues as the most time-consuming task in analyzing data. Data and analytics vendors would not recommend that customers use tools known to have data quality problems. It is somewhat surprising, therefore, that data and analytics vendors are rushing to encourage customers...

Master data management may not attract the same level of excitement as fashionable topics such as DataOps or Data Platforms, but it remains one of the most significant aspects of an organization’s strategic approach to data management. Having trust in data is critical to the ability of an organization to make data-driven business decisions. Along with data quality, MDM enables organizations to ensure data is accurate, complete and consistent to fulfill operational business objectives.

The data platforms market has traditionally been divided between products specifically designed to support operational or analytic workloads, with other market segments having emerged in recent years for data platforms targeted specifically at data science and machine learning (ML), as well as real-time analytics. More recently, we have seen vendor strategies evolving to provide a more consolidated approach, with data platforms designed to address a combination of analytics and data science, as...

The recent publication of our Value Index research highlights the impact of intelligent applications on the operational data platforms sector. While we continue to believe that, for most use cases, there is a clear, functional requirement for either analytic or operational data platforms, recent growth in the development of intelligent applications infused with the results of analytic processes, such as personalization and artificial intelligence (AI)-driven recommendations, has increasing...

As engagement with customers, suppliers and partners is increasingly conducted through digital channels, ensuring that infrastructure and applications are performing as expected is not just important but mission critical. My colleague, David Menninger, recently explained the increasing importance of observability to enable organizations to ensure that their systems and applications are operating efficiently. Observability has previously been the domain of the IT department but is increasingly...

I have written recently about the increasing importance of managing data in motion and at rest as the use of streaming data by enterprise organizations becomes more mainstream. While batch-based processing of application data has been a core component of enterprise IT architecture for decades, streaming data and event processing have often been niche disciplines typically reserved for organizations with the highest-level performance requirements. That has changed in recent years, driven by an...

To execute more data-driven business strategies, organizations need linked and comprehensive data that is available in real time. By consistently managing data across siloed systems and ensuring that data definitions are agreed and current, organizations can overcome the challenges presented by data being distributed across an increasingly disparate range of applications and data-processing locations. Maintaining data quality is a perennial data management challenge, often preventing...

Data Operations (DataOps) has been part of the lexicon of the data market for almost a decade, with the term used to describe products, practices and processes designed to support agile and continuous delivery of data analytics. DataOps takes inspiration from DevOps, which describes a set of tools, practices and philosophy used to support the continuous delivery of software applications in the face of constant changes. DataOps describes a set of tools, practices and philosophy used to ensure...

As data continues to grow and evolve, organizations seek better tools and technologies to employ data faster and more efficiently. Finding and managing data remains a perennial challenge for most organizations, and is exacerbated by increasing volumes of data and an expanding array of data formats. At the same time, organizations must comply with a growing list of national and regional rules and regulations, such as General Data Protection Regulation and the California Consumer Privacy Act....

Success with streaming data and events requires a more holistic approach to managing and governing data in motion and data at rest. The use of streaming data and event processing has been part of the data landscape for many decades. For much of that time, data streaming was a niche activity, however, with standalone data streaming and event-processing projects run in parallel with existing batch-processing initiatives, utilizing operational and analytic data platforms. I noted that there has...

I have previously written about the importance of data democratization as a key element of a data-driven agenda. Removing barriers that prevent or delay users from gaining access to data enables it to be treated as a product that is generated and consumed, either internally by employees or externally by partners and customers. This is particularly important for organizations adopting the data mesh approach to data ownership, access and governance. Data mesh is an organizational and cultural...

I recently wrote about the potential use cases for distributed SQL databases as well as techniques being employed by vendors to accelerate adoption. Distributed SQL is a term that is used by several vendors to describe operational data platform products that combine the benefits of the relational database model and native support for distributed cloud architecture, including resilience that spans multiple data centers and/or cloud regions. I noted that compatibility with existing database tools...

Organizations require faster analytics to continuously improve business operations and stay competitive in today’s market. However, many struggle with slow analytics due to a variety of factors such as slow databases, insufficient data storage capacity, poor data quality, lack of proper data cleansing and inadequate IT infrastructure. Challenges such as data silos can also decrease operational efficiency. And as the data grows, performing complex data modelling becomes challenging for users as...

I have previously written about the functional evolution and emerging use cases for NoSQL databases, a category of non-relational databases that first emerged 15 or so years ago and are now well established as potential alternatives to relational databases. NoSQL is a term that is used to describe a variety of databases that fall into four primary functional categories: key-value stores, wide column stores, document-oriented databases and graph databases. Each is worthy of further exploration,...

The market for data and analytics products is constantly evolving, with the emergence of new approaches to data persistence, data processing and analytics. This enables organizations to constantly adapt data analytics architecture in response to emerging functional capabilities and business requirements. It can, however, also be a challenge. Investments in data platforms cannot be constantly written-off as organizations adopt new products for new approaches. Too little change can lead to...

Data observability was a hot topic in 2022 and looks likely to be a continued area of focus for innovation in 2023 and beyond. As I have previously described, data observability software is designed to automate the monitoring of data platforms and data pipelines, as well as the detection and remediation of data quality and data reliability issues. There has been a Cambrian explosion of data observability software vendors in recent years, and while they have fundamental capabilities in common,...