Analyst Perspectives

As enterprises seek to expand and accelerate the adoption of artificial intelligence (AI) many are finding that longstanding analytics and data challenges are a barrier to success. As was explained in ISG’s State of Generative AI Market Report, AI requires data that is clean, well-organized and compliant with regulatory standards. The need for good data management is by no means new, but the expectations and demands associated with AI are a forcing function for enterprises to take long-overdue...

I have previously written about the functional evolution and emerging use cases for NoSQL databases, a category of non-relational databases that first emerged 15 or so years ago and are now well established as potential alternatives to relational databases. NoSQL is a term used to describe a variety of databases that fall into four primary functional categories: key-value stores, wide-column stores, document-oriented databases and graph databases. Each is worthy of further exploration, which is...

Discussion about potential deployment locations for analytics and data workloads is often based on the assumption that, for enterprise workloads, there is a binary choice between on-premises data centers and public cloud. However, the low-latency performance or sovereignty characteristics of a significant and growing proportion of workloads make them better suited to data and analytics processing where data is generated rather than a centralized on-premises or public cloud environment. ...

As articulated in Ventana Research’s Data Platforms Buyer’s Guide and DataOps Buyer’s Guide research, the combination of cloud computing and advanced analytics has lowered the cost of storing and processing large volumes of data, accelerating the emergence of new data platform and data operations products that enable organizations to gain operational efficiency and competitive advantage. The right combination of data platform and data management products is essential to ensure that the right...

The phrase ‘big data’ may have largely gone out of fashion, but the concept of storing and processing all relevant data continues to be important for enterprises seeking to be more data-driven. Doing so requires analytic data platforms capable of storing and processing data in multiple formats and data models. This will be an important focus for the forthcoming Data Platforms Buyer’s Guide 2024. 

I recently discussed how fashion has a surprisingly significant role to play in the data market as various architectural approaches to data storage and processing take turns enjoying a phase in the limelight. Pendulum swing is a theory of fashion that describes the periodic movement of trends between two extremes, such as short and long hemlines or skinny and baggy/flared trousers. Pendulum swing theory is similarly a factor in data technology trends, with an example being the oscillation...

I previously described how Oracle had positioned its database portfolio to address any and all data platform requirements. The caveat to that statement at the time was that any organization wanting to take advantage of the company’s flagship Oracle Autonomous Database could only do so using Oracle Cloud Infrastructure (OCI) cloud computing service, their own datacenter or a hybrid cloud environment. The widespread popularity of Oracle Database and the advanced automation capabilities delivered...

I recently articulated some of the reasons why IT teams can fail to deliver on the business requirements for data and analytics projects. This is an age-old and multifaceted problem that is not easily solved. Organizations have a role to play in alleviating the issue by ensuring that their business processes and project planning support a collaborative approach in which business and IT professionals work together. Data and analytics product vendors can also help by delivering products that are...

I previously wrote about how document-database providers have added support for ACID transactions and the SQL query language, making their products increasingly suitable for use as replacements for applications that previously depended on relational databases. Adoption of non-relational NoSQL databases is by no means reliant on displacing incumbent relational databases, and initial adoption is often driven by differentiating capabilities, such as developer agility and application flexibility....

I have previously written about the functional evolution and emerging use cases for NoSQL databases, a category of non-relational databases that first emerged 15 or so years ago and are now well established as potential alternatives to relational databases. NoSQL is a term used to describe a variety of databases that fall into four primary functional categories: key-value stores, wide-column stores, document-oriented databases and graph databases. Each is worthy of further exploration, which is...

I previously wrote about the challenge facing distributed SQL database providers to avoid becoming pigeonholed as only being suitable for a niche set of requirements. Factors including performance, reliability, security and scalability provide a focal point for new vendors to differentiate from established providers and get a foot in the door with customer accounts. Expanding and retaining those accounts is not necessarily easy, however, especially as general-purpose data platform providers...

I previously discussed the trust and accuracy limitations of large language models, suggesting that data and analytics vendors provide guidance about potentially inaccurate results and the risks of creating a misplaced level of trust. In the months that have followed, we are seeing some clarity from these vendors about the approaches organizations can take to increase trust and accuracy when developing applications that incorporate generative AI, including fine-tuning and prompt engineering. It...

I am happy to share insights gleaned from our latest Buyers Guide, an assessment of how well vendors’ offerings meet buyers’ requirements. TheVentana Research 2023 Data Observability Buyers Guide is the distillation of a year of market and product research by Ventana Research. Drawing on our Benchmark Research, we apply a structured methodology built on evaluation categories that reflect the real-world criteria incorporated in a request for proposal to data vendors supporting the spectrum of...

The 2023 Ventana Research Buyers Guide for Data Observability research enables me to provide observations about how the market has advanced. 

In my past perspectives, I’ve written about the evolution from data at rest to data in motion and the fact that you can’t rely on dashboards for real-time analytics. Organizations are becoming more and more event-driven and operating based on streaming data. As well, analytics are becoming more and more intertwined with operations. More than one-fifth of organizations (22%) describe their analytics workloads as real time in our Data and Analytics Benchmark Research and nearly half (47%) of...

I am happy to share insights gleaned from our latest Buyers Guide, an assessment of how well vendors’ offerings meet buyers’ requirements. The Ventana Research 2023 Data Orchestration Buyers Guide is the distillation of a year of market and product research by Ventana Research. Drawing on our Benchmark Research, we apply a structured methodology built on evaluation categories that reflect the real-world criteria incorporated in a request for proposal to data vendors supporting the spectrum of...

The 2023 Ventana Research Buyers Guide for Data Orchestration research enables me to provide observations about how the market has advanced. 

I am happy to share insights gleaned from our latest Buyers Guide, an assessment of how well vendors’ offerings meet buyers’ requirements. The Ventana Research 2023 Data Pipelines Buyers Guide is the distillation of a year of market and product research by Ventana Research. Drawing on our Benchmark Research, we apply a structured methodology built on evaluation categories that reflect the real-world criteria incorporated in a request for proposal to Data vendors supporting the spectrum of Data...

The 2023 Ventana Research Buyers Guide for Data Pipelines research enables me to provide observations about how the market has advanced. 

I am happy to share insights gleaned from our latest Buyers Guide, an assessment of how well vendors’ offerings meet buyers’ requirements. The Ventana Research 2023 DataOps Buyers Guide is the distillation of a year of market and product research by Ventana Research. Drawing on our Benchmark Research, we apply a structured methodology built on evaluation categories that reflect the real-world criteria incorporated in a request for proposal to Data vendors supporting the spectrum of DataOps....

I previously described how Databricks had positioned its Lakehouse Platform as the basis for data engineering, data science and data warehousing. The lakehouse design pattern provides a flexible environment for storing and processing data from multiple enterprise applications and workloads for multiple use cases. I assert that by 2025, 8 in 10 current data lake adopters will invest in data lakehouse architecture to improve the business value generated from the accumulated data.

The 2023 Ventana Research Buyers Guide for DataOps research enables me to provide observations about how the market has advanced. 

The data platforms market may appear to have little or nothing to do with haute couture, but it is one of the data sectors most strongly influenced by the fickle finger of fashion. In recent years, various architectural approaches to data storage and processing have enjoyed a phase in the limelight, including data warehouse, data mart, data hub, data lake, cloud data warehouse, object storage, data lakehouse, data fabric and data mesh. These approaches are often heralded as the next big thing,...

Despite a focus on being data-driven, many organizations find that data and analytics projects fail to deliver on expectations. These initiatives can underwhelm for many reasons, because success requires a delicate balance of people, processes, information and technology. Small deviations from perfection in any of those factors can send projects off the rails.

I have written before about the rising popularity of the data fabric approach for managing and governing data spread across distributed environments comprised of multiple data centers, systems and applications. I assert that by 2025, more than 6 in 10 organizations will adopt data fabric technologies to facilitate the management and processing of data across multiple data platforms and cloud environments. The data fabric approach is also proving attractive to vendors, including Microsoft, as a...

At one point, analytics and business intelligence were considered non-mission critical activities. One of the primary concerns in designing analytics systems was to ensure they didn’t interfere with or draw computing resources away from operational systems. But today, analytical systems are integral to many aspects of operations. More than 9 in 10 participants in our Analytics and Data Benchmark Research reported analytics had improved activities and processes. However, most analytics and BI...

Organizations today have an ever-increasing appetite for platforms that improve the speed and efficiency of data analytics and business intelligence (BI). The ability to quickly process data enables organizations to make well-informed decisions in real time. This agile approach to data processing is crucial for staying ahead in today's competitive landscape. With the rising need for data-driven insights, organizations face the difficulty of dealing with massive volumes of distributed business...

I recently wrote about the various technologies used by organizations to process and analyze data in real time. I explained that while the terms streaming data and events and streaming analytics are often used interchangeably, they are separate disciplines that make use of common underlying concepts and technologies such as events, event brokers and event-driven architecture. Confluent’s acquisition of Immerok earlier this year provided a reminder of this fact. Confluent is one of the most...

Real-time business is a modern phenomenon, and business transformation has accelerated many business events in recent years. However, the execution of business events has always occurred in real time. Rather, it is the processing of the data related to business events that has accelerated instead of the event itself.

It is a mark of the rapid, current pace of development in artificial intelligence (AI) that machine learning (ML) models, until recently considered state of the art, are now routinely being referred to by developers and vendors as “traditional.” Generative AI, and large language models (LLMs) in particular, have taken the AI world by storm in the past year, automating and accelerating the development of content, including text, digital images, audio and video, as well as computer programs and...

As I have previously explained, we expect an increased demand for intelligent operational applications infused with the results of analytic processes, such as personalization and artificial intelligence-driven recommendations. These systems rely on the analysis of data in the operational data platform to accelerate worker decision-making or improve customer experience.

I have previously written about the functional evolution and emerging use cases for NoSQL databases, a category of non-relational databases that first emerged 15 or so years ago and are now well established as potential alternatives to relational databases. NoSQL is a term used to describe a variety of databases that fall into four primary functional categories: key-value stores, wide-column stores, document-oriented databases and graph databases. Each is worthy of further exploration, which is...

The publication of Ventana Research’s 2023 Operational Data Platforms Value Index earlier this year highlighted the importance of incorporating analytic processing into operational applications to deliver personalization and recommendations for workers, partners and customers. This importance is being accelerated by interest in generative AI, especially large language models. The emergence of intelligent applications has impacted the requirements for operational data platforms with the need to...

Despite best intentions, many organizations still struggle with some fundamental aspects of data processing and analytics. Taking data from operational applications and making it available for analysis is a first step, but data management remains a perennial challenge. Data movement and transformation difficulties can lead to delays and data quality problems that prevent organizations from generating value from data. The inability to govern and integrate data from multiple data sources prevents...

Maintaining data quality and trust is a perennial data management challenge, often preventing organizations from operating at the speed of business. Recent years have seen the emergence of data observability as a category of DataOps focused on monitoring the quality and reliability of data used for analytics and governance projects and associated data pipelines. There is clear overlap with data quality, which is more established as both a discipline and product category for improving trust in...

Organizations increasingly rely on real-time analytics to make informed decisions and stay competitive in today’s data-driven business landscape. As the complexity of data grows with the continuous addition of diverse sources, customers and workers alike expect real-time responsiveness. Accelerated query performance is crucial to process and extract valuable insights from data in a timely manner. Traditional analytics applications are often insufficient for managing the scale, velocity and...

I previously explained the arguments in favor of adoption of distributed SQL databases, the new generation of operational data platforms designed to combine the benefits of the relational database model and native support for distributed cloud architecture. It is critical for distributed SQL vendors to engage with developers to ensure they are considering the importance of resilience that spans multiple data centers and/or cloud regions as they choose the databases that will underpin...

Data fabric has grown in popularity as organizations struggle to manage data spread across multiple data centers, systems and applications. By providing a technology-driven approach to automating data management and governance across distributed environments, data fabric is attractive to organizations seeking to simplify and standardize data management. I assert that by 2025, more than 6 in 10 organizations will adopt data fabric technologies to facilitate the management and processing of data...

The data and analytics sector rightly places great importance on data quality: Almost two-thirds (64%) of participants in Ventana Research’s Analytics and Data Benchmark Research cite reviewing data for quality and consistency issues as the most time-consuming task in analyzing data. Data and analytics vendors would not recommend that customers use tools known to have data quality problems. It is somewhat surprising, therefore, that data and analytics vendors are rushing to encourage customers...

Master data management may not attract the same level of excitement as fashionable topics such as DataOps or Data Platforms, but it remains one of the most significant aspects of an organization’s strategic approach to data management. Having trust in data is critical to the ability of an organization to make data-driven business decisions. Along with data quality, MDM enables organizations to ensure data is accurate, complete and consistent to fulfill operational business objectives.

The data platforms market has traditionally been divided between products specifically designed to support operational or analytic workloads, with other market segments having emerged in recent years for data platforms targeted specifically at data science and machine learning (ML), as well as real-time analytics. More recently, we have seen vendor strategies evolving to provide a more consolidated approach, with data platforms designed to address a combination of analytics and data science, as...

The recent publication of our Value Index research highlights the impact of intelligent applications on the operational data platforms sector. While we continue to believe that, for most use cases, there is a clear, functional requirement for either analytic or operational data platforms, recent growth in the development of intelligent applications infused with the results of analytic processes, such as personalization and artificial intelligence (AI)-driven recommendations, has increasing...

As engagement with customers, suppliers and partners is increasingly conducted through digital channels, ensuring that infrastructure and applications are performing as expected is not just important but mission critical. My colleague, David Menninger, recently explained the increasing importance of observability to enable organizations to ensure that their systems and applications are operating efficiently. Observability has previously been the domain of the IT department but is increasingly...

I have written recently about the increasing importance of managing data in motion and at rest as the use of streaming data by enterprise organizations becomes more mainstream. While batch-based processing of application data has been a core component of enterprise IT architecture for decades, streaming data and event processing have often been niche disciplines typically reserved for organizations with the highest-level performance requirements. That has changed in recent years, driven by an...

To execute more data-driven business strategies, organizations need linked and comprehensive data that is available in real time. By consistently managing data across siloed systems and ensuring that data definitions are agreed and current, organizations can overcome the challenges presented by data being distributed across an increasingly disparate range of applications and data-processing locations. Maintaining data quality is a perennial data management challenge, often preventing...

Data Operations (DataOps) has been part of the lexicon of the data market for almost a decade, with the term used to describe products, practices and processes designed to support agile and continuous delivery of data analytics. DataOps takes inspiration from DevOps, which describes a set of tools, practices and philosophy used to support the continuous delivery of software applications in the face of constant changes. DataOps describes a set of tools, practices and philosophy used to ensure...

As data continues to grow and evolve, organizations seek better tools and technologies to employ data faster and more efficiently. Finding and managing data remains a perennial challenge for most organizations, and is exacerbated by increasing volumes of data and an expanding array of data formats. At the same time, organizations must comply with a growing list of national and regional rules and regulations, such as General Data Protection Regulation and the California Consumer Privacy Act....

Success with streaming data and events requires a more holistic approach to managing and governing data in motion and data at rest. The use of streaming data and event processing has been part of the data landscape for many decades. For much of that time, data streaming was a niche activity, however, with standalone data streaming and event-processing projects run in parallel with existing batch-processing initiatives, utilizing operational and analytic data platforms. I noted that there has...

I have previously written about the importance of data democratization as a key element of a data-driven agenda. Removing barriers that prevent or delay users from gaining access to data enables it to be treated as a product that is generated and consumed, either internally by employees or externally by partners and customers. This is particularly important for organizations adopting the data mesh approach to data ownership, access and governance. Data mesh is an organizational and cultural...

Now more than ever, effective data management is crucial to enable decision-makers to better assess information and take calculated actions. It is also important to keep up with the latest trends and technologies to derive higher value from data and analytics and maintain a competitive edge in the market. However, every organization faces challenges with data management and analytics. And as organizations scale, the complexity only increases, creating a need for better data governance, data...

I recently wrote about the potential use cases for distributed SQL databases as well as techniques being employed by vendors to accelerate adoption. Distributed SQL is a term that is used by several vendors to describe operational data platform products that combine the benefits of the relational database model and native support for distributed cloud architecture, including resilience that spans multiple data centers and/or cloud regions. I noted that compatibility with existing database tools...

Organizations require faster analytics to continuously improve business operations and stay competitive in today’s market. However, many struggle with slow analytics due to a variety of factors such as slow databases, insufficient data storage capacity, poor data quality, lack of proper data cleansing and inadequate IT infrastructure. Challenges such as data silos can also decrease operational efficiency. And as the data grows, performing complex data modelling becomes challenging for users as...

I have previously written about the functional evolution and emerging use cases for NoSQL databases, a category of non-relational databases that first emerged 15 or so years ago and are now well established as potential alternatives to relational databases. NoSQL is a term that is used to describe a variety of databases that fall into four primary functional categories: key-value stores, wide column stores, document-oriented databases and graph databases. Each is worthy of further exploration,...

The market for data and analytics products is constantly evolving, with the emergence of new approaches to data persistence, data processing and analytics. This enables organizations to constantly adapt data analytics architecture in response to emerging functional capabilities and business requirements. It can, however, also be a challenge. Investments in data platforms cannot be constantly written-off as organizations adopt new products for new approaches. Too little change can lead to...

Data observability was a hot topic in 2022 and looks likely to be a continued area of focus for innovation in 2023 and beyond. As I have previously described, data observability software is designed to automate the monitoring of data platforms and data pipelines, as well as the detection and remediation of data quality and data reliability issues. There has been a Cambrian explosion of data observability software vendors in recent years, and while they have fundamental capabilities in common,...

I have written about the increased demand for data-intensive operational applications infused with the results of analytic processes, such as personalization and artificial intelligence-driven recommendations. I previously described the use of hybrid data processing to enable analytics on application data within operational data platforms. As is often the case in the data platforms sector, however, there is more than one way to peel an orange. Recent years have also seen the emergence of...

Organizations across various industries collect multiple types of data from disparate systems to answer key business questions and deliver personalized experiences for customers. The expanding volume of data increases complexity, and data management becomes a challenge if the process is manual and rules-based. There can be numerous siloed, incomplete and outdated data sources that result in inaccurate results. Organizations must also deal with concurrent errors – from customers to products to...

Despite the emphasis on organizations being more data-driven and making an increasing proportion of business decisions based on data and analytics, it remains the case that some of the most fundamental questions about an organization are difficult to answer using data and analytics. Ostensibly simple questions such as, “how many customers does the organization have?” can be fiendishly difficult to answer, especially for organizations with multiple business entities, regions, departments and...

I am happy to share insights from our latest Ventana Research Value Index research, which assesses how well vendors’ offerings meet buyers’ requirements. The 2023 Analytic Data Platforms Value Index is the distillation of a year of market and product research by Ventana Research. Drawing on our Benchmark Research, we apply a structured methodology built on evaluation categories that reflect real-world criteria incorporated in a request for proposal to data platform vendors supporting the...

Ventana Research recently published the 2023 Analytic Data Platforms Value Index. As organizations strive to be more data-driven, increasing reliance on data as a fundamental factor in business decision-making, the importance of the analytic data platform has never been greater. In this post, I’ll share some of my observations about how the analytic data platforms market is evolving.

I am happy to share insights from our latest Ventana Research Value Index research, which assesses how well vendors’ offerings meet buyers’ requirements. The 2023 Operational Data Platforms Value Index is the distillation of a year of market and product research by Ventana Research. Drawing on our Benchmark Research, we apply a structured methodology built on evaluation categories that reflect real-world criteria incorporated in a request for proposal to data platform vendors supporting the...

Ventana Research recently announced its 2023 Market Agenda for Data, continuing the guidance we have offered for two decades to help organizations derive optimal value and improve business outcomes.

Ventana Research recently announced its 2023 Market Agenda for Analytics, continuing the guidance we have offered for nearly two decades to help organizations derive optimal value from technology investments to improve business outcomes.

Ventana Research recently published the 2023 Operational Data Platforms Value Index. The importance of the operational data platform has never been greater as organizations strive to be more data-driven, incorporating intelligence into operational applications via personalization and recommendations for workers, partners and customers. In this post, I’ll share some of my observations on how the operational data platforms market is evolving.

Ventana Research recently announced its 2023 research agenda for the Office of Revenue, continuing the guidance we’ve offered for nearly two decades to help organizations realize their optimal value from applying technology to improve business outcomes. Chief Sales and Revenue Officers face an imperative to manage their sales and revenue organizations, but they don’t always have the guidance they need to embrace technology to achieve the best possible outcomes. As we look forward to 2023, we...

I am happy to share insights from our latest Ventana Research Value Index, which assesses how well vendors’ offerings meet buyers’ requirements. The 2023 Data Platforms Value Index is the distillation of a year of market and product research by Ventana Research. Drawing on our Benchmark Research, we apply a structured methodology built on evaluation categories that reflect real-world criteria incorporated in a request for proposal to data platform vendors that support the spectrum of...

I’m proud to share Ventana Research’s 2023 Market Agenda for Digital Technology. Our focus in this agenda is to deliver expertise to help organizations prioritize technology investments that improve customer, partner and workforce experiences while also increasing organizational effectiveness and agility.

Data observability is a hot topic and trend. I have written about the importance of data observability for ensuring healthy data pipelines, and have covered multiple vendors with data observability capabilities, offered both as standalone and part of a larger data engineering system. Data observability software provides an environment that takes advantage of machine learning and DataOps to automate the monitoring of data quality and reliability. The term has been adopted by multiple vendors...

Having recently completed the 2023 Data Platforms Value Index, I want to share some of my observations about how the market is evolving. Although this is our inaugural assessment of the market for data platforms, the sector is mature and products from many of the vendors we assess can be used to effectively support operational and analytic use cases.

Ventana Research has announced its market agenda for 2023, continuing a 20-year tradition of credibility and trust in our objective efforts to educate and guide the technology market. Our research and insights are backed by our expertise and independence, as we do not share our Market Agenda or our market research – including analyst and market perspectives – with any external party before it is published. We continuously refine our Market Agenda throughout the year to ensure we offer the...

In today’s organization, the myriad of analytics and permutations of dashboards challenge workers’ ability to take contextual actions efficiently. Unfortunately, conventional wisdom for investing in analytics does not recognize the benefits of empowering the workforce to understand the situation, examine options and work together to make the best possible decision.

The shift from on-premises server infrastructure to cloud-based and software-as-a-service (SaaS) models has had a profound impact on the data and analytics architecture of many organizations in recent years. More than one-half of participants (59%) in Ventana Research’s Analytics and Data Benchmark research are deploying data and analytics workloads in the cloud, and a further 30% plan to do so. Customer demand for cloud-based consumption models has also had a significant impact on the products...

Analytics processes are all about how organizations use data to create metrics that help manage and improve operations. Yet, the discipline applied to analytics processes seems to be lacking compared to data processes. I’ve pointed out that the weak link in data governance is often analytics. Organizations can also do a better job tying AnalyticOps to DataOps and do more to define and manage metrics. Our research has shown that creating and managing metrics in a semantic model improves...

There is always space for innovation in the data platforms sector, and new vendors continue to emerge at regular intervals with new approaches designed to serve specialist data storage and processing requirements. Factors including performance, reliability, security and scalability provide a focal point for new vendors to differentiate from established vendors, especially for the most demanding operational or analytic data platform requirements. It is never easy, however, for developers of new...

Ventana Research uses the term “data pantry” to describe a method of data storage (and the technology and process blueprint for its construction) created for a specific set of users and use cases in business-focused software. It’s a pantry because all the data one needs is readily available and easily accessible, with labels that are immediately recognized and understood by the users of the application. In tech speak, this means the semantic layer is optimized for the intended audience. It is...

The technology industry has established itself as a pivotal force in its ability to help organizations become more intelligent and automated. But doing so has required a journey of epic proportions for most organizations that have had to endure a transition of competencies and skills that was, in many places, transitioned to consulting firms who were hired appropriately to manage changes. Unfortunately, this step led, in many cases, to an extended focus on digital transformation rather than the...

Earlier this year, I wrote about the increasing importance of data observability, an emerging product category that takes advantage of machine learning (ML) and Data Operations (DataOps) to automate the monitoring of data used for analytics projects to ensure its quality and lineage. Monitoring the quality and lineage of data is nothing new. Manual tools exist to ensure that it is complete, valid and consistent, as well as relevant and free from duplication. Data observability vendors,...

One of the most significant considerations when choosing an analytic data platform is performance. As organizations compete to benefit most from being data-driven, the lower the time to insight the better. As data practitioners have learnt over time, however, lowering time to insight is about more than just high-performance queries. There are opportunities to improve time to insight throughout the analytics life cycle, which starts with data ingestion and integration, includes data preparation...

Organizations are increasingly utilizing cloud object storage as the foundation for analytic initiatives. There are multiple advantages to this approach, not least of which is enabling organizations to keep higher volumes of data relatively inexpensively, increasing the amount of data queried in analytics initiatives. I assert that by 2024, 6 in ten organizations will use cloud-based technology as the primary analytics data platform, making it easier to adopt and scale operations as necessary.

Almost all organizations are investing in data science, or planning to, as they seek to encourage experimentation and exploration to identify new business challenges and opportunities as part of the drive toward creating a more data-driven culture. My colleague, David Menninger, has written about how organizations using artificial intelligence and machine learning (AI/ML) report gaining competitive advantage, improving customer experiences, responding faster to opportunities and threats, and...

I have previously written about growing interest in the data lakehouse as one of the design patterns for delivering hydroanalytics analysis of data in a data lake. Many organizations have invested in data lakes as a relatively inexpensive way of storing large volumes of data from multiple enterprise applications and workloads, especially semi- and unstructured data that is unsuitable for storing and processing in a data warehouse. However, early data lake projects lacked structured data...

I have written recently about the similarities and differences between data mesh and data fabric. The two are potentially complementary. Data mesh is an organizational and cultural approach to data ownership, access and governance. Data fabric is a technical approach to automating data management and data governance in a distributed architecture. There are various definitions of data fabric, but key elements include a data catalog for metadata-driven data governance and self-service, agile data...

In their pursuit to be data-driven, organizations are collecting and managing more data than ever before as they attempt to gain competitive advantage and respond faster to worker and customer demands for more innovative, data-rich applications and personalized experiences. As data is increasingly spread across multiple data centers, clouds and regions, organizations need to manage data on multiple systems in different locations and bring it together for analysis. As the data volumes increase...

I have written a few times in recent months about vendors offering functionality that addresses data orchestration. This is a concept that has been growing in popularity in the past five years amid the rise of Data Operations (DataOps), which describes more agile approaches to data integration and data management. In a nutshell, data orchestration is the process of combining data from multiple operational data sources and preparing and transforming it for analysis. To those unfamiliar with the...

Ventana Research’s Data Lakes Dynamics Insights research illustrates that while data lakes are fulfilling their promise of enabling organizations to economically store and process large volumes of raw data, data lake environments continue to evolve. Data lakes were initially based primarily on Apache Hadoop deployed on-premises but are now increasingly based on cloud object storage. Adopters are also shifting from data lakes based on homegrown scripts and code to open standards and open...

I have written before about the continued use of specialist operational and analytic data platforms. Most database products can be used for operational or analytic workloads, and the number of use cases for hybrid data processing is growing. However, a general-purpose database is unlikely to meet the most demanding operational or analytic data platform requirements. Factors including performance, reliability, security and scalability necessitate the use of specialist data platforms. I assert...

Earlier this year I described the growing use-cases for hybrid data processing. Although it is anticipated that the majority of database workloads will continue to be served by specialist data platforms targeting operational and analytic workloads respectively, there is increased demand for intelligent operational applications infused with the results of analytic processes, such as personalization and artificial intelligence-driven recommendations. There are multiple data platform approaches to...

I have recently written about the organizational and cultural aspects of being data-driven, and the potential advantages data-driven organizations stand to gain by responding faster to worker and customer demands for more innovative, data-rich applications and personalized experiences. I have also explained that data-driven processes require more agile, continuous data processing, with an increased focus on extract, load and transform processes — as well as change data capture and automation...

The data catalog has become an integral component of organizational data strategies over the past decade, serving as a conduit for good data governance and facilitating self-service analytics initiatives. The data catalog has become so important, in fact, that it is easy to forget that just 10 years ago it did not exist in terms of a standalone product category. Metadata-based data management functionality has had a role to play within products for data governance and business intelligence for...

Zoho presented analysts with a deep look at its strategy and roadmap at its July analyst conference, describing how it intends to meld its many business applications together through integration at the level of the platform. The company, which is privately owned and funded, has generally sought to build its own tools rather than buy or partner. This approach has allowed the firm to create a suite of tightly linked tools that share a common interface.

I have written recently about increased demand for data-intensive applications infused with the results of analytic processes, such as personalization and artificial intelligence (AI)-driven recommendations. Almost one-quarter of respondents (22%) to Ventana Research’s Analytics and Data Benchmark Research are currently analyzing data in real time, with an additional 10% analyzing data every hour. There are multiple data platform approaches to delivering real-time data processing and analytics...

I recently noted that as demand for real-time interactive applications becomes more pervasive, the use of streaming data is becoming more mainstream. Streaming data and event processing has been part of the data landscape for many decades, but for much of that time, data streaming was a niche activity. Although adopted in industry segments with high-performance, real-time data processing and analytics requirements such as financial services and telecommunications, data streaming was far less...

I have recently written about the importance of healthy data pipelines to ensure data is integrated and processed in the sequence required to generate business intelligence, and the need for data pipelines to be agile in the context of real-time data processing requirements. Data engineers, who are responsible for monitoring, managing and maintaining data pipelines, are under increasing pressure to deliver high-performance and flexible data integration and processing pipelines that are capable...

I recently explained how emerging application requirements were expanding the range of use cases for NoSQL databases, increasing adoption based on the availability of enhanced functionality. These intelligent applications require a close relationship between operational data platforms and the output of data science and machine learning projects. This ensures that machine learning and predictive analytics initiatives are not only developed and trained based on the relationships inherent in...

Streaming data has been part of the industry landscape for decades but has largely been focused on niche applications in segments with the highest real-time data processing and analytics performance requirements, such as financial services and telecommunications. As demand for real-time interactive applications becomes more pervasive, streaming data is becoming a more mainstream pursuit, aided by the proliferation of open-source streaming data and event technologies, which have lowered the cost...

When joining Ventana Research, I noted that the need to be more data-driven has become a mantra among large and small organizations alike. Data-driven organizations stand to gain competitive advantage, responding faster to worker and customer demands for more innovative, data-rich applications and personalized experiences. Being data-driven is clearly something to aspire to. However, it is also a somewhat vague concept without clear definition. We know data-driven organizations when we see them...

I recently wrote about the growing range of use cases for which NoSQL databases can be considered, given increased breadth and depth of functionality available from providers of the various non-relational data platforms. As I noted, one category of NoSQL databases — graph databases — are inherently suitable for use cases that rely on relationships, such as social media, fraud detection and recommendation engines, since the graph data model represents the entities and values and also the...

I previously described the concept of hydroanalytic data platforms, which combine the structured data processing and analytics acceleration capabilities associated with data warehousing with the low-cost and multi-structured data storage advantages of the data lake. One of the key enablers of this approach is interactive SQL query engine functionality, which facilitates the use of existing business intelligence (BI) and data science tools to analyze data in data lakes. Interactive SQL query...

I previously explained how the data lakehouse is one of two primary approaches being adopted to deliver what I have called a hydroanalytic data platform. Hydroanalytics involves the combination of data warehouse and data lake functionality to enable and accelerate analysis of data in cloud storage services. The term data lakehouse has been rapidly adopted by several vendors in recent years to describe an environment in which data warehousing functionality is integrated into the data lake...

As I recently described, it is anticipated that the majority of database workloads will continue to be served by specialist data platforms targeting operational and analytic workloads, albeit with growing demand for hybrid data processing use-cases and functionality. Specialist operational and analytic data platforms have historically been the since preferred option, but there have always been general-purpose databases that could be used for both analytic and operational workloads, with tuning...