I have written before about how enterprises are realizing the importance of data management if they are to succeed in implementing artificial intelligence and enabling agentic decision-making. Data quality and veracity are important aspects of increasing trust in the data used to generate content and automate decisions. More fundamentally, however, a successful data strategy depends on an enterprise knowing what data it has at its disposal, understanding when and how it is used and managing it accordingly. Given the significant volumes of data under management, a strategy that treats all data as equally important is doomed to fail. The concept of critical data elements is designed to enable enterprises to manage data in accordance with its perceived value. Without it, an enterprise will likely overspend unnecessarily on the storage, processing and governance of low-value data, while running the risk of failing to store, process and correctly govern the highest-value data.
The volume of data generated by enterprises continues to accelerate rapidly. A strategy that treats all data generated by the enterprise as equally important is impractical. It is also fundamentally risky. Data that is subject to regulatory constraints (such as personally identifiable information) needs to be managed differently from data that is not. Specifically, this data is likely to be subject to regulatory controls and sovereign data policies that dictate where and how it can be stored and processed. The challenge many enterprises face, especially as they look to expand their use of data to fuel AI initiatives, is that while they understand that a proportion of their data is subject to sovereign data policies, they may not have a clear idea of what proportion of their data is subject to those policies or in which environments that data resides.
At a more micro-level, having a good understanding of which business intelligence reports and dashboards are reliant on which data sources is important for assessing the potential
The concept of critical data elements is not new. Data stewards and master data management practitioners have long understood that some data is more important than others and should be managed accordingly. As the name suggests, CDEs represent an enterprise’s most important data assets: those that are considered essential for the organization to operate effectively. Example CDEs that are common to many enterprises include customer data, financial data, product data and operational metrics. Even within those categories, not all data will be considered critical, however. CDEs represent the most business-critical, high-impact and sensitive data for each enterprise. As such, the definition of CDEs is extremely subjective and the precise set of data assets that are considered critical will be unique to each enterprise, depending on its industry, compliance requirements and strategic goals.
While the concept of CDEs is not new, the criticality of CDEs has increased due to the emergence of agentic decision-making. I recently discussed the importance of business semantics, ontologies, business logic, regulatory requirements, policies, metrics and key performance indicators in relation to providing agents with a contextual understanding of business concepts and processes. Identifying the most important data assets, metrics and policies as CDEs provides important context for agents by facilitating prioritization, which is crucial in automated decision-making. AI also has a role in the evolution of approaches to defining and managing CDEs. As noted, each enterprise’s CDEs will be somewhat unique and defining and managing CDEs has traditionally required significant manual effort and expertise. We are beginning to see a growing number of data governance providers—including Alation, Ataccama and Microsoft—delivering new AI capabilities for automating the identification of critical data and recommending policies and processes for how it should be processed and managed.
Automated CDE management functionality can be used to identify the relative criticality and risk associated with individual data assets based on usage and associated policies and regulations, for example. Additionally, while traditional manual approaches to CDE management relied on assessing and tagging individual data elements, the automated management of CDEs enables criticality to be applied to higher-level business processes and inherited by associated data elements, terms and attributes. Automated functionality to identify and manage CDEs is still in its infancy but is likely to become a key requirement for enterprises as they assess the suitability of their data management software to support agentic AI. I recommend that enterprises evaluating potential data management software providers add functionality for CDE management to the list of capabilities being assessed.
Regards,
Matt Aslett