Everything is Context: The Many Layers of Context for Enterprise AI

I have previously described how context is everything when it comes to establishing enterprise trust in the output of generative and agentic artificial intelligence. Based on our recent interactions with analytics and data software providers, it might be more accurate to say that context is everywhere, as almost all providers of analytics and data software are currently positioning products in relation to context for AI. This is due in part to the tendency of provider messaging to gravitate to the latest shiny object, marketing products in relation to the concepts identified as driving enterprise budgets. However, it is also due in part to every provider being able to make a valid case that its product plays a role in relation to providing context from enterprise data and information. When it comes to establishing the trust required for enterprise AI, context is everything, and everything is context.

As soon as generative AI models and services entered mainstream consciousness, it became clear that ensuring the factual accuracy of the generated content would be critical to enterprise adoption. ISG’s State of Enterprise AI Adoption report identified numerous potential use cases for GenAI, including classification, recognition, optimization, generation and summarization. Also listed are multiple potential challenges to taking initiatives from experimentation into production. Accuracy and trust are important to all enterprise AI initiatives, but particularly essential to agentic AI initiatives involving automated decision-making based on multistep reasoning. GenAI and agentic AI are probabilistic. Enterprise decision-making requires determinism. While there are tasks that play to the strengths of GenAI (such as processing and interpreting code and unstructured information) and agentic AI (such as reasoning and recommendation), any enterprise AI use case needs to be at least partially deterministic to ensure that the generated output is grounded in real data and trusted information. This grounding is now collectively referred to as context and involves multiple layers of tools, data and information as well as the ability to route tasks between probabilistic or deterministic systems.

Early initiatives to improve trust in the output of GenAI involved context engineering, a combination of prompt engineering and retrieval-augmented generation used to limit the information the model has access to and control the format required for responses. Context engineering is an important component of the wider discipline of providing agents with guardrails at the runtime layer to ensure they operate within the parameters required to accomplish specific tasks. This is delivered through the development of agent harnesses (also known as harness engineering).

While the model or models at the heart of an agent remain responsible for generating the responses required for a given task, it is the agent harness that dictates how the task is accomplished. Agent harnesses are particularly important for maintaining reliability in multi-step agent pipelines where small failure rates for each task could otherwise add up to a high likelihood of failure in a chain of multi-step tasks. In addition to context engineering, agent harnesses also address identity and authorization, the monitoring of the state, lifecycle and health of agents, memory management, task verification and human-in-the-loop approvals, as well as tool connectivity and orchestration—typically utilizing Model Context Protocol—to ensure that models have access to appropriate tools and information. I assert that through 2028, almost all data-related software providers will adopt Model Context Protocol to facilitate interoperability between agentic applications and trusted enterprise data and business workflows.

Harness engineering and context engineering are used to control the information that agents have access to but do not address the quality and veracity of that information. This is context that needs to be provided by the underlying data layer. I have written before about the unreasonable effectiveness of data management for AI, explaining that while data is integral to AI, poor data management can be an impediment to success. The data layer is essential in supporting accuracy and trust by ensuring that the data used to ground AI is clean, well-organized, compliant with regulatory standards and fit for purpose.

The data layer has multiple roles, the first of which is ensuring that data is reliably and securely persisted and processed. In theory, the data would ideally be persisted and processed in a single environment, but it is likely to be distributed across multiple data platforms. As such, the data layer must also support federation, including data integration and transformation, as well as zero-copy access and processing. The data must also be governed and managed to ensure that it is fit for purpose. This includes support for data lineage capabilities to establish provenance and integrity, as well as data quality and observability functionality to establish veracity and freshness and data security and access controls that reflect regulatory and sovereignty requirements. AI and data governance are symbiotic. Data governance processes and products can help improve AI, but AI also has a role in data governance by automating and accelerating previously manual processes.

Agents also need to understand business concepts and processes. This is provided by a metadata-driven knowledge layer that addresses business semantics (agreed definitions and meanings), ontologies (organizational structures and relationships), business logic, regulatory requirements and policies and metrics and key performance indicators. I previously explained the importance of semantic data modeling and semantic layer software in providing business context for AI, as well as the knowledge graph as a structured representation of information that identifies entities, their attributes and the relationships between them. Making this information available across the enterprise is essential to human and agentic decision-making.

More than one-half (52%) of participants in ISG’s Data and AI Market Lens stated that a lack of access to semantic logic is a common disruption to data initiatives, making it hard to understand what the data means. A clear understanding of business goals and KPIs is also essential to success with AI, with 25% of participants in ISG’s AI Market Lens Study naming the need to define clear goals and KPIs as a lesson learned from their AI initiatives, followed by data quality and governance (23%) and a focus on business outcomes (16%).

The need to capture the actions and outcomes from human and agentic decision-making is an important component of ensuring that agents not only perform tasks but also learn from completed tasks to improve over time. In addition to memory at the runtime layer, which retains user preferences, conversation history and personalized experiences, the memory layer provides critical context for reasoning and planning, decision intelligence and actions and outcomes. If agents are to learn from previous decisions—both human and agentic—it is critical that they understand not just what decisions were taken, but why they were taken, and with what outcome.

Reasoning models are increasingly being adopted for enterprise requirements to ensure that AI use cases incorporate planning, use tools effectively and avoid ambiguity, thereby enhancing reliability. Integration with enterprise planning is also essential to ensure that automated decisions are taken in the context of planning cycles and strategies, as decision intelligence provides information on whether previous decisions were successfully aligned with desired goals and KPIs and whether proposed decisions are predicted to fulfil objectives.

While almost all providers are positioning products in relation to context for AI, I see the greatest activity today at the runtime layer with agent harnesses, the data layer with governance, quality and security, and the knowledge layer with semantic modeling and knowledge graphs. While these are all critical elements for improving trust in enterprise AI, enterprises and providers alike should not overlook the importance of the memory layer for capturing the impact of decisions and actions as they are made, measuring their effectiveness and applying lessons learned in future decision-making. I recommend that software providers and enterprises invest in memory and self-learning processes alongside enhancements at the runtime, data and knowledge layers.

Regards,

Matt Aslett