Context architecture for AI agents: the five-layer guide

A financial services team watched an analytical AI agent hand a board member a confidently wrong revenue number. The model performed exactly as designed. No one had designed what the agent was allowed to see before it answered.

A context window is a container. Context architecture decides what data, definitions, and lineage fill that container at each stage, while governance decides which inputs to trust. Most enterprises run these as two separate workstreams. Five layers stand between an agent and the data it acts on, and each one forces a governance decision that most architecture diagrams skip entirely.

What is context architecture for AI agents?

Context architecture for AI agents is the design that decides which governed data artifacts, from business definitions to lineage to access policies, an analytical agent sees at each stage of a task. It determines what loads into the context window before a task starts, what gets pulled in mid-task, and what the agent carries forward into its next session.

Architecture decides where and when an artifact shows up. Governance decides whether it can be trusted once it's there. Most architecture diagrams stop at the technical question: where does data live, and how fast can an agent retrieve it?

A complete context architecture answers a second question at every layer: who approved this data, how current is it, and is the agent allowed to see it right now? Skip that second question, and an agent can be fast, well retrieved, and still wrong.

Five layers make up that architecture:

1. System context

2. Session context

3. Memory

4. Artifacts

5. On-demand or retrieved context

Architecture design and governance design are usually run as two separate projects. The five layers below are where the two finally have to meet.

OvalEdge expert insight: Architecture decides what an agent can see. Governance decides what it should trust. Most enterprises already have the governed artifacts, from catalog metadata to business definitions to lineage, spread across existing programs. The Enterprise Context Graph connects and activates that existing foundation so both answers stay identical across all layers, rather than drifting apart as an agent moves from one stage of a task to the next.

For a closer look at the governed layer that feeds all five, read what an AI context layer is.

The five layers of context architecture, and the governance decision each one forces

Each layer plays a different structural role inside an agent's context window, and each one forces a governance decision that most architecture diagrams skip entirely.

The pattern repeats across all five: name the layer, decide what belongs in it, then name who is accountable for keeping what's in it current and correct.

Start with the layer that loads first, before an agent sees a single query.

1. System context

System context covers the system-wide instructions and configuration an agent loads before any task starts. It includes operating rules, tool definitions, and the business logic an agent applies consistently across every session, regardless of who is asking or what they're asking about.

The design decision is whether approved business definitions live natively in system context or get retrieved on demand later in a task. If "revenue" resolves differently depending on which system answers, the architecture is already broken before the first token is generated. Definitions should be sourced from a single governed layer and loaded at the system level, so an agent starts every task working from the same meanings the business already uses.

In practice, this means loading approved definitions from one governed source rather than whichever document a model happened to see most recently. When a business glossary feeds system context directly, every agent session starts from the same meaning.

2. Session context

Session context is the running log of interactions and tool calls within a single work session. It accumulates as an agent moves through a task, tracking what it queried, what it retrieved, what a user clarified, and what a tool returned.

The design decision is which governed data assets an agent references during that session, and whether it re-resolves them against the same source of truth on every call rather than caching an early answer and reusing it as the session runs longer. Business definitions, certified sources, and access policies need to resolve the same way on the tenth call of a session as they did on the first.

At OvalEdge, we believe a session should become more consistent the longer it runs, never less. An agent that resolves the same metric two different ways inside one conversation has a session context design gap to close, regardless of which model is generating the answer.

3. Memory

Memory tiers carry knowledge across sessions, letting an agent remember prior decisions, user preferences, and past task outcomes instead of starting from zero every time.

The design decision is what an agent is allowed to remember, and what it has to re-check against the current governed source before acting on it again. A definition the governed layer has since deprecated, or a source flagged as an exception last quarter, should not persist in agent memory simply because the agent saw it once and never checked back.

OvalEdge expert insight: Memory that never gets re-validated turns into a liability instead of an asset. Treat agent memory as a cache with an expiration date tied to the governed source it came from, and revalidate it before reuse instead of letting an agent carry it forward unchallenged.

Closing that gap is exactly what an agent memory governance framework is built for.

4. Artifacts

Artifacts are the governed data artifacts an agent references at runtime: data models, certified datasets, glossary entries, and lineage graphs it pulls into a task as evidence.

The design decision is how an agent tells current from stale. It’s a provenance and certification problem, and it requires built-in certification and column-level lineage to resolve. If an agent can pull a data model but can't tell whether it's the certified version, the architecture has skipped the trust decision entirely, no matter how well the retrieval step performed.

Column-level lineage is what lets an agent, or the person reviewing its output, trace a number back to the exact table and transformation it came from.

5. On-demand or retrieved context

On-demand or retrieved context is data pulled at query time through RAG, direct database queries, or MCP-exposed tools. It's the layer most teams build first, because it's the one that makes an agent feel capable of answering anything.

The design decision is whether that retrieval call hits the governed layer or bypasses it entirely. Lineage and access policy matter most right here, because retrieval that quietly routes around governance is how document search often introduces wrong answers.

Example: A healthcare analytics agent pulls “readmitted patient” to answer a quality benchmarking query. CMS defines readmission as 30 days, internal quality uses 7 days, and finance uses 90 days. Without a governed retrieval layer resolving which definition applies, the agent picks whichever version its search returns first.

Context architecture vs RAG: what changes when the Enterprise Context Graph is native

RAG is a technique that lives inside one layer, on-demand, and retrieves context. It plays one role among five and doesn't stand in for system context, session context, memory, or artifacts. Treated as the whole architecture, RAG ends up doing double duty: resolving what's semantically similar to a query while also deciding what's true, current, and allowed, a governance decision rather than a retrieval outcome.

The pattern shows up in enterprise data too. A VB Pulse survey of 101 enterprises found document retrieval is the default way agents get business context for 38 percent of respondents, nearly double the next closest approach, and enterprises tend to pick that retrieval system for ease of ingestion rather than accuracy.

The retrieval system answers the question it was built to answer: which document looks most similar to the query. It was never built to answer whether that document holds the approved definition, whether the definition is still current, or whether the person asking is allowed to see it. Those are context architecture questions, and RAG alone has no layer designed to answer them.

A governed on-demand layer supplies the approved definition and access rule as inputs before a retrieval call even fires. The retrieval system does the resolving. The governed layer supplies what it resolves against.

At OvalEdge, we believe RAG answers what's semantically similar, while governance, through the Enterprise Context Graph, answers what's true, current, and allowed. An architecture needs both questions answered, and confusing one for the other is how confidently wrong answers ship to production.

The governed layer is what governance supplies. RAG resolves against it. Neither replaces the other; each handles a different job inside the same architecture. If context architecture is the blueprint, context engineering is the day-to-day practice of building and tuning what actually runs inside it.

Data quality signals determine whether what RAG retrieves is even worth resolving against in the first place.

Designing an enterprise context architecture: a decision framework

Most published architecture guides list governance as one optimization technique among several, somewhere after latency and cost. In practice, it's the layer that decides whether the other four are worth optimizing at all. An architecture team can tune retrieval speed, expand the context window, and add more tools to an agent's toolkit, and still ship wrong answers faster, because none of those changes touch what the agent is allowed to trust in the first place.

Enterprises using a governed platform to feed data artifacts into agent-facing systems have cut the effort required to catalog metadata, fulfill data requests, and compile lineage by up to 30 percent, according to Forrester's Total Economic Impact study of OvalEdge.

That's effort reclaimed from manual work and redirected toward the governance decisions this framework requires.

Five steps turn the framework into a working checklist

Each step closes a specific gap between architecture and governance, in the order most teams discover they need it: visibility first, then ownership, then enforcement.

Here's the checklist:

• Inventory which layer each governed asset feeds

• Decide what "current" means per layer, and name the owner

• Set a token budget for each layer instead of one global limit

• Expose the governed layer through MCP or API

• Log completions so a wrong answer traces back to its layer

OvalEdge expert insight: The architecture question is which layer a signal belongs in. The governance question is who’s accountable for that signal being right. When both questions resolve against the same governed source, teams stop debating answers after an agent has already shipped a wrong one.

Teams ready to expose this layer through APIs and agent tooling should start with building an enterprise MCP server.

Conclusion

A million-token context window doesn't fix a context architecture problem any more than a bigger filing cabinet fixes a filing system no one trusts. System context sets the shared meanings. Session context keeps them consistent as a task runs. Memory decides what an agent carries forward and what it has to re-check. Artifacts prove whether an agent's evidence is current.

On-demand retrieval decides whether a query reaches the governed layer or quietly routes around it. Each layer forces its own governance decision, and skipping any one is how a well-built agent still hands someone a confidently wrong number.

Enterprises have already built this governance foundation: the catalog, the glossary, the lineage, the access policies. The Enterprise Context Graph connects and activates that existing work so every layer of context architecture feeds governed inputs into an analytical AI agent, rather than leaving architecture and governance as two separate projects that only meet after something has gone wrong.

Book a demo to see how the Enterprise Context Graph connects an existing governance foundation to an analytical AI agent architecture, layer by layer.