Every AI agent you run keeps what it learns to itself. Work through a problem with Claude Code in the morning, then ask Codex about it that afternoon, and the second agent has no idea the first one exists. Add a home-server agent like OpenClaw or Hermes into the mix, and you end up reexplaining the same context to a different tool every time you switch.

When AI engineer Sajal Sharma ran into this problem in his own work, he solved it by building a personal knowledge base to act as a shared brain for every agent he runs. On this week’s episode of Zero to Agent in 30 Minutes, Sajal showed how to set up that shared workspace yourself so that a task added on one tool shows up for all the others.

How to set up a knowledge base for your agents

Here’s how Sajal’s setup breaks down:

1. Create a workspace map. Set up an AGENTS.md file that lists where everything in your knowledge base lives, from current tasks to project notes to decision logs. This will help each of your agents navigate your workspace without guessing.
2. Layer daily notes into summaries. Keep the most detailed notes at the daily level, then roll several days into a weekly summary and several weeks into a monthly one. An agent can then work from the summarized view instead of reading through months of individual files, which keeps token use manageable as the knowledge base grows.
3. Bridge AGENTS.md with CLAUDE.md. Claude Code reads CLAUDE.md, not AGENTS.md, so add a short pointer in CLAUDE.md that redirects to the AGENTS.md or link the two files directly. Sajal uses this pattern to avoid maintaining two files separately and having them drift out of sync.
4. Package repeatable tasks as skills. Turn routines you do often, like producing a daily briefing or turning a saved article into a note, into skill files stored in the shared workspace. Any agent that can read the workspace can then run the task the same way, rather than working out the steps on its own each time.
5. Sync the workspace across machines. Use a file-sync tool, Git, or a shared server to keep your local copy of the knowledge base and your server copy aligned. That way, you ensure that an agent running on a laptop and one running on a home server, through a gateway like OpenClaw, are working from the same files.
6. Have agents reread the state before every write. Add an instruction in AGENTS.md telling every agent to check the current version of the knowledge base before making a change. When you have several agents writing to the same files, this step keeps one agent from acting on information another has already updated.

Sajal closed by pointing to two projects as evidence that this “shared brain” pattern is spreading beyond his own setup. LangChain recently released OpenWiki, a tool that generates and maintains repository documentation that both people and coding agents can use. And Y Combinator president Garry Tan built and open-sourced GBrain, a memory layer for agents built on the same principle.

Sajal’s starter repo is available on GitHub if you want to set up your own version, and you can reach out to him on LinkedIn to discuss the topic further.

Coming up next

On September 16, data science educator and AI consultant Chester Ismay joins Zero to Agent in 30 Minutes to build a personal sports concierge agent that will read the schedules for every sport he follows, decide what’s worth his time, and send a single weekly update to his phone. Viewers can take the pattern home to plan their own week.

Follow along with Zero to Agent in 30 Minutes on Radar, or watch the latest episode on YouTube, Spotify, Apple, or wherever you get your podcasts. If you’re an O’Reilly member, you can watch live. Save your seat.

In 17 years of building enterprise data platforms, I’ve watched every organization eventually ask the same question: “Can I ask one question and get one answer across everything my company knows?” A finance analyst wants actual revenue from the warehouse, pipeline data from the CRM, commentary from planning documents, and market signals from external providers. The information already exists, but it lives across systems that were never designed to reason together.

For decades we tried to solve this by consolidating data. We built larger warehouses, semantic layers, APIs, and dashboards. Each solved part of the problem, but none solved the fundamental one: orchestrating reasoning across heterogeneous sources in response to an arbitrary business question. Earlier systems supported limited federation and semantic querying, yet they struggled to reason across those sources at enterprise scale without significant custom engineering.

Modern LLMs change this. Instead of replacing databases, they facilitate a new architectural primitive: an intelligent orchestration layer that dynamically reasons across specialized systems. Rather than consolidating the data into a single store, this layer consolidates the access pattern to data that stays where it lives.

This article presents a reference architecture for LLM-powered enterprise analytics agents that coordinate purpose-built, heterogeneous data stores through intelligent orchestration while preserving security, performance, and auditability.

What specifically changed with GenAI

BI tools have always been constrained to predefined reports and dashboards. Before GenAI, building a cross-system query engine meant hardcoding every possible query pattern, data source combination, and synthesis path. And because the number of possible questions grows exponentially with the number of data sources, exhaustive coverage is impossible through traditional engineering. GenAI changes this in three specific ways.

Intent understanding replaces query templates: An LLM parses natural language and determines which data sources are relevant based on semantic understanding rather than keyword matching. Unlike a keyword search, an LLM understands that “Why did retention drop in Asia last quarter?” and “What is driving churn in Asian markets?” are the same question expressed differently. More importantly, it infers that answering the question requires customer relationship data, revenue metrics, and possibly support ticket sentiment, even though none of those systems are named.

Dynamic query decomposition replaces static pipelines: A question like “What are the biggest risk factors in our supply chain?” might require relationship data from a graph database, metrics from a key-value store, contract details from a document repository, and market intelligence from an API. The agent decomposes it into specialized subqueries on the fly, each optimized for the target store’s access pattern. There’s no prebuilt pipeline and no engineering ticket to wire up a new combination, because the decomposition happens at inference time. The system handles novel questions without code changes.

Semantic synthesis replaces manual consolidation: Before GenAI, making sense of the data together was the real work. An analyst would pull numbers from the warehouse, check relationships in a CRM, read through documents, and mentally synthesize an answer. That took hours or days and was bounded by one person’s ability to hold context. I’ve watched senior analysts spend entire Mondays answering a single leadership question. An LLM reasons about how metrics relate to the relationship patterns in a knowledge graph and the strategic context in unstructured documents, and it does so in seconds with full source attribution. A dashboard shows numbers; an analytics agent explains what those numbers mean in the context of everything else it knows.

The architecture: Consolidate the access pattern, not the data

Rather than consolidating the data into a single store, consolidate the access pattern through an intelligent orchestration layer. If your instinct is to get everything into one place, you aren’t alone, but every time we did that we lost something. Graph relationships flattened into join tables, hierarchical documents shredded into rows, and real-time signals turned stale in batch loads. The warehouse was always a compromise.

The better approach is to keep each data store optimized for its specific query pattern:

• Graph database for relationship traversal and multihop reasoning
• Key-value store for instant metric lookups with sub-millisecond latency
• Vector store for semantic document search and similarity matching
• External APIs for market intelligence and real-time signals
• Data Warehouse for large-scale historical aggregation and ad hoc SQL

The LLM-powered agent coordinates across all of them through a unified orchestration layer. This follows the same principle that makes microservices work: specialized services with well-defined interfaces, coordinated by an orchestrator. The difference is that the orchestrator now understands natural language, reasons about which services to call based on intent rather than explicit routing rules, and synthesizes results semantically rather than through programmatic joins. Think of it as a data mesh for inference, where each node keeps its operational independence while an intelligent layer federates queries across them.

The orchestration protocol

The agent follows a multiphase protocol for every query. The full reasoning loop with security enforcement and parallel execution goes well beyond a simple RAG pattern.

Let’s trace a business question through each phase:

“Why did Q2 revenue fall short of forecast in the enterprise segment?”

This question requires revenue metrics (metrics store), account relationships and sales coverage (graph), deal commentary and executive notes (vector store), and market benchmarks (external APIs). No single system holds the answer.

Phase 1: Intent analysis. The LLM determines what the user is asking and which data sources are relevant.

“Why did Q2 revenue fall short of forecast in the enterprise segment?”
                                                   ↓
Intent: Revenue variance root cause analysis
Entities: Enterprise segment
Timeframe: Q2
Metric: Revenue vs. forecast
Required stores: Metrics + Graph + Vector + External API

Not every query needs every store. “What is our current ARR?” might need to hit the metrics store only. This revenue variance question requires all four.

Phase 2: Query decomposition. The original question is broken into specialized subqueries optimized for each target store:

• Metrics store: “Q2 revenue actuals vs. forecast for enterprise, by region and product line”
• Graph store: “Enterprise accounts with closed-lost or slipped deals in Q2; common patterns in sales coverage, partner relationships, deal stage progression”
• Vector store: “Deal notes, QBR summaries, and executive correspondence referencing enterprise deal delays or losses in Q2”
• External API: “Industry benchmark data for enterprise software spending in Q2”

Each is tailored to the target system’s access pattern, not forced through a common query language.

Phase 3: Parallel execution. Tools execute concurrently. This is critical for latency. Sequential execution across four stores would blow past any reasonable response time. With parallel execution, the total data retrieval time equals the slowest individual store, not the sum of all stores.

Phase 4: Deterministic security enforcement. A security layer enforces user permissions at the data query level before any retrieval happens. This is structural rather than application-level: It routes queries to the appropriate data partitions, applies row-level filters at the database query level, and gates the input rather than filtering the output. If our user is a regional VP without access to EMEA deal data, the graph returns no EMEA accounts, the metrics lookup excludes EMEA revenue, and the vector search filters out EMEA deal notes. The LLM can’t leak what it never received. (More on why this must be deterministic below.)

Phase 5: Result synthesis. The LLM reasons about how results from different stores relate, identifies patterns across sources, resolves contradictions, and generates a unified answer with source attribution.

For our revenue question, synthesis might produce:

Q2 enterprise revenue missed forecast by $4.2M (8% variance). Three root causes account for 85% of the gap: (1) Four deals totaling $2.8M slipped to Q3 due to extended legal review cycles—all four involved customers who adopted a new procurement platform in Q1 (graph: common vendor relationship). (2) Two large renewals downsized by $1.1M; deal notes cite budget reallocation toward AI initiatives, a pattern appearing across 6 QBR summaries (vector store). (3) Enterprise software spending contracted 3% QoQ industry-wide (external benchmark), suggesting partial macro headwind. Additionally, all four slipped deals lacked executive sponsor engagement in the final 30 days (graph: engagement signal absent).

No single store contains that answer. On a dashboard, the miss shows up as a red number. The orchestrated synthesis explains why it happened, surfaces the structural patterns behind it, and points to what needs to change.

The core orchestration pattern

The orchestration loop itself is straightforward. Here’s the core pattern:

from concurrent.futures import ThreadPoolExecutor, as_completed


def run_agent(question, tools, execute_fn, model="gpt-4o"):
    # Phase 1-2: LLM analyzes intent and decides which tools to call
    response = client.chat.completions.create(
        model=model, messages=[{"role": "user", "content": question}],
        tools=tools, tool_choice="auto"
    )
    tool_calls = response.choices[0].message.tool_calls


    # Phase 3: Execute tool calls in parallel
    with ThreadPoolExecutor(max_workers=len(tool_calls)) as executor:
        futures = {
            executor.submit(execute_fn, tc.function.name,
                            json.loads(tc.function.arguments)): tc
            for tc in tool_calls
        }
        results = {futures[f].id: f.result() for f in as_completed(futures)}


    # Phase 5: Synthesize results into unified answer
    messages = [response.choices[0].message]
    for tc_id, result in results.items():
        messages.append({"role": "tool", "tool_call_id": tc_id,
                         "content": json.dumps(result)})
    return client.chat.completions.create(model=model, messages=messages)

The tool definitions tell the LLM what each store is optimized for. The LLM decides which to invoke based on the question’s intent. With parallel execution, data retrieval completes in milliseconds even when hitting multiple stores simultaneously, making LLM inference the dominant latency factor, not the data layer.

Why the knowledge graph is the highest-leverage component

Knowledge graphs have existed for decades and have always been powerful. They’ve also stayed on the exotic end of the enterprise stack, and the reason is human rather than technical. The last-mile problem was translating between natural language and graph traversals. A graph database can answer extraordinarily complex relationship questions, such as “Which accounts have overlapping stakeholders with our churned customers from last quarter who also evaluated competitor products?” but asking that question required an engineer fluent in both the graph schema and the business domain. That combination of skills is rare and expensive, which is exactly why graph databases have never quite gone mainstream.

GenAI removes this bottleneck, and it does so precisely where the barrier was highest: the translation step that used to require a specialist. With an LLM as the translation layer, the graph becomes accessible to anyone who can type a question in plain language. The LLM generates graph queries, traverses multihop relationship paths, and explains results in business context. In our revenue variance example, the graph reveals that all four slipped deals share a common pattern of customers who adopted a new procurement platform in Q1 and lacked executive sponsor engagement in the final 30 days. That pattern is invisible in revenue metrics alone, because it requires relationship traversal across account nodes, vendor relationships, and engagement signals.

Designing the graph schema

The critical design decision is aligning the graph schema with your business ontology. The temptation is to model the graph around your data model (tables, columns, foreign keys). The correct approach is to model it around how your organization actually thinks about its domain:

• Entity nodes with lifecycle stages, engagement indicators, and health scores
• Product nodes connected to use cases, adoption patterns, and competitive alternatives
• Relationship edges that capture not just connections but context: strength, recency, business driver, and directionality

When the data model matches the business mental model, the agent’s responses feel natural rather than technically correct but practically useless.

GraphRAG: Deterministic reasoning chains

A powerful extension is GraphRAG (graph retrieval-augmented generation), where the agent constructs deterministic inference paths by traversing the graph rather than relying on the LLM’s parametric knowledge. The LLM isn’t remembering something from training; it’s following an explicit path through verified data. The result is auditable reasoning chains: “Account A connects to Partner B through implementation relationship X, and Partner B appears in three other churned accounts, suggesting a systemic delivery issue.” Each step is verifiable against source data, which is critical in enterprise environments where decisions need justification beyond model confidence scores.

GraphRAG also reduces hallucination risk. When the LLM follows graph edges rather than generating from parametric memory, the actual data constrains the space of possible outputs. The graph acts as a factual guardrail on the reasoning process.

Production considerations

Moving from prototype to production exposes a set of challenges that don’t appear in demos.

Model flexibility and cost management

Decouple the orchestration layer from any specific LLM provider. I can’t stress this enough. If it’s tightly coupled to one provider’s API, you’ll end up rewriting it within a few months, when pricing changes or a better model drops. Implement model fallback for throttling resilience. The landscape moves fast, and you don’t want architectural lock-in baked into your data infrastructure.

Multihop model selection is also worth considering. Use a smaller, faster model for intent classification and query decomposition, where the task is well-defined, and a larger model for synthesis, where reasoning quality matters. Intent classification with a small model takes around 200 ms and costs a fraction of a full reasoning pass, so reserve the expensive inference for synthesis, where quality directly impacts user experience.

Security at the data layer

Row-level security must be enforced deterministically before data reaches the agent, as described in Phase 4 above. This is a nonnegotiable architectural constraint. LLMs are probabilistic systems, and security enforcement can’t be. Don’t rely on the model to filter sensitive information after the fact; the data should never enter the context window in the first place.

Data freshness and serving architecture

Separate data construction from data serving. An offline batch pipeline refreshes the stores from source systems on a scheduled cadence, and the real-time agent only reads preprocessed data. This keeps query latency low while letting computationally intensive transformations such as graph construction, embedding generation, and metric aggregation happen asynchronously. The pipeline should be idempotent and observable, with freshness monitoring per store. When the graph is six hours stale but the metrics store is real-time, the agent should know this and communicate its confidence accordingly. Add freshness metadata to every tool response rather than waiting for a user to catch a stale number and lose trust for weeks; that metadata becomes part of the agent’s context for answer generation.

Evaluation and observability

Agent responses need evaluation at both the tool level (“Did the graph query return the right entities?”) and the synthesis level (“Did the final answer correctly combine tool outputs?”). These are different failure modes that require different approaches.

Log every tool call and result, every query decomposition decision, and every synthesis step. You’ll need these traces when something goes wrong, and in a multistore system that can mean a bad graph query, a stale metric, a poorly matched document, or a synthesis error. Without traces, debugging is guesswork. Build automated evaluation pipelines that test known questions against expected answers and track accuracy over time. Degradation usually signals a data quality issue in one of the stores rather than an LLM regression.

Design that observability around the failure modes that actually occur, because production deployments rarely fail because the LLM is inaccurate. They fail because supporting systems drift. Typical examples include:

• Stale graph data causing the agent to reference relationships that no longer exist
• Poorly generated embeddings returning irrelevant documents from the vector store
• Inconsistent business definitions across stores (e.g., “active customer” means different things in the graph and the metrics store)
• Conflicting metrics from different source systems with no reconciliation logic
• Missing source attribution making answers unverifiable

Handling these well matters just as much as prompt engineering. Most debugging sessions trace back to data quality, not model quality.

Implementation guidance

Start with the questions your BI tool can’t answer today (and beyond). The trap with a question inventory is that it captures only what people already know how to ask. The most valuable questions are usually the ones missing from every existing report. They require stitching together three systems, so users either answer them by hand in a spreadsheet or quietly give up. So catalog what your users actually care about, and pay special attention to the questions they route around. Interview analysts about the analysis they abandon halfway, the recurring spreadsheet they dread, and the follow-up question they never bother to ask because the current system makes it too expensive. If most of what you find can still be answered from a single store, you need a better dashboard, not an agent. The pattern earns its complexity only when synthesis across sources is the bottleneck rather than the data access itself.

Pick one use case and go deep. Build for the case where the manual synthesis burden is highest, where an analyst currently spends four hours pulling data from three systems to answer a leadership question. Prove value in that narrow corridor, then expand; adding a new store is incremental once the orchestration layer exists.

Invest in the knowledge graph early. It’s the hardest component to build and the highest-leverage one to have. The schema will be wrong on the first attempt and less wrong on the third. It evolves with your understanding of the domain, and that understanding deepens only through iteration with real users asking real questions.

Design for the analyst, not the engineer. The success metric isn’t technical elegance. It’s whether the finance analyst stops building the same three-system Excel mashup every Monday morning. Talk to your users, watch them work, and build for their actual workflow rather than your ideal architecture.

Measure what matters. Track response accuracy, latency (P50 and P99), user adoption, and reduction in manual synthesis time. Track the questions the agent can’t answer, because those gaps are your roadmap for which stores to add or which schemas to extend.

What this enables at scale

This pattern does more than make existing workflows faster. It enables workflows that weren’t possible before, no matter how many analysts or engineers you threw at the problem. An agent querying a graph, a metrics store, and a document repository at once can surface patterns no human would find by checking each system manually. A finding like “entities in segment X who adopted product Y and had a support escalation in the last 90 days are 3x more likely to churn” requires reasoning across three data sources in a single inference; no dashboard surfaces that, and no analyst checks that specific combination unprompted. In the same motion it democratizes access, opening information that was previously reachable only by engineers who could write Cypher or SQL to anyone who can ask a question in plain language. The analyst’s role shifts from answering routine questions toward building the ontology, curating the graph, and tackling problems that require genuine human judgment.

It also delivers auditable reasoning at enterprise scale. GraphRAG provides deterministic inference paths that are verifiable against source data, so every conclusion traces back through explicit edges and nodes. This builds trust where decisions carry financial, regulatory, or strategic weight. “The AI said so” becomes “the data shows that A connects to B through X, B exhibits property Y, and historical pattern Z suggests the following.” That traceability changes the conversation from “Can we trust AI?” to a review of the work the AI actually did.

Conclusion

The future of enterprise analytics is unlikely to be a larger warehouse or a smarter dashboard. It is an orchestration layer capable of reasoning across specialized systems while preserving each system’s strengths. Purpose-built data stores remain exactly where they are; what changes is how we access them.

The architectural pattern described here doesn’t replace warehouses, graphs, vector stores, or APIs. It coordinates them. That distinction is subtle, but it fundamentally changes what enterprise analytics systems can deliver. The next generation of analytics platforms will do more than answer questions faster. They’ll reason across enterprise knowledge in ways that previously required experienced human analysts. The data already existed. The orchestration layer did not.

Disclaimer: The views and architectural perspectives in this article are entirely my own and do not represent my employer or any affiliated organization. References to patterns and technologies are based on publicly available information and personal experience. No proprietary or internal information was used.

The most consequential AI news of the past year came from a standards body. In December 2025, Anthropic donated the Model Context Protocol to the newly formed Agentic AI Foundation, a directed fund under the Linux Foundation cofounded by Anthropic, Block, and OpenAI, with support from Google, Microsoft, AWS, Cloudflare, and Bloomberg. Six months earlier, Google had handed its Agent2Agent protocol to the same foundation family. Companies that compete fiercely on models are now cooperating, formally and under neutral governance, on the interfaces between them.

For three years, the agent story has centered on capability: Models got better at planning, tool use, and long tasks. Integration improved more slowly. Every agent was still wired to its tools, data sources, and host application with custom glue, leaving even successful systems difficult to move or reuse. Standard interfaces change the economics of that work. Networking became an ecosystem when machines could agree on interfaces; programming tools followed the same path with the Language Server Protocol. In my judgment, the standardization now underway will influence what engineering teams ship over the next three years as much as further gains in model capability.

The Model Context Protocol (MCP) standardizes how an AI application connects to tools and context. It uses JSON-RPC messages between hosts, clients, and servers. Servers can expose tools that a model invokes, resources containing context and data, and prompts that describe templated workflows. The spec credits the Language Server Protocol as an inspiration. Language servers replaced a separate integration for every editor-language pair with one interface on each side; MCP applies the same idea to AI applications and integrations. The current spec is dated July 28, 2026, the fifth dated protocol revision since the project launched in November 2024. That pace reflects active governance and warns implementers to expect movement.

Adoption has spread across competing hosts. The MCP maintainers’ December 2025 announcement cited more than 97 million monthly SDK downloads, over 10,000 active servers, and first-class client support across ChatGPT, Claude, Cursor, Gemini, Microsoft Copilot, and Visual Studio Code. Any individual count can be debated. Cross-vendor implementation is harder to dismiss, because a shared interface becomes infrastructure when rivals support it independently.

MCP covers the boundary between an application and its tools. The Agent2Agent protocol (A2A) addresses the next boundary: agents discovering and communicating with one another across vendors. It reached v1.0 in April 2026 with more than 150 supporting organizations, signed Agent Cards for verifiable identity, SDKs in five languages, and general availability in Microsoft Copilot Studio, Azure AI Foundry, and Amazon Bedrock AgentCore. A2A is younger and less proven in production than MCP, but its governance and adoption are moving agent-to-agent communication beyond the confines of a single platform.

The emerging stack extends beyond those two protocols. AGENTS.md, OpenAI’s convention for giving coding agents repository-level instructions, joined MCP as a founding project of the Agentic AI Foundation. The OpenTelemetry generative AI semantic conventions are developing a shared vocabulary for traces and metrics from model and tool calls, though that work has yet to reach stable status. Connection, cooperation, instruction, and telemetry are beginning to acquire common interfaces.

For an engineering organization, the immediate consequence is reuse. An MCP server for an internal ticketing system can serve every compatible IDE, chat application, and agent. Models and hosts will keep changing; the interface contract, server implementation, schemas, and authorization wiring can endure. The integration becomes an asset shared across applications. Language servers followed the same path as editors came and went around them. Agent integrations deserve the same treatment as libraries and services, with owners, versions, tests, and upgrade policies.

Reuse also concentrates attention on a common security boundary. Custom integrations tend to receive separate reviews, when they receive them at all. A shared protocol creates a recognizable seam where teams can specify identity, permissions, data flow, and audit behavior, then apply those controls across many tools and hosts. The MCP authorization specification builds that seam from established IETF work, including OAuth 2.1, protected resource metadata, authorization server metadata, and resource indicators. It requires Proof Key for Code Exchange (PKCE) to protect authorization codes from interception. MCP servers must also reject tokens issued for another audience and must not forward them downstream. The value comes from familiarity: Teams can draw on years of OAuth deployment experience instead of rediscovering its failure modes inside a new protocol.

The spec is equally useful when it states what metadata cannot prove. Tools can carry behavioral annotations describing them as read-only, destructive, idempotent, or open-world. Those terms give hosts a vocabulary for policy. The tools specification still requires clients to treat annotations as untrusted unless they come from a trusted server. Self-description can inform a security decision; it cannot establish the trust on which that decision rests.

Once servers share an interface, they also become discoverable. The MCP Registry is an open catalog and API for public servers, designed to feed downstream marketplaces and private catalogs. Its role resembles npm or PyPI as a discovery mechanism, with an important difference: It stores standardized server metadata and leaves package distribution elsewhere. The registry remains in preview, with possible breaking changes and no durability guarantee.

Discovery brings familiar supply-chain risks to components with unusually powerful access. Typosquatting, abandoned packages, malicious updates, and uncertain provenance now concern software that may hold live credentials and act on production systems. The registry provides namespace verification and moderation, and its downstream model allows organizations to build curated catalogs. An internal subregistry or allowlist is therefore a sensible first control. The standard interface makes that curation practical across multiple hosts.

The same contract improves testing. MCP tools declare a JSON Schema for their inputs and may declare one for structured outputs. Under the current tools specification, servers that declare an output schema must return conforming structured results, and clients should validate them. Teams can test a server without putting a model in the loop, mock it with recorded or synthetic behavior, and contract-test both sides as they would a REST or gRPC boundary.

This separates two kinds of uncertainty that agent evaluations often mix together. Protocol tests can determine whether an integration exchanged valid messages and enforced its contract. Model evaluation can focus on whether the agent chose the right tool and interpreted the result well. OpenTelemetry’s emerging conventions extend this approach to runtime evidence by giving systems a common language for tool calls. Teams will still have to absorb changes as those conventions mature.

These benefits stop at the edge of what the interfaces describe. A schema captures the shape of a tool’s arguments, while its meaning still lives largely in a free-form description that a model must interpret. A schema cannot tell an agent when a tool is appropriate, how its effects interact with other tools, or whether two similarly named operations have equivalent semantics. Portability makes a server available across hosts; behavior can still vary across models and contexts.

The semantic gap leads to a trust gap. MCP can carry a server’s claims, and an A2A Agent Card can carry a digital signature, but a signature only ties a statement to an identity. Engineering organizations still need a basis for deciding which identities, publishers, code, and claims deserve authority. The protocol can carry that decision through a system. It does not make the decision for the organization.

Delegation makes the problem harder. MCP’s authorization model handles a client calling a server with an audience-bound token. Production systems increasingly involve an agent calling another agent, which calls a tool or a third agent. Preventing token passthrough closes a serious hole, yet each downstream hop still needs a narrower grant derived from the user’s original authority. No common mechanism defines how those rights should attenuate across an arbitrary chain. Platforms currently solve this locally or leave too much authority in place.

All of this work is unfolding on young infrastructure. The registry is in preview, the telemetry conventions are unstable, and MCP has produced five dated protocol revisions in less than two years. Revision is how standards mature, so teams should version-pin, keep protocol code behind thin internal adapters, and budget for migrations. Some abstractions will prove wrong because the standards and the underlying practice are developing at the same time.

A sensible response begins with ownership and containment. Each internal server needs a durable owner, and its protocol surface should be treated as an architecture decision. Schema validation, conformance tests, and protocol mocks put that seam under contract. Third-party servers belong behind an allowlist or private catalog, with provenance requirements and corroboration for their annotations. An internal abstraction around the emerging telemetry vocabulary can limit exposure to draft revisions.

Organizations with a large stake in agents should also participate in the standards work. The Agentic AI Foundation and the A2A project are young enough that engineering teams outside the founding companies can still influence what becomes portable, observable, and enforceable. The rules written now will become assumptions embedded in future products.

Model improvements will continue to generate the headlines. Standards determine whether those models can participate in an ecosystem. Ethernet mattered because it became an interface that many implementers could agree on, and language servers mattered because editors and language tools no longer needed to be designed in pairs. Agent systems now have their first interfaces that competitors jointly govern and independently implement. Engineering teams should build on them with two expectations: The interfaces are likely to last, and their current forms will change.

During a service incident, a customer-remediation workflow is moved onto an emergency route because the situation is critical and the team needs a fast resolution. Approvals are shortened, a priority queue is opened, and an on-call agent is cleared to use an alternate procedure until the service recovers. The incident ends, but the route stays active for a small customer segment after everyone has moved on.

The emergency route itself was fine. Production systems need emergency routes, and a human approved this one. The trouble is that the route now runs without a live incident, an owner, or an expiry condition. A controlled exception has quietly settled into the platform’s standard runtime behavior.

This article is about that execution layer: the temporary operating state that lingers in runtime behavior. Temporary authority is easy to grant under pressure and hard to retire once the pressure lifts. An enterprise agent shouldn’t have to guess from prompts or conversation history whether it’s running under normal, incident, or recovery conditions. That state should be handed to it as authoritative runtime input, the way platforms already hand over identity, tenant, environment, and permissions.

An exception is safe while the incident that justified it is live. It becomes dangerous the moment the platform can no longer show the incident has ended.

The problem is unmanaged exception state

Exception drift is what happens when temporary exception behavior outlives its authorized scope, authority, or duration, and emergency accommodations settle into normal execution. The drift is usually quiet: a routing rule that stays reachable, an approval shortcut that survives closure, a tool permission that keeps shaping execution after the triggering condition has passed.

Enterprises already have the human machinery for this. Incident management defines abnormal conditions, change control governs deviations from standard practice, and postincident reviews confirm that temporary measures have closed.¹,² The gap is architectural. Most agent platforms still treat organizational operating state as something outside the runtime rather than an input to it.

Once an accommodation proves useful, it fades into an invisible operating state. The routing rule stays enabled, the shortened approval path stays reachable, and the temporary queue keeps taking work. No dramatic model failure is required, only a platform with no reliable way to close runtime state.

The practical question to ask is “Which operating mode is active for this workflow, this user, this segment, at this moment?” When the answer is left implicit, every agent, workflow, and tool gateway invents its own. Discussions of agent architecture tend to dwell on capabilities such as models, tools, and orchestration, yet production behavior depends just as much on runtime context: identity, tenant, environment, permissions, session state, policy, and tool access. One piece of that context is usually absent. The runtime knows who is acting and what they may do but not whether the organization is under normal conditions, incident response, recovery review, or a declared exception. Exception drift begins in that blind spot.

The exception lifecycle

Organizations move through a predictable sequence: normal operations, incident declaration, a temporary exception window, closure review, and return to normal. Each stage answers a question, from why the exception exists and who authorized it to what evidence shows it’s no longer shaping execution. Most enterprises handle the front of that sequence well; they’re practiced at declaring incidents and authorizing workarounds. The hard part is the retirement: proving that the exception behavior actually disappeared.

Declaring an exception is loud. Retiring one is quiet, especially when the workaround improved throughput or helped the team recover faster. That asymmetry is where drift lives, because an incident can be closed on paper while emergency routing, override policies, or alternate workflows keep influencing execution. From the platform’s point of view, the lifecycle should close only when it can show that every exception path has been retired or formally adopted through change governance.

Why this matters more for AI agents

Agents raise the stakes because they act. Rather than sitting in a config file, they select tools, trigger workflows, coordinate with other agents, and adapt their paths at runtime. An accommodation introduced during an incident can spread through routing, tool use, approval paths, and downstream agents. A traditional exception stays legible in a runbook or workflow definition; an agent can carry the same exception along many paths at once, which makes it harder to find and retire.

For engineers, the missing layer is an authoritative operational-state context that arrives with the request rather than being inferred from it. Historical traces and retained memory can explain why an accommodation once existed. They should never decide whether it’s still authorized. Memory informs execution; operating mode governs it. And when the two disagree, authoritative runtime state wins.

A small example shows the shape of that context. It carries the minimum needed to bind an exception to a mode, a scope, an authority, an expiry, and a status:

{
  "mode": "incident",
  "exception_id": "INC-4721",
  "scope": {
    "segment": "premium-customers",
    "region": "us-east",
    "workflow": "customer-remediation"
  },
  "authority": "service-owner",
  "expires": "incident-close",
  "status": "active"
}

The pattern sits near familiar ones without matching them. Feature-flag platforms such as LaunchDarkly target behavior by context, RBAC governs what a principal may do, and tenancy metadata tells a service where a request belongs.³ Operating mode serves a different purpose. It doesn’t replace policy, permissions, or memory: Permissions determine who may act, and policies determine how they may act. Operating mode determines whether exception behavior is authorized at all. As a result, it acts as a higher-order governance constraint on agents, workflows, approvals, tools, and escalation paths. Operating mode draws its authority from the organization’s incident and change process and represents a governed state of the enterprise, consumed by the runtime.

A design pattern for exception-aware agents

Operating mode becomes actionable once the platform treats it as a first-class runtime construct. Most agent architectures already inject identity, permissions, tenant context, and policy into every request, and operating mode belongs in that set. Agents consume it as authoritative state after the organization declares the exception, scopes it, assigns authority, and sets an expiry, instead of reconstructing it from prompts or accumulated context.

That reframes the architectural question as whether the platform can guarantee that exception behavior is impossible outside an authorized operating mode. Emergency behavior exists because the platform enables it, and for no other reason. Table 1 shows the minimum contract that makes the boundary testable.

A workflow invocation receives that state alongside the user request, and orchestration, routing, and tool gateways read the same state. Figure 1 shows the shape.

Figure 1. Operating mode is published by the systems that own it and injected into the agent runtime.

Building an exception-aware control plane

The natural home for operating mode is an external control plane. Incident management platforms, maintenance window services, and change management workflows already hold authoritative operational state, and exception-aware architectures extend those signals into execution with explicit scope, authority, expiry, and closure semantics.⁴,⁵ Implementation will vary across organizations, but the principle holds: Exception state should be authoritative, observable, and externally managed, kept out of prompts and workflow definitions and away from agent memory.

Exception-aware execution

With explicit operating state, behavior shifts when system state shifts, and prompt wording stops being the lever. Under normal operations, agents run standard workflows, routing, and approvals. Under incident mode, the same workflows can expose scoped accommodations that the authorized mode makes available, such as expedited approvals, alternate routing, deferred reviews, or emergency runbooks.

Table 2’s payoff is testability. A workflow in normal mode should never reach an emergency path, and a workflow in incident mode should reach only the accommodations its scope, authority, and expiry allow. Governance becomes an enforceable runtime property the platform can check at execution time.

Return to the opening incident. With operating mode as runtime state, the emergency route opens only inside a declared exception: scoped to the affected segment and workflow, owned by the service owner, and stamped with an expiry tied to incident closure. When the incident closes, the mode returns to normal and the routing gate stops handing that route to the segment, ahead of any manual cleanup. Closure then runs as a check. The platform replays the exception’s scope against live routing, approval, tool, and queue configuration, and confirms that no path still resolves to the emergency behavior. No one has to remember to retire the route; it was bounded by state, and the platform can show it is gone.

A common operating model for multi-agent systems

The problem compounds across collaborating agents. Customer-facing, orchestration, and execution agents may share a workflow while disagreeing about state, so one keeps applying emergency routing after another has returned to standard controls. A shared operating state gives them a single governance boundary: The exception is represented once and read consistently everywhere. As agent ecosystems grow more autonomous, shared operational state matters as much as shared identity and authorization. Fragmented state produces fragmented accountability.

Making exception drift observable

The payoff is observability. These conditions have historically been hard to see because accommodations scatter across workflow definitions, approval policies, routing configurations, and tool permissions. Explicit operating state makes them measurable at runtime. Closure can trigger automated validation, and the platform can watch for residual exception behavior between closures. The revealing checks are direct: Is the exception’s routing path still reachable, do its temporary approvals or elevated permissions still resolve, and does any expired exception still touch behavior? Tracking how many exceptions are open, how long they stay open, and how often they harden into permanent change turns drift from an audit finding into a monitored signal.

From adaptive agents to governed systems

Table 3 captures the shift. Traditional governance documents exceptions and trusts that they are retired. An exception-aware platform represents them as runtime state that can be propagated, validated, monitored, and closed.

The test is whether the platform can demonstrate that the exception stayed bounded to its purpose and duration.

None of this demands a new governance model. Organizations already know how to govern identity, permissions, change, and operational risk, and the recommendation is to extend those disciplines to operating state. Once operating mode is authoritative, observable, and testable, drift becomes detectable, closure verifiable, and recovery an engineering result. The aim is a platform that can make its own operating conditions explicit, hold every agent to them, and demonstrate a clean return to normal. In that model, temporary accommodations stay temporary, governance lives in the architecture, and enterprise agents earn trust at scale.

References

1. ISO, “ISO 22301:2019: Security and resilience—Business continuity management systems—Requirements,” International Organization for Standardization, 2019. https://www.iso.org/standard/75106.html ︎
2. Axelos, “ITIL 4: Change enablement,” IT service management practice, 2019. https://www.axelos.com ︎
3. LaunchDarkly, “Target with Flags,” LaunchDarkly Documentation. https://launchdarkly.com/docs/home/flags/target ︎
4. ServiceNow, “Change Management,” ServiceNow Product Documentation. https://docs.servicenow.com ︎
5. PagerDuty, “Maintenance Windows,” PagerDuty Support Documentation. https://support.pagerduty.com/main/docs/maintenance-windows ︎

There is more technical content available today than any human being could read in a thousand lifetimes. Every topic has a dozen YouTube videos, three Substack posts, a GitHub repo, and a Reddit thread, most created in the last six months and, in many cases, technically accurate. And yet most of the professionals I talk to say they don’t know what to trust. They can’t tell what’s important to read first, or which of 10 plausible answers is the one that holds up. That was true before AI, and AI has made it more true.

For most of the history of technical publishing, editing and verification were the same process, and that process was slow and expensive. Getting a book out took years. We found an author, vetted them, had them work with an editor, and checked their claims with technical reviewers. A lot of that time went into separating what was correct and useful from what was confusing or only sounded right. It was laborious, but it meant a reader could depend on the claims on the page. The credibility of the book, and of the publisher behind it, mattered as much as the information itself.

When the cost of production drops to zero, that credibility becomes worth more, not less. Content is easier to make than ever, but without a transparent process behind it, readers have no idea where the knowledge came from or whether it holds up. As Jasmine Sun puts it in “The Independent Writer’s Advantage in the Age of AI,” “Trust is not about information and its quality alone. It’s about the messenger. It’s about who says it and their track record and what they’ve told me before.” A practitioner has confidence in a source because someone she respects has put their reputation on the line for it. They believe what the author is saying because the publisher has a history of being right and of correcting itself when it isn’t, and because the work is attributed and verifiable.

The corpus matters, but it’s the assurances around it that are hard to replicate, and that comes not just from the people who produce the content but from the people whose judgment vouches for it. Sometimes a creator brings their own credibility with them. Other times, the publisher spots someone unknown and lends them its own. The art critic Dave Hickey said this about gallery owners in Air Guitar: They gain status from the famous artists they represent and share it with emerging talent who have something to offer but who haven’t had the chance to earn a reputation. This is what O’Reilly has done for nearly half a century, build a network of experts who vouch for what’s worth knowing.

Expertise is alive, and it compounds

Expertise is a living thing, continuously expanding. Content starts to decay the moment it’s published, because frameworks evolve, libraries deprecate, and yesterday’s best practice becomes today’s security incident. Keeping expertise alive requires a pipeline of people who stay current and an editorial layer that notices when something has gone stale, and either retires it or calls for a fix.

That pipeline isn’t something you switch on when an author has a book to ship. At O’Reilly, we’ve always prided ourselves on living at the bleeding edge, finding what Tim O’Reilly calls “the alpha geeks” and spreading what they know to everyone else. Content sits at the center of our platform, but we think about it in pace layers. Some advice is timeless, some moves but has a long shelf life (some of our books are still in print after nearly 50 years!), and some changes weekly. We work with experts at each pace layer, capturing what lasts while doing our best to keep pace with an industry that seems to have changed every time we wake up. We have relationships with hundreds of the best practitioners in the world, and our job is to keep them engaged continuously, with quick takes when something breaks, structured responses when major research drops, and live sessions on emerging topics while they’re still emerging.

An institution doesn’t stamp trust onto content. In a technical community, trust is conferred in both directions. A practitioner earns standing because people who already have standing engage with her work, cite it, argue with it, and build on it. That insight was the whole idea behind PageRank, Google’s first great innovation. A page mattered because other pages that mattered linked to it. Reputation works the same way.

The audience isn’t just consuming reputation signals; it’s generating them. When a senior engineer whose judgment others respect says out loud that something is worth reading, she spends a little of her own credibility; the author gains a little; and everyone watching recalibrates whom to trust next time. O’Reilly plugs into that existing economy of reputation. When we put our mark on someone’s work, we aren’t the sole source of its credibility. We’re amplifying a judgment the community is already making and adding our own track record to it. The reader who finds it reliable hands status back to the source.

When the readers are machines

Human practitioners aren’t the only ones who need trusted engineering knowledge. The AI systems now sitting in every workflow, the coding and debugging agents and architecture advisors, need it just as badly since most of them are built on scraped web data and documentation that was stale before it was ever indexed. They’re fluent, but they’re wrong often enough that you can’t just take their word for it.

The stakes grow with AI increasingly being used to generate not just provably correct types of content like code, which either works or it doesn’t, but persuasive documents in fuzzier areas like hiring, strategy, and so on. Like everyone else leaning on these tools, we at O’Reilly are reckoning with the consequences of the ability to talk to a model and get back something that looks smart at a glance. A few rounds in, the slop is still there. In the last few months, maybe 10 times as many documents have crossed our desks, from new product ideas to strategic plans and proposals. But the ease of generating the text hides the fact that either the model or the person prompting it doesn’t actually know what they’re talking about. Knowledge workers need ways to ground their work in insights from human experts, particularly when that work is AI-assisted. So we’re building tools that let agents draw on our repository of expertise to support their proposed decisions.

Credible sources are particularly important when thinking through and justifying important choices. Our CTO, Andrew Odewahn, describes the shift this way: “18 months ago, it was all about how to get engineers to be more productive, but now it’s about how to get organizations to make better decisions. The engineering tasks are moving away from coding output to planning.” For planning tasks like comparing implementation approaches, you need expert-over-your-shoulder guidance for contextual decision-making. You can’t just rely on an LLM’s best guess to solve your problem, which is why we see great opportunity for new products like O’Reilly’s Expert Intelligence offering that delivers grounded knowledge embedded in your AI tools and your workflows to help navigate what you do. Trust is foundational because the expertise behind it stays genuine, practical, and human.

Pete Warden has spent his career on the frontier of small, local AI, first as one of deep learning’s earliest engineers (he coined the term “TinyML”) and now as founder of Useful Sensors and Moonshine AI, where he builds voice models that run entirely on-device. Pete joined Ben to make the case that local AI no longer has to be a compromise. They get into what it actually takes to run a capable model on a laptop today; why the voice interface’s bad reputation is a consequence of rough, early implementations rather than a reflection of current capabilities; and where he stands in the ongoing debate between general “end-to-end” models and the compound AI approach of chaining specialized models together. Pete also explains why he thinks browser-based inference could be an “iPhone moment” for local AI and why more and more enterprises are considering self-hosted local models over commercial options. “The shape of [LLMs] is perfect for running locally,” Pete says, and local models could be a boon to enterprises worried about cost, privacy, and stability.

About the Generative AI in the Real World podcast: In 2023, ChatGPT put AI on everyone’s agenda. In 2026, the challenge will be turning those agendas into reality. In Generative AI in the Real World, Ben Lorica interviews leaders who are building with AI. Learn from their experience to help put AI to work in your enterprise.

Check out other episodes of this podcast on the O’Reilly learning platform or follow us on YouTube, Spotify, Apple, or wherever you get your podcasts.

Takeaways

01.26 The usability gap is smaller than the marketing gap. The capabilities of local models are only a few months behind those from the big commercial companies, but because there’s no subscription revenue model behind local models, they often go unpromoted. “It’s very hard to make money off local models,” Pete explains, so the big companies aren’t focused on selling them. “Every company is going to go for the [product] that has an easy subscription revenue model. And that means you have a massive ton of marketing around all of these tools that are kind of like, ‘Oh, let’s have a little text box on a website.’ And so it means mostly that people have never heard of these local models.”

04.20 Local models are already good enough for most use cases. Pete compares the moment to the early web, when free alternatives like Apache eventually overtook expensive commercial servers. “All of these alternatives, once people actually had time to look around and they had a little bit of time to improve, they just wiped the floor with the commercial [offerings],” he points out. “I don’t know if we’re going to quite get there, but that’s the kind of pattern that I’m seeing.”

07.26 “The hardware barriers are a lot lower than people think.” Ben and Pete discuss what hardware you actually need to get up and running, from parameter counts, quantization (Q4, 8-bit), and VRAM requirements to the new Apple M5 Studio’s unified memory as a way to run very large models locally at usable speed. “The key thing is whether you can fit [your model] into your graphics card’s memory,” Pete says. “So with weight quantization, 9 billion [parameters] if it was 8 bits is like 9 GB. A lot of mid-end decent laptops that are shipping now have more than that.”

18.33 “It’s not that people don’t like voice interfaces. It’s that people don’t like bad voice interfaces.” We’ve solved most of the big problems, like dealing with background noise, phrasing, and speech in a range of accents—or at least have improved tools’ capabilities. However, “there’s no commercial incentive to kind of pull them all together,” Pete says. Most tools feel like they haven’t caught up to the LLM era, but “open source can be a really strong lever” to updating them, argues Pete.

28.26 We’re navigating the split between “LLM maximalist” end-to-end models (favored by big AI companies with the most capital) and the “compound AI” approach of chaining together specialized models from different sources. “If the future is end-to-end models, then only the people with the most money can actually build and train them,” Pete notes. Compound AI lets you “actually train all of the models independently” to accomplish your particular goals. While the performance of end-to-end models continues to improve, especially for multimodal models like Qwen or Gemma, using one can be a bit like choosing a Swiss Army knife over a tool specially designed to accomplish a single specific task, to use Pete’s metaphor. It may get the job done, but it’s probably not the most effective way to do it.

36.10 Voice capabilities in the browser could be a game changer. Embedding a model directly in the browser—Chrome has a built-in ~4B parameter model that’s accessible from any website via JavaScript, for instance—makes it part of the operating system. “Once you are able to transcribe fast and accurately in the browser, it’s a way for people to easily start experimenting with this stuff,” Pete explains. Could this be an iPhone moment for LLMs?

39:58 The “gravitational pull” is toward on-prem. Unlike most recent technological advances that depend on the cloud to function, LLMs are well-suited to running locally, even with no internet connectivity. Enterprises are grappling with concerns about cost, privacy, capabilities changing with no notice, or even the models they depend on disappearing. Hosting your own model, whether on your laptop or in your corporate infrastructure, gives you the stability to plan for the long term.

44:21 GPUs are fantastic for training but “complete overkill for inference.” Pete likens it to “trying to use an oil tanker to go and do your shopping.” Memory bandwidth is the real limiting factor, and it’s a problem that companies like Apple, with its new chip designs and unified memory bandwidth, are working on solving. “Even if you’re running on the CPU, if you have something that’s got high-enough bandwidth to pull 27 billion weights in a fraction of a second, then the rest of it is fairly easy in terms of actually doing the processing,” Pete says. “I think we’re going to see a lot of really imaginative solutions now that people understand what the workload looks like.”

The following article originally appeared on Addy Osmani’s blog and is being republished here with the author’s permission.

In the past year, the conversation around agentic engineering has moved to harnesses and loops, fleets and software factories. My 2 cents is engineers need to own the outer loop—the accountability for these systems. This only gets more true as powerful models like Fable and GPT-5.6 become available.

Agents have leverage, and leverage creates obligations. Someone must be able to explain exactly what changed, why it was safe, and what will happen if they’re wrong. Otherwise, their actions can’t be justified. Which makes it unlikely their organization will ask for them in the first place.

And so I want to talk about three terms. The first, Quality, refers to all the checks we install before we let the system loose. Those checks produce evidence, and from that evidence we derive a Verdict.

The second, Verdict, refers to the final decision we make before work enters our dependent system: I’m the line-producer of this content. I run the team whose work is shipped under my name. The model may write the line, but the Verdict is mine. The work of my team will not enter our dependent systems without my decision. A Verdict is the production decision: Should we ship, block, redirect, narrow the response, add a guardrail, or reject outright?

The third, Answerability, refers to the guarantee that if someone asks, I can explain why.

To say this another way: Our agent (which I define as a model plus a harness of files, tools, memory, skills, sandboxes, permissions, observability, and recovery) is what runs our loop (which I define as investigation, implementation, verification, and repeat). And it’s what creates our software factory.

The model is just the engine. The harness—tools, memory, permissions, sandboxes, tests—is the car you build around it so it can do real work safely.

The loop is how one good run becomes a process you can trust to run again. Wrap that harness in a repeatable cycle—investigate, implement, verify, repeat—where an independent check, not the model’s own say-so, decides when the work is done.

Now run many loops at once. A factory is loops at scale: The agents ship the work inside, while humans own the decisions at the boundary.

And at the heart of that factory is a careful boundary between what’s inside the system and what’s outside it. Inside the system we collect inputs (from the product team’s intent, or knowledge of previously shipped work, or of recent incidents, or of specific feedback from users). The agent loop investigates the task, implements a plan, and verifies the result. Then, evidence crosses that boundary. A human, who owns the dependent system, sees the evidence and decides whether to proceed.

And that, friends, is the shift we’re trying to make. Before, our agents were doing the inner loop of the execution loop. Now they run the inner execution loop. Engineers own the outer loop.

Inside the system, there’s really just one kind of thing our agents are doing: capability. The capability to investigate tasks, implement plans, test their results, and report back. That’s the capability of a model. And as we’ve said, that future is already here.

Outside the system, there’s a single kind of thing: agency. The agency to decide, verify, approve, and own.

We’re still talking about code, you see. It just needs to live in a place and be performed by people who know what they’re doing.

The potential for AI code is no longer marginal. In a Sonar 2026 survey, we asked teams about the share of their commits that were AI-assisted. It was small but nontrivial. And several of the respondents said they expect the share of AI-assisted commits to grow substantially.

Sonar’s 2026 State of Code report found that 42% of committed code was AI-generated or significantly AI-assisted, with expectations for that share to keep growing rather than plateauing.

Creation, in other words, is getting cheaper. Scarcer resources are review, validation, understanding, and maintenance.

We moved the speed of generation faster than we moved the speed of control, and so we have a trust-verification gap. A lot of people we talk to still express some degree of distrust in AI code. Yet fewer of them seem to consistently build that distrust into their verification processes.

That’s a dangerous place to be. We’re going to need cheaper, clearer ways to verify the trustworthiness of AI code.

If you look at the GitLab June 2026 report, you’ll see that governance questions have shifted.

GitLab’s June 2026 AI accountability research shows that review and validation are the current bottlenecks when using AI and, more worryingly, that governance usually happens after code creation, after we’ve accepted the risk and lost control over ownership. Today, it’s not just about control. It’s about what constraints we set on the system. It’s about how we’ll check the work with evidence, and how we’ll hold teams accountable. It’s about who will own what part of the AI lifecycle.

So the final distinction in this series is between process and quality. Quality is the concept of backpressure. We mean it literally. We don’t want to grant our agents as much autonomy as they can possibly exercise. We want to grant them just enough autonomy that we have enough backpressure to stop them, regulate them, check their work, and ensure our humanity.

Ordinary engineering holds up a lot of signals that indicate that the work being done is doing the right thing. Type checks, tests, hooks, sandbox limits, audit logs, monitors. Our engineering systems are full of these kinds of signals, and they’re designed to provide enough backpressure to keep the system honest.

And so as long as our agents are emitting these same signals, we can trust our ordinary engineering to provide appropriate backpressure.

Trusting our systems doesn’t mean we don’t want a human in the loop. It just means that the human doesn’t need to be in the inner loop. We want them in the constraints loop (What inputs, architectures, instructions, or invariants should we set?), the sampling loop (How much output should we sample and review?), the audit loop (What evidence should we keep, and how do we make sure our audit log is effective?), and the ownership loop (What part of the production boundary should we own?).

But the human doesn’t need to be in the inner loop.

The agent can ship more than you can review.

And the scarce resource is your own core human judgment, informed by quality signals like logs or tests.

The AI June 2026 report shows that, in the experimental setting, agentic delegation along hour-scale time horizons is essentially here. The work by OpenAI this year on agents and the future of work was a great source for these ideas. So we need to start thinking about how to establish this ownership boundary, as our systems start shipping more than we can review.

And that’s where the answerability comes in.

Because with long-horizon agents, the decisions made over hour-scale time horizons are just that—decisions. And not all the decisions are going to be recorded. You can’t trace them all back to input tokens. If all you’re doing is trusting that the output you get is the correct choice for the problem at hand, the hundreds or even thousands of human hours of work you’re going to need to reconstruct the chain of decisions that lead to it become impossible. And so, again, answerability becomes something that must be at the core of our system design.

Three hidden costs

And there are three hidden costs:

Cognitive surrender ~ blindly accepting what AI gives you. When you delegate work to an agent, the work itself may appear to be the work of the agent. But it’s actually your work. It’s your reputation. It’s your responsibility. And it’s your software that suffered the defects in the output. And it’s your software that needs to be changed to reflect that output. So the agent’s output becomes your answer. And with it comes all the accountability. The Wharton study that put this together is reassuring when the AI is right. But when it’s wrong, the news isn’t great. When the AI was wrong, nearly three-quarters of people accepted it anyway, and felt more confident than they would have without the AI.

Cognitive debt ~ erosion of your understanding and memory of how to solve problems. When you delegate work to an agent, you’re offloading all the thought work to the agent. And while thinking it all out yourself takes time and energy, thinking it out on a massive codebase takes resources that aren’t available when you’re trying to run up the learning curve. So the output you get is often unattainable by you. And the longer the time horizon of the agentic planning, the bigger the gap between the code the agent produces and your understanding of it becomes. The gap compounds. The debt accumulates. And the cost of climbing the learning curve grows almost exponentially.

There’s a randomized controlled trial from Anthropic looking at whether engineers who lean on AI to write code understand it as well as engineers who write it themselves. The conclusion was gloomy: On a comprehension quiz, the engineers who worked through AI scored 17 percentage points lower than those who didn’t, 50% versus 67%.

And then there’s the orchestration tax: It’s easy to spin up lots of agents now, but your cognitive bandwidth doesn’t parallelize in the same way. Steering your agent away from the worst behaviors, sorting the work the agent produces to identify the ones that need your attention, directing it to focus on the work you care about first, verifying your most important constraints and your most dangerous assumptions before you let it run. . .

All of that takes work, and it can’t be automated. There’s no substitute for human judgment.

Brownfield systems are especially dangerous here, because the system behavior you have to audit doesn’t live in the code. It lives in the scars.

Fixes? Make attention the priority in your architectural decisions. Use worktrees, scopes, and evidence to reduce the coupling between your initial plan and the work that emerges from it. Time-box the effort to resolve unactionable steps. And make change in your software strictly an opt-in permission.

Alpha, decay, and taste: These are the three core patterns that shape careers and performances across domains.

Alpha is the lead part taken up by the highest achiever in the competition, when you’re playing your highest-value game move. Decays are established patterns that everyone learns through repetition and watching others (plateaus, if you like). Taste is the earliest we can sense the lead in an alpha or the change in a decay. It’s our judgment of what’s coming before we have any evidence that anything is happening.

Paul Graham’s point is that when anyone can make anything, choosing what to make matters more, and Mitchell Hashimoto’s definition is the operational one: making high-quality qualitative judgments where no objective metric exists yet. From now on, taste drives everything. Alpha shifts are taste changes. And decays fade out because we start to taste something different.

Next step? Operationalize your taste. How? Give it a name that reflects what you’re trying to move from limbic to conscious. Practice it in critique and examples. Make its rationale explicit.

And keep making the move that delivers the most durable competitive advantage in your industry. What’s that? Keep moving the edge up from just doing the task to teaching it, systematizing it, deciding when it should be done, and owning the result.

Everyone is a developer, but not everyone is an engineer. Engineering is what a developer turns into when they embrace a work discipline that is more strict: thorough and logically sound reasoning, consideration of constraints and tradeoffs, recognition of risk and exposure, and practical accountability.

In the future, people will leave the administrative work of engineering and embrace new roles that emerge as engineering becomes more demanding. Roles that are unbundled from the spirit of craft but make clear what each person does. There will be those who prototype. Those who build. Those who sweep. Those who grow. Those who maintain.

The humans hold the edge of the system in the other direction too. Increasing the alpha: choosing what is worth doing, defining the constraints within which it should be done, deciding if the evidence is sufficient to proceed, and caring for the result. Whether it’s a single team or a hundred teams, this is the edge that only humans can hold.

Accountability will scale the factory. Like attention and taste, accountability is also one of the three dualities that makes everything work. Without accountability, there are no rules. No wrangling with questioners. No trade-offs. No risks. No safety nets. If nobody owns the consequence of a decision, then high agency can only bring chaos.

The half-life of an edge is one release, but the half-life of a signature is a career. A signature is your name on the work, such that you feel you can stand behind what was shipped. Skills get you leverage; accountability turns leverage into trust.

Only people can choose. Only people inherit consequence. Agents can be asked to choose, route, merge, and escalate safely inside a policy, but they cannot inherit the consequences.

Every codebase should perhaps come with some kind of accountability contract that explicitly states the checklist that was understood when the change was accepted, the evidence that went into the decision, who was accountable for the change, and the system status after the change was blocked. Just like:

• Your attention and taste
• Your evidence, verdict, and ownership
• Your alpha, decay, and taste

High agency

In a typical agentic workflow, high agency is the art of knowing when to delegate, when to inspect, when to stop, and when to own the result of a process. The ladder of agency runs from low to high: flag a potential problem, investigate it, execute against it, diagnose it, propose solutions, recommend fixes, and resolve the issue. A high rung on the agency ladder is discernment: found it, it’s not worth fixing, moving on.

The 12 pillars that hold up the software factory

Brownfield is the frontier for factories that hope to scale. All those clever little innovations may not feel like much yet, but the production environment is a lot. When building an entirely new system, it’s much easier to plan and implement sufficient backpressure mechanisms because you have full control. When you’re adding intelligent agents to a legacy system, however, it’s another matter entirely.

Legacy systems include the entirety of production behavior, future expectations from customers, migration histories, release and budget cycle durations, unspoken assumptions, edge cases, data weirdness, runbook procedurals, and all the scars that accumulated without the will to care for the system.

To be a steward of brownfield requires a form of durable engineering. Work has to be done to turn implicit knowledge into explicit constraints, keep it coherent across teams and through generations, formalize that knowledge into test procedures and functional specifications, and tie that knowledge to objective evidence. All while ratcheting failure into more learning. Because if the system doesn’t get the care it has always received, everything will come crashing down.

New work is real work

The work will get more interesting as you scale. Because when everything else is built, people will want to build new things. They’ll want to employ the alpha and taste they have developed through their craft to design new loops that can be grafted onto the software factory. Or they’ll want to build greenfield systems that employ all the knowledge of the software factory to one elegant, well-meaning, principled effort. They’ll want to design and implement new forms of evidence that will rise to the level of verification for the new systems. They’ll want to take care of brownfield systems that are now so complex they need dedicated attention. They’ll want to design and manage new backpressure mechanisms. They’ll want to design new agents. And they’ll want to build agency.

And, as they do, they’ll come to see that all this is real work. That’s a good thing.

Automation creates bottlenecks. Bottlenecks in production that are worth owning. Because automation gives us control over industrial scale. But there’s also new bottlenecks that arise from industrial scale. The bottleneck moves from “Can we build this?” to “Should this exist? Can we answer for it?”

What I’m suggesting is a practical operating model for scaling agentic engineering. There’s inner and outer loops. The inner loop is where the work is done. Loops are designed to be as independent as possible. Put all quality assurances and verification inside the loop. Once you’ve designed and validated the loop itself, the only thing you have left to do is to grant autonomy by putting in place a back-pressure mechanism that acts to control the rate at which the loop is run and its scope of operation. And put humans in their rightful place, on the right decisions. Don’t treat understanding as a hand-off or a release gate but rather as a point of decision where humans are primed to provide their insight. And then for every artifact that exists and is fed back into production and into new teams and engineers, leave behind better artifacts.

Build the factory; keep the lights on; make work legible, verifiable, owned.

An agent can write it. But before it reaches users, someone must explain why it should exist, why it’s safe enough to be part of production, and what they will do when it is wrong.

That’s agentic engineering at the outer loop—that’s the work now.

The following article originally appeared on Sean Goedecke’s blog and is being republished here with the author’s permission.

In the 2010s, if you had technical gaps (say, you couldn’t write CSS), you had to either rely on a skilled colleague or just hope that the answer to your exact problem was out there on the internet. Today, everyone can write sort-of-okay CSS by delegating the task to an LLM. LLMs make everybody into a generalist.

Because of this, lots of people don’t think there’s any skill involved in working with LLMs. If you want the product that LLMs can deliver—PhD-level mathematics, pretty good but sometimes tasteless computer code, or awkward LinkedIn-style writing—you can simply ask for it. Since everyone is talking to the same models, “skilled prompters” are getting the same results as people touching LLMs for the first time.

This is wrong. The most important skill in prompting is expertise in the domain you’re prompting for.

A good illustration of this is Terence Tao’s conversation with ChatGPT about the recently discovered counterexample to the Jacobian conjecture. This is not the same ChatGPT I talk to! I couldn’t get to where Tao gets, even with unlimited tokens to burn.

There’s a lot to learn about good prompting from Tao’s conversation. Here are a few observations:

• Tao’s messages are very short and to the point. He doesn’t respond point by point to the model, just to the gist.
• The model outputs are much more concise than when I try to talk to GPT-5.6 Sol about mathematics. By signalling expertise, Tao shunts the model into “talking-to-mathematicians” mode, not “explaining-to-amateurs” mode.
• Tao pushes back when the model’s responses look wrong, but he doesn’t directly contradict; instead, he says things like “This looks more complex than I was hoping for.”
• Tao makes several leaps and suggestions himself. He almost never takes the model’s advice about where to go next.

However, you can’t prompt like Tao on mathematical questions just by following these tips. The key to his technique is actually understanding the mathematics: pulling the relevant idea out of ChatGPT’s multiparagraph response, suggesting alternate approaches or formulations, and identifying what “looks weird.”

Terence Tao is a better mathematician than I am a programmer. But the idea here—that domain knowledge makes you better at using LLMs—is something I’ve also experienced in my own work. If you have a good theory of your codebase, you can push the LLM much harder than if you have no familiarity. Because you have your own sense of what a good solution might look like, you can say, “No, I think it could be simpler here” or “But don’t we already do X?” or “Can we express this problem in these familiar terms?”

This touches on an idea I’ve written about before: that system design problems are dominated by concrete specifics, not generic principles. Of course both are useful, but I’d rather have familiarity with the codebase than a deep general understanding of software systems. In his conversation, Terence Tao asks a lot of specific questions like “Does X work here?” or “Given Y and Z, why A?” I can’t ask those questions about the Jacobian conjecture, but I can ask them about the systems I own at GitHub.

If you have no domain knowledge, you can cling onto the LLM to at least get something. That’s not bad! But if you have domain knowledge, you can wring far more value out of the same LLM by steering it hard in the direction you want. Most of us will have to do a mix of both these approaches, since we have domain knowledge in some areas but not others.

The usefulness of domain knowledge suggests that human expertise will continue to be useful even as models get stronger. For many tasks, the human is the bottleneck, not the model, because the difficult part is in communicating to the model exactly what kind of solution the human wants. The information is “in the model” already, but it takes a very smart human to pull it out.

This post got many comments on Hacker News. Some commenters share their anecdotes about how expertise has helped and lack of expertise has hurt. Other commenters say it’s plausible, but they have a sensible suspicion of a view that’s reassuring them about how they’re still valuable. I agree with that, though I suspect by the time we get around to studying this, the landscape will have changed under our feet again. Some commenters point out that OpenAI’s math prompts were inexpert, and so expertise isn’t required. Here I’d respond that OpenAI does have a team of expert mathematicians that checked and filtered the model’s suggested discoveries, and that you cannot currently skip that step.

We still haven’t solved the problem of keeping track of everything we’re feeding our AI agents. Developers now install agent skills, instructions, and other customizations from public repositories by the dozens, and those files end up scattered across user profiles, application folders, and codebases with no record of where they came from or whether they’ve changed since they were first installed.

In this episode of Zero to Agent in 30 Minutes, Microsoft senior solution engineer Maxim Salnikov walked through the Agent Package Manager (APM), a terminal-driven open source product from Microsoft that treats agent context the way modern software already treats its dependencies: versioning it, pinning it, and checking it before it ships. It’s a technical session, but rather than building an AI agent, you’ll discover how to manage all the customizations you’ve installed for your agents and make them portable, secure, and governed by policies you or your company define.

How to build a supply chain for agent context

Maxim demoed the process of setting up and using APM step-by-step. Here’s how it works.

1. Keep every customization in your codebase. Agent skills, custom instructions, and hooks tend to spread across a developer’s machine, a harness’s install folder, and enterprise-level defaults. Maxim recommends storing them in the codebase instead so that anyone on a new machine or using a new harness ends up with the same setup as everyone else. Yes, your codebase will be bigger, but the trade-off for consistency is worth it. But be sure that skills and other customizations are secure before adding them to your codebase. Consider building your very own enterprise repository of the customizations you’ve identified as safe to use. (See step 5.)
2. Install APM and initialize a project. APM works like npm for AI agent customizations. Running apm init sets up an apm.yaml file targeting one or more harnesses, such as GitHub Copilot, Claude Code, or Cursor, and apm install pulls a skill from a repository into the right location for each one. When you install a skill, APM also creates a log file documenting the entire resolution history.
3. Pin to a version or a commit hash. Once a skill is installed, pin it the way you’d pin a library dependency. Doing so protects against a scenario where a legitimate skill later gets compromised and a bad actor slips malicious content into a new version under the same name. APM also scans files for hidden Unicode characters that may contain malicious instructions.
4. Reproduce the exact setup with a lock file. APM logs every file and hash it installs. Running apm install --frozen rebuilds that exact environment from the log instead of reresolving everything from apm.yaml, so a teammate’s machine ends up with precisely the same setup as yours.
5. Source customizations from a vetted registry, and set a policy for it. Public repositories work for experimentation, but Maxim suggests enterprises build a private, gated registry of reviewed skills. APM policy files enforce which sources are approved, so a customization from an unvetted location gets flagged.
6. Audit for drift, then enforce it in CI. The apm audit command checks installed customizations against policy and catches unauthorized sources or content that has changed since installation. Run that same audit as a gate in a CI/CD pipeline to protect the entire organization against skill drift and bad actors.

The software supply chain already has decades of tooling behind it. That discipline hasn’t caught up with agentic AI, but APM is attempting to close that gap. Explore the project GitHub repo and get started.

Coming up next

On September 9, Menyala’s Sajal Sharma joins Zero to Agent in 30 Minutes to build a shared knowledge base that acts as a common brain across agents. He’ll show how a single repository of research, daily logs, and notes can give Claude Code, Codex, OpenClaw, and Hermes access to the same accumulated information instead of starting from zero with every new session.

Follow along with Zero to Agent in 30 Minutes on Radar, or watch the latest episode on YouTube, Spotify, Apple, or wherever you get your podcasts. If you’re an O’Reilly member, you can watch live. Save your seat.

AI is changing how fast organizations can move. Ideas that used to take months to build now take days, sometimes hours. That sounds like good news, and it is. But it creates a new problem. When execution is fast, teams can move in many directions at once. Marketing can ship a new campaign, product can deliver a new feature, and support can change its scripts with the blink of an eye. Each team moves quickly and independently, because they can. Sure, activity moves quickly. But there’s also the chance of chaos. When everyone can move fast on their own, the need for people to move together only grows. Collaboration, cocreation, and alignment need to increase, not decrease, as execution speed increases.

Dashboards don’t interpret themselves

AI and real-time data give organizations more information than ever. Dashboards. Live metrics. Instant customer feedback. All of it moving fast. But data doesn’t make decisions. People do. A dashboard can tell you that cart abandonment jumped 12% this week. It can’t tell you why, and it definitely can’t tell your marketing, product, and support teams what to do about it together. That takes a conversation. It takes people in a room—virtual or real—looking at the same thing, arguing about what it means and what to do next.

The map freezes a moment so you can talk about it

Experience mapping is a broad field of visualizing human experiences. You’re probably familiar with things like journey maps, service blueprints, and other similar diagrams of the experiences. But none of them hand you an answer. What they do is take a fast-moving, chaotic situation and freeze it for a moment. It gives a team something to point at, argue about, and align around.

Picture a typical working session. People from different parts of the business sit down around a map of the customer experience. Each of them already knows a piece of the picture. None of them has the whole picture, together, at the same time. That’s what the map provides. That’s usually the moment something surprising surfaces. Not because the map contains secret information but because it puts scattered knowledge in one place, in front of the people who each hold a piece of it. The visual aspect of maps is critical. Laying out an abstract concept like a “customer experience” allows teams to engage with it in new ways and reach new conclusions that are hard to get from a spreadsheet or data alone. Grasping cause and effect in one visual overview helps teams find the patterns of behavior that matter the most and to conceive of viable interventions.

AI can surface these kinds of patterns in seconds. But it cannot create the moment when a cross-functional team collectively recognizes how its silos are hurting customers. Only people, looking at the same picture, can do that.

The map isn’t the point

Some claim that journey mapping is dead. That static maps can’t keep up with real-time data and AI-driven personalization. This confuses the artifact with the activity. A map that gets built, presented once, and filed away never helps anyone. It fails for the same reason a report fails: Nobody’s talking about it anymore. The value was never in the diagram. It’s in the conversation the diagram makes possible.

Take how I got that team to reach their own conclusions about the invoice problem rather than just telling them about it. After scoping the customer type and situation we wanted to understand better, I interviewed a dozen or so customers about their billing experience. Nothing unusual came up at first. People described the routine steps: get an invoice, check it, pay it. But a few mentioned something in passing. They’d disputed a charge and kept getting late payment warnings anyway, even while the dispute was still open.

From those interviews, I built a draft map of the invoicing journey. I called it a draft on purpose. I didn’t want to hand stakeholders a finished diagram and ask them to approve it. I wanted them to lean into it, question it, and add to it. Then I scheduled a working session. The room included people who’d never worked together before, despite being at the same company for years: billing, support, and product.

We didn’t rush through the map. We slowed down, section by section, and used structured exercises to pinpoint the moments that mattered most to customers. That’s when someone in the room realized: A customer who’s actively disputing an invoice can still get a warning notice for that same invoice. Nobody had designed it that way on purpose. It fell through the gap between two systems that never talked to each other. But once it was visible, laid out in front of the people who owned each part of the process, it became impossible to ignore. The room got quiet, then loud. People were genuinely upset, not at each other, but at what customers were going through.

Of course, I had uncovered this already in my research. And sure, it was also visible on the map. But my diagram wasn’t about giving a magic answer. The process of learning together is the point. That reaction didn’t come from a dashboard. It came from people confronting the evidence together, in the same room, at the same time.

What actually changed

Before the workshop, this problem was invisible in a specific way. Support knew customers complained about warning notices. Billing knew disputes existed. Product knew the systems didn’t sync. But no one held all three pieces at once. The map put all three in the same field of view. That’s the mechanism. Mapping doesn’t create new information. It puts existing, scattered information into one shared picture, at the same time, in front of the people who each hold a piece of it.

What changed after that: Billing and product agreed to flag disputed invoices so no warning could go out. Support got a way to check dispute status before responding to a complaint. And the three teams kept meeting monthly, something none of them had done before. The map didn’t do any of that. The conversation the map created did.

What good collaboration looks like

We started with customer evidence and a deliberately unfinished map. We included people who owned different parts of the experience and asked them to question what the map showed, identify what they knew and what they were assuming, and examine the gaps between their systems. The session ended with specific commitments, and the teams continued meeting as they learned more.

That is what getting collaboration right requires: the right people, shared evidence, visible disagreement, clear ownership of the next decision, and a cadence for revisiting what the team thinks it knows. Without those conditions, mapping can easily become another workshop that produces an attractive artifact but little change.

What this means for your team

As AI speeds up execution, don’t cut the time you spend aligning as a team. Protect it. Expand it. AI won’t give you an edge. Your competitors have access to the same models you do, trained on much of the same data, producing much of the same output. If everyone moves at the same speed, speed stops being an advantage. It becomes the minimum bar for staying in the game. AI also works like a spotlight, amplifying whatever’s already happening in your organization. If your teams collaborate well, AI makes that strength visible fast. If they’re siloed, AI exposes it just as fast.  Now is the time to get collaboration right, while staying focused on the customer. Waiting until AI forces the issue is waiting too long.

In the end, AI can help with customer discovery and accelerate insights. But it doesn’t replace human judgment and decision making. Rallying around a map—a visual depiction of customer experiences—provides a natural forum for discussion, debate, and shared understanding to align before acting. The tools will keep getting faster. The organizations that win won’t be the ones with the best dashboards. They’ll be the ones who are best at coming together, again and again, to make sense of what those dashboards show them.

If you want to dive deeper into mapping, join Jim on October 9 for his Beyond the Book conversation about the latest edition of Mapping Experiences. He and host Vicki Reyzelman will chat about how experience mapping has evolved from a UX technique into a strategic capability for organizations, how AI is transforming the way we create and analyze maps, and how you can use mapping to align business goals with customer needs, facilitate collaboration across teams, and drive transformation at scale. It’s free to attend. Register now.