This post first appeared on Addy Osmani’s Elevate Substack newsletter and is being republished here with the author’s permission.

TL;DR: Aim for a clear spec covering just enough nuance (this may include structure, style, testing, boundaries. . .) to guide the AI without overwhelming it. Break large tasks into smaller ones versus keeping everything in one large prompt. Plan first in read-only mode, then execute and iterate continuously.

“I’ve heard a lot about writing good specs for AI agents, but haven’t found a solid framework yet. I could write a spec that rivals an RFC, but at some point the context is too large and the model breaks down.”

Many developers share this frustration. Simply throwing a massive spec at an AI agent doesn’t work—context window limits and the model’s “attention budget” get in the way. The key is to write smart specs: documents that guide the agent clearly, stay within practical context sizes, and evolve with the project. This guide distills best practices from my use of coding agents including Claude Code and Gemini CLI into a framework for spec-writing that keeps your AI agents focused and productive.

We’ll cover five principles for great AI agent specs, each starting with a bolded takeaway.

1. Start with a High-Level Vision and Let the AI Draft the Details

Kick off your project with a concise high-level spec, then have the AI expand it into a detailed plan.

Instead of overengineering upfront, begin with a clear goal statement and a few core requirements. Treat this as a “product brief” and let the agent generate a more elaborate spec from it. This leverages the AI’s strength in elaboration while you maintain control of the direction. This works well unless you already feel you have very specific technical requirements that must be met from the start.

Why this works: LLM-based agents excel at fleshing out details when given a solid high-level directive, but they need a clear mission to avoid drifting off course. By providing a short outline or objective description and asking the AI to produce a full specification (e.g., a spec.md), you create a persistent reference for the agent. Planning in advance matters even more with an agent: You can iterate on the plan first, then hand it off to the agent to write the code. The spec becomes the first artifact you and the AI build together.

Practical approach: Start a new coding session by prompting 

You are an AI software engineer. Draft a detailed specification for
[project X] covering objectives, features, constraints, and a step-by-step plan.

Keep your initial prompt high-level: e.g., “Build a web app where users can
track tasks (to-do list), with user accounts, a database, and a simple UI.”

The agent might respond with a structured draft spec: an overview, feature list, tech stack suggestions, data model, and so on. This spec then becomes the “source of truth” that both you and the agent can refer back to. GitHub’s AI team promotes spec-driven development where “specs become the shared source of truth…living, executable artifacts that evolve with the project.” Before writing any code, review and refine the AI’s spec. Make sure it aligns with your vision and correct any hallucinations or off-target details.

Use Plan Mode to enforce planning-first: Tools like Claude Code offer a Plan Mode that restricts the agent to read-only operations—it can analyze your codebase and create detailed plans but won’t write any code until you’re ready. This is ideal for the planning phase: Start in Plan Mode (Shift+Tab in Claude Code), describe what you want to build, and let the agent draft a spec while exploring your existing code. Ask it to clarify ambiguities by questioning you about the plan. Have it review the plan for architecture, best practices, security risks, and testing strategy. The goal is to refine the plan until there’s no room for misinterpretation. Only then do you exit Plan Mode and let the agent execute. This workflow prevents the common trap of jumping straight into code generation before the spec is solid.

Use the spec as context: Once approved, save this spec (e.g., as SPEC.md) and feed relevant sections into the agent as needed. Many developers using a strong model do exactly this. The spec file persists between sessions, anchoring the AI whenever work resumes on the project. This mitigates the forgetfulness that can happen when the conversation history gets too long or when you have to restart an agent. It’s akin to how one would use a product requirements document (PRD) in a team: a reference that everyone (human or AI) can consult to stay on track. Experienced folks often “write good documentation first and the model may be able to build the matching implementation from that input alone” as one engineer observed. The spec is that documentation.

Keep it goal oriented: A high-level spec for an AI agent should focus on what and why more than the nitty-gritty how (at least initially). Think of it like the user story and acceptance criteria: Who is the user? What do they need? What does success look like? (For example, “User can add, edit, complete tasks; data is saved persistently; the app is responsive and secure.”) This keeps the AI’s detailed spec grounded in user needs and outcome, not just technical to-dos. As the GitHub Spec Kit docs put it, provide a high-level description of what you’re building and why, and let the coding agent generate a detailed specification focusing on user experience and success criteria. Starting with this big-picture vision prevents the agent from losing sight of the forest for the trees when it later gets into coding.

2. Structure the Spec Like a Professional PRD (or SRS)

Treat your AI spec as a structured document (PRD) with clear sections, not a loose pile of notes.

Many developers treat specs for agents much like traditional product requirement documents (PRDs) or system design docs: comprehensive, well-organized, and easy for a “literal-minded” AI to parse. This formal approach gives the agent a blueprint to follow and reduces ambiguity.

The six core areas

GitHub’s analysis of over 2,500 agent configuration files revealed a clear pattern: The most effective specs cover six areas. Use this as a checklist for completeness:

1. Commands: Put executable commands early—not just tool names but full commands with flags: npm test, pytest -v, npm run build. The agent will reference these constantly.
2. Testing: How to run tests, what framework you use, where test files live, and what coverage expectations exist.
3. Project structure: Where source code lives, where tests go, where docs belong. Be explicit: “src/ for application code, tests/ for unit tests, docs/ for documentation.”
4. Code style: One real code snippet showing your style beats three paragraphs describing it. Include naming conventions, formatting rules, and examples of good output.
5. Git workflow: Branch naming, commit message format, PR requirements. The agent can follow these if you spell them out.
6. Boundaries: What the agent should never touch—secrets, vendor directories, production configs, specific folders. “Never commit secrets” was the single most common helpful constraint in the GitHub study.

Be specific about your stack: Say “React 18 with TypeScript, Vite, and Tailwind CSS,” not “React project.” Include versions and key dependencies. Vague specs produce vague code.

Use a consistent format: Clarity is king. Many devs use Markdown headings or even XML-like tags in the spec to delineate sections because AI models handle well-structured text better than free-form prose. For example, you might structure the spec as:

# Project Spec: My team's tasks app


## Objective
- Build a web app for small teams to manage tasks...


## Tech Stack
- React 18+, TypeScript, Vite, Tailwind CSS
- Node.js/Express backend, PostgreSQL, Prisma ORM


## Commands
- Build: `npm run build` (compiles TypeScript, outputs to dist/)
- Test: `npm test` (runs Jest, must pass before commits)
- Lint: `npm run lint --fix` (auto-fixes ESLint errors)


## Project Structure
- `src/` – Application source code
- `tests/` – Unit and integration tests
- `docs/` – Documentation


## Boundaries
- ✅ Always: Run tests before commits, follow naming conventions
- ⚠ Ask first: Database schema changes, adding dependencies
- 🚫 Never: Commit secrets, edit node_modules/, modify CI config

This level of organization not only helps you think clearly but also helps the AI find information. Anthropic engineers recommend organizing prompts into distinct sections (like <background>, <instructions>, <tools>, <output_format> etc.) for exactly this reason: It gives the model strong cues about which info is which. And remember, “minimal does not necessarily mean short”—don’t shy away from detail in the spec if it matters, but keep it focused.

Integrate specs into your toolchain: Treat specs as “executable artifacts” tied to version control and CI/CD. The GitHub Spec Kit uses a four-phase gated workflow that makes your specification the center of your engineering process. Instead of writing a spec and setting it aside, the spec drives the implementation, checklists, and task breakdowns. Your primary role is to steer; the coding agent does the bulk of the writing. Each phase has a specific job, and you don’t move to the next one until the current task is fully validated:

1. Specify: You provide a high-level description of what you’re building and why, and the coding agent generates a detailed specification. This isn’t about technical stacks or app design—it’s about user journeys, experiences, and what success looks like. Who will use this? What problem does it solve? How will they interact with it? Think of it as mapping the user experience you want to create, and letting the coding agent flesh out the details. This becomes a living artifact that evolves as you learn more.

2. Plan: Now you get technical. You provide your desired stack, architecture, and constraints, and the coding agent generates a comprehensive technical plan. If your company standardizes on certain technologies, this is where you say so. If you’re integrating with legacy systems or have compliance requirements, all of that goes here. You can ask for multiple plan variations to compare approaches. If you make internal docs available, the agent can integrate your architectural patterns directly into the plan.

3. Tasks: The coding agent takes the spec and plan and breaks them into actual work—small, reviewable chunks that each solve a specific piece of the puzzle. Each task should be something you can implement and test in isolation, almost like test-driven development for your AI agent. Instead of “build authentication,” you get concrete tasks like “create a user registration endpoint that validates email format.”

4. Implement: Your coding agent tackles tasks one by one (or in parallel). Instead of reviewing thousand-line code dumps, you review focused changes that solve specific problems. The agent knows what to build (specification), how to build it (plan), and what to work on (task). Crucially, your role is to verify at each phase: Does the spec capture what you want? Does the plan account for constraints? Are there edge cases the AI missed? The process builds in checkpoints for you to critique, spot gaps, and course-correct before moving forward.

This gated workflow prevents what Willison calls “house of cards code”: fragile AI outputs that collapse under scrutiny. Anthropic’s Skills system offers a similar pattern, letting you define reusable Markdown-based behaviors that agents invoke. By embedding your spec in these workflows, you ensure the agent can’t proceed until the spec is validated, and changes propagate automatically to task breakdowns and tests.

Consider agents.md for specialized personas: For tools like GitHub Copilot, you can create agents.md files that define specialized agent personas—a @docs-agent for technical writing, a @test-agent for QA, a @security-agent for code review. Each file acts as a focused spec for that persona’s behavior, commands, and boundaries. This is particularly useful when you want different agents for different tasks rather than one general-purpose assistant.

Design for agent experience (AX): Just as we design APIs for developer experience (DX), consider designing specs for “agent experience.” This means clean, parseable formats: OpenAPI schemas for any APIs the agent will consume, llms.txt files that summarize documentation for LLM consumption, and explicit type definitions. The Agentic AI Foundation (AAIF) is standardizing protocols like MCP (Model Context Protocol) for tool integration. Specs that follow these patterns are easier for agents to consume and act on reliably.

PRD versus SRS mindset: It helps to borrow from established documentation practices. For AI agent specs, you’ll often blend these into one document (as illustrated above), but covering both angles serves you well. Writing it like a PRD ensures you include user-centric context (“the why behind each feature”) so the AI doesn’t optimize for the wrong thing. Expanding it like an SRS ensures you nail down the specifics the AI will need to actually generate correct code (like what database or API to use). Developers have found that this extra upfront effort pays off by drastically reducing miscommunications with the agent later.

Make the spec a “living document”: Don’t write it and forget it. Update the spec as you and the agent make decisions or discover new info. If the AI had to change the data model or you decided to cut a feature, reflect that in the spec so it remains the ground truth. Think of it as version-controlled documentation. In spec-driven workflows, the spec drives implementation, tests, and task breakdowns, and you don’t move to coding until the spec is validated. This habit keeps the project coherent, especially if you or the agent step away and come back later. Remember, the spec isn’t just for the AI—it helps you as the developer maintain oversight and ensure the AI’s work meets the real requirements.

3. Break Tasks into Modular Prompts and Context, Not One Big Prompt

Divide and conquer: Give the AI one focused task at a time rather than a monolithic prompt with everything at once.

Experienced AI engineers have learned that trying to stuff the entire project (all requirements, all code, all instructions) into a single prompt or agent message is a recipe for confusion. Not only do you risk hitting token limits; you also risk the model losing focus due to the “curse of instructions”—too many directives causing it to follow none of them well. The solution is to design your spec and workflow in a modular way, tackling one piece at a time and pulling in only the context needed for that piece.

The curse of too much context/instructions: Research has confirmed what many devs anecdotally saw: as you pile on more instructions or data into the prompt, the model’s performance in adhering to each one drops significantly. One study dubbed this the “curse of instructions”, showing that even GPT-4 and Claude struggle when asked to satisfy many requirements simultaneously. In practical terms, if you present 10 bullet points of detailed rules, the AI might obey the first few and start overlooking others. The better strategy is iterative focus. Guidelines from industry suggest decomposing complex requirements into sequential, simple instructions as a best practice. Focus the AI on one subproblem at a time, get that done, then move on. This keeps the quality high and errors manageable.

Divide the spec into phases or components: If your spec document is very long or covers a lot of ground, consider splitting it into parts (either physically separate files or clearly separate sections). For example, you might have a section for “backend API spec” and another for “frontend UI spec.” You don’t need to always feed the frontend spec to the AI when it’s working on the backend, and vice versa. Many devs using multi-agent setups even create separate agents or subprocesses for each part (e.g., one agent works on database/schema, another on API logic, another on frontend—each with the relevant slice of the spec). Even if you use a single agent, you can emulate this by copying only the relevant spec section into the prompt for that task. Avoid context overload: Don’t mix authentication tasks with database schema changes in one go, as the DigitalOcean AI guide warns. Keep each prompt tightly scoped to the current goal.

Extended TOC/summaries for large specs: One clever technique is to have the agent build an extended table of contents with summaries for the spec. This is essentially a “spec summary” that condenses each section into a few key points or keywords, and references where details can be found. For example, if your full spec has a section on security requirements spanning 500 words, you might have the agent summarize it to: “Security: Use HTTPS, protect API keys, implement input validation (see full spec §4.2).” By creating a hierarchical summary in the planning phase, you get a bird’s-eye view that can stay in the prompt, while the fine details remain offloaded unless needed. This extended TOC acts as an index: The agent can consult it and say, “Aha, there’s a security section I should look at,” and you can then provide that section on demand. It’s similar to how a human developer skims an outline and then flips to the relevant page of a spec document when working on a specific part.

To implement this, you can prompt the agent after writing the spec: “Summarize the spec above into a very concise outline with each section’s key points and a reference tag.” The result might be a list of sections with one or two sentence summaries. That summary can be kept in the system or assistant message to guide the agent’s focus without eating up too many tokens. This hierarchical summarization approach is known to help LLMs maintain long-term context by focusing on the high-level structure. The agent carries a “mental map” of the spec.

Utilize subagents or “skills” for different spec parts: Another advanced approach is using multiple specialized agents (what Anthropic calls subagents or what you might call “skills”). Each subagent is configured for a specific area of expertise and given the portion of the spec relevant to that area. For instance, you might have a database designer subagent that only knows about the data model section of the spec, and an API coder subagent that knows the API endpoints spec. The main agent (or an orchestrator) can route tasks to the appropriate subagent automatically.

The benefit is each agent has a smaller context window to deal with and a more focused role, which can boost accuracy and allow parallel work on independent tasks. Anthropic’s Claude Code supports this by letting you define subagents with their own system prompts and tools. “Each subagent has a specific purpose and expertise area, uses its own context window separate from the main conversation, and has a custom system prompt guiding its behavior,” as their docs describe. When a task comes up that matches a subagent’s domain, Claude can delegate that task to it, with the subagent returning results independently.

Parallel agents for throughput: Running multiple agents simultaneously is emerging as “the next big thing” for developer productivity. Rather than waiting for one agent to finish before starting another task, you can spin up parallel agents for non-overlapping work. Willison describes this as “embracing parallel coding agents” and notes it’s “surprisingly effective, if mentally exhausting.” The key is scoping tasks so agents don’t step on each other: One agent codes a feature while another writes tests, or separate components get built concurrently. Orchestration frameworks like LangGraph or OpenAI Swarm can help coordinate these agents, and shared memory via vector databases (like Chroma) lets them access common context without redundant prompting.

Single versus multi-agent: When to use each

In practice, using subagents or skill-specific prompts might look like: You maintain multiple spec files (or prompt templates)—e.g., SPEC_backend.md, SPEC_frontend.md—and you tell the AI, “For backend tasks, refer to SPEC_backend; for frontend tasks refer to SPEC_frontend.” Or in a tool like Cursor/Claude, you actually spin up a subagent for each. This is certainly more complex to set up than a single-agent loop, but it mimics what human developers do: We mentally compartmentalize a large spec into relevant chunks. (You don’t keep the whole 50-page spec in your head at once; you recall the part you need for the task at hand, and have a general sense of the overall architecture.) The challenge, as noted, is managing interdependencies: The subagents must still coordinate. (The frontend needs to know the API contract from the backend spec, etc.) A central overview (or an “architect” agent) can help by referencing the subspecs and ensuring consistency.

Focus each prompt on one task/section: Even without fancy multi-agent setups, you can manually enforce modularity. For example, after the spec is written, your next move might be: “Step 1: Implement the database schema.” You feed the agent the database section of the spec only, plus any global constraints from the spec (like tech stack). The agent works on that. Then for Step 2, “Now implement the authentication feature”, you provide the auth section of the spec and maybe the relevant parts of the schema if needed. By refreshing the context for each major task, you ensure the model isn’t carrying a lot of stale or irrelevant information that could distract it. As one guide suggests: “Start fresh: begin new sessions to clear context when switching between major features.” You can always remind the agent of critical global rules (from the spec’s constraints section) each time, but don’t shove the entire spec in if it’s not all needed.

Use in-line directives and code TODOs: Another modularity trick is to use your code or spec as an active part of the conversation. For instance, scaffold your code with // TODO comments that describe what needs to be done, and have the agent fill them one by one. Each TODO essentially acts as a mini-spec for a small task. This keeps the AI laser focused (“implement this specific function according to this spec snippet”), and you can iterate in a tight loop. It’s similar to giving the AI a checklist item to complete rather than the whole checklist at once.

The bottom line: Small, focused context beats one giant prompt. This improves quality and keeps the AI from getting “overwhelmed” by too much at once. As one set of best practices sums up, provide “One Task Focus” and “Relevant info only” to the model, and avoid dumping everything everywhere. By structuring the work into modules—and using strategies like spec summaries or subspec agents—you’ll navigate around context size limits and the AI’s short-term memory cap. Remember, a well-fed AI is like a well-fed function: Give it only the inputs it needs for the job at hand.

4. Build in Self-Checks, Constraints, and Human Expertise

Make your spec not just a to-do list for the agent but also a guide for quality control—and don’t be afraid to inject your own expertise.

A good spec for an AI agent anticipates where the AI might go wrong and sets up guardrails. It also takes advantage of what you know (domain knowledge, edge cases, “gotchas”) so the AI doesn’t operate in a vacuum. Think of the spec as both coach and referee for the AI: It should encourage the right approach and call out fouls.

Use three-tier boundaries: GitHub’s analysis of 2,500+ agent files found that the most effective specs use a three-tier boundary system rather than a simple list of don’ts. This gives the agent clearer guidance on when to proceed, when to pause, and when to stop:

Always do: Actions the agent should take without asking. “Always run tests before commits.” “Always follow the naming conventions in the style guide.” “Always log errors to the monitoring service.”

Ask first: Actions that require human approval. “Ask before modifying database schemas.” “Ask before adding new dependencies.” “Ask before changing CI/CD configuration.” This tier catches high-impact changes that might be fine but warrant a human check.

Never do: Hard stops. “Never commit secrets or API keys.” “Never edit node_modules/ or vendor/.” “Never remove a failing test without explicit approval.” “Never commit secrets” was the single most common helpful constraint in the study.

This three-tier approach is more nuanced than a flat list of rules. It acknowledges that some actions are always safe, some need oversight, and some are categorically off-limits. The agent can proceed confidently on “Always” items, flag “Ask first” items for review, and hard-stop on “Never” items.

Encourage self-verification: One powerful pattern is to have the agent verify its work against the spec automatically. If your tooling allows, you can integrate checks like unit tests or linting that the AI can run after generating code. But even at the spec/prompt level, you can instruct the AI to double-check (e.g., “After implementing, compare the result with the spec and confirm all requirements are met. List any spec items that are not addressed.”). This pushes the LLM to reflect on its output relative to the spec, catching omissions. It’s a form of self-audit built into the process.

For instance, you might append to a prompt: “(After writing the function, review the above requirements list and ensure each is satisfied, marking any missing ones).” The model will then (ideally) output the code followed by a short checklist indicating if it met each requirement. This reduces the chance it forgets something before you even run tests. It’s not foolproof, but it helps.

LLM-as-a-Judge for subjective checks: For criteria that are hard to test automatically—code style, readability, adherence to architectural patterns—consider using “LLM-as-a-Judge.” This means having a second agent (or a separate prompt) review the first agent’s output against your spec’s quality guidelines. Anthropic and others have found this effective for subjective evaluation. You might prompt “Review this code for adherence to our style guide. Flag any violations.” The judge agent returns feedback that either gets incorporated or triggers a revision. This adds a layer of semantic evaluation beyond syntax checks.

Conformance testing: Willison advocates building conformance suites—language-independent tests (often YAML based) that any implementation must pass. These act as a contract: If you’re building an API, the conformance suite specifies expected inputs/outputs, and the agent’s code must satisfy all cases. This is more rigorous than ad hoc unit tests because it’s derived directly from the spec and can be reused across implementations. Include conformance criteria in your spec’s success section (e.g., “Must pass all cases in conformance/api-tests.yaml”).

Leverage testing in the spec: If possible, incorporate a test plan or even actual tests in your spec and prompt flow. In traditional development, we use TDD or write test cases to clarify requirements—you can do the same with AI. For example, in the spec’s success criteria, you might say, “These sample inputs should produce these outputs…” or “The following unit tests should pass.” The agent can be prompted to run through those cases in its head or actually execute them if it has that capability. Willison noted that having a robust test suite is like giving the agents superpowers: They can validate and iterate quickly when tests fail. In an AI coding context, writing a bit of pseudocode for tests or expected outcomes in the spec can guide the agent’s implementation. Additionally, you can use a dedicated “test agent” in a subagent setup that takes the spec’s criteria and continuously verifies the “code agent’s” output.

Bring your domain knowledge: Your spec should reflect insights that only an experienced developer or someone with context would know. For example, if you’re building an ecommerce agent and you know that “products” and “categories” have a many-to-many relationship, state that clearly. (Don’t assume the AI will infer it—it might not.) If a certain library is notoriously tricky, mention pitfalls to avoid. Essentially, pour your mentorship into the spec. The spec can contain advice like “If using library X, watch out for memory leak issue in version Y (apply workaround Z).” This level of detail is what turns an average AI output into a truly robust solution, because you’ve steered the AI away from common traps.

Also, if you have preferences or style guidelines (say, “use functional components over class components in React”), encode that in the spec. The AI will then emulate your style. Many engineers even include small examples in the spec (for instance, “All API responses should be JSON, e.g., {“error”: “message”} for errors.”). By giving a quick example, you anchor the AI to the exact format you want.

Minimalism for simple tasks: While we advocate thorough specs, part of expertise is knowing when to keep it simple. For relatively simple, isolated tasks, an overbearing spec can actually confuse more than help. If you’re asking the agent to do something straightforward (like “center a div on the page”), you might just say, “Make sure to keep the solution concise and do not add extraneous markup or styles.” No need for a full PRD there. Conversely, for complex tasks (like “implement an OAuth flow with token refresh and error handling”), that’s when you break out the detailed spec. A good rule of thumb: Adjust spec detail to task complexity. Don’t underspec a hard problem (the agent will flail or go off-track), but don’t overspec a trivial one (the agent might get tangled or use up context on unnecessary instructions).

Maintain the AI’s “persona” if needed: Sometimes, part of your spec is defining how the agent should behave or respond, especially if the agent interacts with users. For example, if building a customer support agent, your spec might include guidelines like “Use a friendly and professional tone” and “If you don’t know the answer, ask for clarification or offer to follow up rather than guessing.” These kinds of rules (often included in system prompts) help keep the AI’s outputs aligned with expectations. They are essentially spec items for AI behavior. Keep them consistent and remind the model of them if needed in long sessions. (LLMs can “drift” in style over time if not kept on a leash.)

You remain the exec in the loop: The spec empowers the agent, but you remain the ultimate quality filter. If the agent produces something that technically meets the spec but doesn’t feel right, trust your judgement. Either refine the spec or directly adjust the output. The great thing about AI agents is they don’t get offended—if they deliver a design that’s off, you can say, “Actually, that’s not what I intended, let’s clarify the spec and redo it.” The spec is a living artifact in collaboration with the AI, not a one-time contract you can’t change.

Simon Willison humorously likened working with AI agents to “a very weird form of management” and even “getting good results out of a coding agent feels uncomfortably close to managing a human intern.” You need to provide clear instructions (the spec), ensure they have the necessary context (the spec and relevant data), and give actionable feedback. The spec sets the stage, but monitoring and feedback during execution are key. If an AI was a “weird digital intern who will absolutely cheat if you give them a chance,” the spec and constraints you write are how you prevent that cheating and keep them on task.

Here’s the payoff: A good spec doesn’t just tell the AI what to build; it also helps it self-correct and stay within safe boundaries. By baking in verification steps, constraints, and your hard-earned knowledge, you drastically increase the odds that the agent’s output is correct on the first try (or at least much closer to correct). This reduces iterations and those “Why on Earth did it do that?” moments.

5. Test, Iterate, and Evolve the Spec (and Use the Right Tools)

Think of spec writing and agent building as an iterative loop: test early, gather feedback, refine the spec, and leverage tools to automate checks.

The initial spec is not the end—it’s the beginning of a cycle. The best outcomes come when you continually verify the agent’s work against the spec and adjust accordingly. Also, modern AI devs use various tools to support this process (from CI pipelines to context management utilities).

Continuous testing: Don’t wait until the end to see if the agent met the spec. After each major milestone or even each function, run tests or at least do quick manual checks. If something fails, update the spec or prompt before proceeding. For example, if the spec said, “Passwords must be hashed with bcrypt” and you see the agent’s code storing plain text, stop and correct it (and remind the spec or prompt about the rule). Automated tests shine here: If you provided tests (or write them as you go), let the agent run them. In many coding agent setups, you can have an agent run npm test or similar after finishing a task. The results (failures) can then feed back into the next prompt, effectively telling the agent “Your output didn’t meet spec on X, Y, Z—fix it.” This kind of agentic loop (code > test > fix > repeat) is extremely powerful and is how tools like Claude Code or Copilot Labs are evolving to handle larger tasks. Always define what “done” means (via tests or criteria) and check for it.

Iterate on the spec itself: If you discover that the spec was incomplete or unclear (maybe the agent misunderstood something or you realized you missed a requirement), update the spec document. Then explicitly resync the agent with the new spec: “I have updated the spec as follows… Given the updated spec, adjust the plan or refactor the code accordingly.” This way the spec remains the single source of truth. It’s similar to how we handle changing requirements in normal dev, but in this case you’re also the product manager for your AI agent. Keep version history if possible (even just via commit messages or notes), so you know what changed and why.

Utilize context management and memory tools: There’s a growing ecosystem of tools to help manage AI agent context and knowledge. For instance, retrieval-augmented generation (RAG) is a pattern where the agent can pull in relevant chunks of data from a knowledge base (like a vector database) on the fly. If your spec is huge, you could embed sections of it and let the agent retrieve the most relevant parts when needed, instead of always providing the whole thing. There are also frameworks implementing the Model Context Protocol (MCP), which automates feeding the right context to the model based on the current task. One example is Context7 (context7.com), which can auto-fetch relevant context snippets from docs based on what you’re working on. In practice, this might mean the agent notices you’re working on “payment processing” and it pulls the payments section of your spec or documentation into the prompt. Consider leveraging such tools or setting up a rudimentary version (even a simple search in your spec document).

Parallelize carefully: Some developers run multiple agent instances in parallel on different tasks (as mentioned earlier with subagents). This can speed up development (e.g., one agent generates code while another simultaneously writes tests, or two features are built concurrently). If you go this route, ensure the tasks are truly independent or clearly separated to avoid conflicts. (The spec should note any dependencies.) For example, don’t have two agents writing to the same file at once. One workflow is to have an agent generate code and another review it in parallel, or to have separate components built that integrate later. This is advanced usage and can be mentally taxing to manage. (As Willison admitted, running multiple agents is surprisingly effective, if mentally exhausting!) Start with at most 2–3 agents to keep things manageable.

Version control and spec locks: Use Git or your version control of choice to track what the agent does. Good version control habits matter even more with AI assistance. Commit the spec file itself to the repo. This not only preserves history, but the agent can even use git diff or blame to understand changes. (LLMs are quite capable of reading diffs.) Some advanced agent setups let the agent query the VCS history to see when something was introduced—surprisingly, models can be “fiercely competent at Git.” By keeping your spec in the repo, you allow both you and the AI to track evolution. There are tools (like GitHub Spec Kit mentioned earlier) that integrate spec-driven development into the Git workflow—for instance, gating merges on updated specs or generating checklists from spec items. While you don’t need those tools to succeed, the takeaway is to treat the spec like code: Maintain it diligently.

Cost and speed considerations: Working with large models and long contexts can be slow and expensive. A practical tip is to use model selection and batching smartly. Perhaps use a cheaper/faster model for initial drafts or repetitions, and reserve the most capable (and expensive) model for final outputs or complex reasoning. Some developers use GPT-4 or Claude for planning and critical steps, but offload simpler expansions or refactors to a local model or a smaller API model. If using multiple agents, maybe not all need to be top tier; a test-running agent or a linter agent could be a smaller model. Also consider throttling context size: Don’t feed 20K tokens if 5K will do. As we discussed, more tokens can mean diminishing returns.

Monitor and log everything: In complex agent workflows, logging the agent’s actions and outputs is essential. Check the logs to see if the agent is deviating or encountering errors. Many frameworks provide trace logs or allow printing the agent’s chain of thought (especially if you prompt it to think step-by-step). Reviewing these logs can highlight where the spec or instructions might have been misinterpreted. It’s not unlike debugging a program—except the “program” is the conversation/prompt chain. If something weird happens, go back to the spec/instructions to see if there was ambiguity.

Learn and improve: Finally, treat each project as a learning opportunity to refine your spec-writing skill. Maybe you’ll discover that a certain phrasing consistently confuses the AI, or that organizing spec sections in a certain way yields better adherence. Incorporate those lessons into the next spec. The field of AI agents is rapidly evolving, so new best practices (and tools) emerge constantly. Stay updated via blogs (like the ones by Simon Willison, Andrej Karpathy, etc.), and don’t hesitate to experiment.

A spec for an AI agent isn’t “write once, done.” It’s part of a continuous cycle of instructing, verifying, and refining. The payoff for this diligence is substantial: By catching issues early and keeping the agent aligned, you avoid costly rewrites or failures later. As one AI engineer quipped, using these practices can feel like having “an army of interns” working for you, but you have to manage them well. A good spec, continuously maintained, is your management tool.

Avoid Common Pitfalls

Before wrapping up, it’s worth calling out antipatterns that can derail even well-intentioned spec-driven workflows. The GitHub study of 2,500+ agent files revealed a stark divide: “Most agent files fail because they’re too vague.” Here are the mistakes to avoid:

Vague prompts: “Build me something cool” or “Make it work better” gives the agent nothing to anchor on. As Baptiste Studer puts it: “Vague prompts mean wrong results.” Be specific about inputs, outputs, and constraints. “You are a helpful coding assistant” doesn’t work. “You are a test engineer who writes tests for React components, follows these examples, and never modifies source code” does.

Overlong contexts without summarization: Dumping 50 pages of documentation into a prompt and hoping the model figures it out rarely works. Use hierarchical summaries (as discussed in principle 3) or RAG to surface only what’s relevant. Context length is not a substitute for context quality.

Skipping human review: Willison has a personal rule—“I won’t commit code I couldn’t explain to someone else.” Just because the agent produced something that passes tests doesn’t mean it’s correct, secure, or maintainable. Always review critical code paths. The “house of cards” metaphor applies: AI-generated code can look solid but collapse under edge cases you didn’t test.

Conflating vibe coding with production engineering: Rapid prototyping with AI (“vibe coding”) is great for exploration and throwaway projects. But shipping that code to production without rigorous specs, tests, and review is asking for trouble. I distinguish “vibe coding” from “AI-assisted engineering”—the latter requires the discipline this guide describes. Know which mode you’re in.

Ignoring the “lethal trifecta”: Willison warns of three properties that make AI agents dangerous: speed (they work faster than you can review), nondeterminism (same input, different outputs), and cost (encouraging corner cutting on verification). Your spec and review process must account for all three. Don’t let speed outpace your ability to verify.

Missing the six core areas: If your spec doesn’t cover commands, testing, project structure, code style, git workflow, and boundaries, you’re likely missing something the agent needs. Use the six-area checklist from section 2 as a sanity check before handing off to the agent.

Conclusion

Writing an effective spec for AI coding agents requires solid software engineering principles combined with adaptation to LLM quirks. Start with clarity of purpose and let the AI help expand the plan. Structure the spec like a serious design document, covering the six core areas and integrating it into your toolchain so it becomes an executable artifact, not just prose. Keep the agent’s focus tight by feeding it one piece of the puzzle at a time (and consider clever tactics like summary TOCs, subagents, or parallel orchestration to handle big specs). Anticipate pitfalls by including three-tier boundaries (always/ask first/never), self-checks, and conformance tests—essentially, teach the AI how to not fail. And treat the whole process as iterative: use tests and feedback to refine both the spec and the code continuously.

Follow these guidelines and your AI agent will be far less likely to “break down” under large contexts or wander off into nonsense.

Happy spec-writing!

Think about what happens when you onboard a new employee.

First, you provision them tools. Email access. Slack. CRM. Office software. Project management software. Development environment. Connecting a person to the system they’ll need to do their job. However, this is necessary but not sufficient. Nobody becomes effective just because they can log into Salesforce.

Then comes the harder part: teaching them how your organization actually works. The analysis methodology your team developed over years of iteration. The quality bar that is not written down anywhere. The implicit ways of working. The judgment calls about when to escalate and when to handle something independently. The institutional knowledge that separates a new hire from someone who’s been there for years.

This second part—the expertise transfer—is where organizations struggle. It’s expensive and inconsistent, and does not scale. It lives in mentorship relationships, institutional knowledge, and documentation that goes stale the moment it’s written.

Claude Skills and MCP (Model Context Protocol) follow exactly this pattern. MCP gives AI agents such as Claude the tools: access to systems, databases, APIs, and resources. Skills are the training materials that teach Claude how to work and how to use these tools.

This distinction matters more than it might first appear. While we have gotten reasonably good at provisioning tools, we have never had a good way to package expertise. Skills change that. They package expertise into a standardized format.

Tools Versus Training

MCP is tool provisioning. It’s the protocol that connects AI agents to external systems: data warehouse, CRM, GitHub repositories, internal APIs, and knowledge bases. Anthropic describes it as “USB-C for AI”—a standardized interface that lets Claude plug into your existing infrastructure. An MCP server might give Claude the ability to query customer records, commit code, send Slack messages, or pull analytics data with authorized permissions.

This is necessary infrastructure. But like giving a new hire database credentials, it does not tell AI agents what to do with that access. MCP answers the question “What tools can an agent use?” It provides capabilities without opinions.

Skills are the training materials. They encode how your organization actually works: which segments matter, what churn signal to watch for, how to structure findings for your quarterly business review, when to flag something for human attention.

Skills answer a different question: “How should an AI agent think about this?” They provide expertise, not just access.

Consider the difference in what you’re creating. Building an MCP server is infrastructure work; it’s an engineering effort to connect systems securely and reliably. Creating a Skill is knowledge work; domain experts articulating what they know, in markdown files, for AI agents to operationalize and understand. These require different people, different processes, and different governance.

The real power emerges when you combine them. MCP connects AI agents to your data warehouse. A Skill teaches AI agents your firm’s analysis methodology and which MCP tools to use. Together, AI agents can perform expert-level analysis on live data, following your specific standards. Neither layer alone gets you there, just as a new hire with database access but no training, or training but no access, won’t be effective at their jobs.

MCP is the toolbox. Skills are the training manuals that teach how to use those tools.

Why Expertise Has Been So Hard to Scale

The training side of onboarding has always been the bottleneck.

Your best analyst retires, and their methods walk out of the door. Onboarding takes months because the real tacit knowledge lives in people’s heads, not in any document a new hire can read. Consistency is impossible when “how we do things here” varies by who trained whom and who worked with whom. Even when you invest heavily in training programs, they produce point-in-time snapshots of expertise that immediately begin to rot.

Previous approaches have all fallen short:

Documentation is passive and quickly outdated. It requires human interpretation, offers no guarantee of correct application, and can’t adapt to novel situations. The wiki page about customer analysis does not help when you encounter an edge case the author never anticipated.

Training programs are expensive, and a certificate of completion says nothing about actual competency.

Checklists and SOPs capture procedure but not judgment. They tell you what to check, not how to think about what you find. They work for mechanical tasks but fail for anything requiring expertise.

We’ve had Custom GPTs, Claude projects, and Gemini Gems attempting to address this. They are useful but opaque. You cannot invoke them based on context; the AI agent working as Copy Editing Gem stays in copy editing and can’t switch to Laundry Buddy Custom GPTs mid-task. They are not transferable and cannot be packaged for distribution.

Skills offer something new: expertise packaged as a versionable, governable artifact.

Skills are files in folders—a SKILL.md document with supporting assets, scripts, and resources. They leverage all the tooling we have built for managing code. Track changes in Git. Roll back mistakes. Maintain audit trails. Review Skills before deployment through PR workflows with version control. Deploy organization-wide and ensure consistency. AI agents can compose Skills for complex workflows, building sophisticated capabilities from simple building blocks.

The architecture also enables progressive disclosure. AI agents see only lightweight metadata until a Skill becomes relevant, then loads the full instruction on demand. You can have dozens of Skills available without overwhelming the model’s precious context window, which is like a human’s short-term memory or a computer’s RAM. Claude loads expertise as needed and coordinates multiple Skills automatically.

This makes the enterprise deployment model tractable. An expert creates a Skill based on best practices, with the help of an AI/ML engineer to audit and evaluate the effectiveness of the Skill. Administrators review and approve it through governance processes. The organization deploys it everywhere simultaneously. Updates propagate instantly from a central source.

One report cites Rakuten achieving 87.5% faster completion of a finance workflow after implementing Skills. Not from AI magic but from finally being able to distribute their analysts’ methodologies across the entire team. That’s the expertise transfer problem, solved.

Training Materials You Can Meter

The onboarding analogy also created a new business model.

When expertise lives in people, you can only monetize it through labor—billable hours, consulting engagements, training programs, maintenance contracts. The expert has to show up, which limits scale and creates key-person dependencies.

Skills separate expertise from the expert. Package your methodology as a Skill. Distribute it via API. Charge based on usage.

A consulting firm’s analysis framework can become a product. A domain expert’s judgment becomes a service. The Skill encodes the expertise; the API calls become the meter. This is service as software, the SaaS of expertise. And it’s only possible because Skills put knowledge in a form that can be distributed, versioned, and billed against.

The architecture is familiar. The Skill is like an application frontend (the expertise, the methodology, the “how”), while MCP connections or API calls form the backend (data access, actions, the “what”). You build training material once and deploy them everywhere, metering usage through the infrastructure layer.

No more selling API endpoints with 500-page obscure documentation explaining what each endpoint does then staffing a team to support it. Now we can package the expertise of how to use those API directly into Skills. Customers can realize the value of an API via their AI agents. Cost to implement and time to implement drop to zero with MCP. Time to value becomes immediate with Skills.

The Visibility Trade-Off

Every abstraction has a cost. Skills trade visibility for scalability, and that trade-off deserves honest examination.

When expertise transfers human to human, through mentorship, working sessions, apprenticeship, the expert sees how their knowledge gets applied and becomes better in the process. They watch the learner struggle with edge cases. They notice which concepts don’t land. They observe how their methods get adapted to new situations. This feedback loop improves the expertise over time.

Skills break that loop. As a Skill builder, you do not see the conversations that trigger your Skill. You do not know how users adapted your methodology or which part of your guidance AI agents weighted most heavily. Users interact with their own AI agents; your Skill is one influence among many.

Your visibility is limited to the infrastructure layer: API calls, MCP tool invocations, and whatever outputs you explicitly capture. You see usage patterns, not the dialogue that surrounds them. Those dialogues reside with the user’s AI agents.

This parallels what happened when companies moved from in-person training to self-service documentation and e-learning. You lost the ability to watch every learner, but you gained the ability to train at scale. Skills make the same exchange; less visibility per user interaction, vastly more interactions possible.

Managing the trade-off requires intentional design. Build logging and tracing into your Skills where appropriate. Create feedback mechanisms inside skills for AI agents to surface when users express confusion or request changes. And in the development process, focus on outcomes—Did the Skill produce good results?—rather than process observation.

In production, the developer of Skills or MCPs will not have most of the context of how a user’s AI agent uses their Skills.

What to Watch

For organizations going through AI transformations, the starting point is an audit of expertise. What knowledge lives only in a specific person’s head? Where does inconsistency emerge because “how we do things” isn’t written down in an operationalizable form? These are your candidates for Skills.

Start with bounded workflows: a report format, an analysis methodology, a review checklist. Prove the pattern before encoding more complex expertise. Govern early. Skills are artifacts that require review, evaluation, and lifecycle management. Establish those processes before Skills proliferate.

For builders, the mental shift is from “prompt” to “product.” Skills are versioned artifacts with users. Design accordingly. Combine Skills with MCP for maximum leverage. Accept the visibility trade-off as the cost of scale.

Several signals suggest where this is heading. Skill marketplaces are emerging. Agent Skills are now a published open standard being adopted by multiple AI agents and soon agent SDKs. Enterprise governance tooling with version control, approval workflows, and audit trails organizations need will determine adoption in regulated industries.

Expertise Can Finally Be Packaged

We’ve gotten good at provisioning tools as APIs. MCP extends that to AI with standardized connections to systems and data.

But tools access was never the bottleneck. Expertise transfer was. The methodology. The judgment. The caveats. The workflows. The institutional knowledge that separates a new hire from a veteran.

Skills are the first serious attempt to package the expertise into a file format, where AI agents can operationalize it while humans can still read, review, and govern. They are training materials that actually scale.

The organizations that figure out how to package their expertise, both for internal and external consumption, will have a structural advantage. Not because AI replaces expertise. Because AI amplifies the expertise of those who know how to share it.

MCP gives AI agents the tools. Skills teach AI agents how to work. The question is whether you can encode what your best people know. Skills are the first real answer.

References

• “What Is the Model Context Protocol (MCP)?,” LF Projects, https://modelcontextprotocol.io/docs/getting-started/intro.
• Michael Nuñez, “How Anthropic’s ‘Skills’ Make Claude Faster, Cheaper, and More Consistent for Business Workflows,” VentureBeat, October 16, 2025, https://venturebeat.com/ai/how-anthropics-skills-make-claude-faster-cheaper-and-more-consistent-for.
• “Skills,” Anthropics, https://github.com/anthropics/skills.
• “Create and Distribute a Plugin Marketplace,” Claude Code Docs, https://code.claude.com/docs/en/plugin-marketplaces.

The following article includes clips from a recent Live with Tim O’Reilly interview. You can watch the full version on the O’Reilly Media learning platform.

Addy Osmani is one of my favorite people to talk with about the state of software engineering with AI. He spent 14 years leading Chrome’s developer experience team at Google, and recently moved to Google Cloud AI to focus on Gemini and agent development. He’s also the author of numerous books for O’Reilly, including The Effective Software Engineer (due out in March), and my cohost for O’Reilly’s AI Codecon. Every time I talk with him I come away feeling like I have a better grip on what’s real and what’s noise. Our recent conversation on Live with Tim O’Reilly was no exception.

Here are some of the things we talked about.

The hard problem is coordination, not generation

Addy pointed out that there’s a spectrum in how people are working with AI agents right now. On one end you have solo founders running hundreds or thousands of agents, sometimes without even reviewing the code. On the other end you have enterprise teams with quality gates, reliability requirements, and long-term maintenance to think about.

Addy’s take is that for most businesses, “the real frontier is not necessarily having hundreds of agents for a task just for its own sake. It’s about orchestrating a modest set of agents that solve real problems while maintaining control and traceability.” He pointed out that frameworks like Google’s Agent Development Kit now support both deterministic workflow agents and dynamic LLM agents in the same system, so you get to choose when you need predictability and when you need flexibility.

The ecosystem is developing fast. A2A (the agent-to-agent protocol Google contributed to the Linux Foundation) handles agent-to-agent communication while MCP handles agent-to-tool calls. Together they start to look like the TCP/IP of the agent era. But Addy was clear-eyed about where things stand: “Almost nobody’s figured out how to make everything work together as smoothly as possible. We’re getting as close to that as we can. And that’s the actual hard problem here. Not generation, but coordination.”

The “Something Big Is Happening” debate

In response to one of the audience questions, we spent some time on Matt Shumer’s viral essay arguing that the current moment in AI is like just before the COVID-19 epidemic hit. Those in the know were sounding the alarm, but most people weren’t hearing it.

Addy’s take was that “it felt a little bit like somebody who hadn’t been following along, just finally getting around to trying out the latest models and tools and having an epiphany moment.” He thinks the piece lacked grounding in data and didn’t do a great job distinguishing between what AI can do for prototypes and what it can do in production. As Addy put it, “Yes, the models are getting better, the harnesses are getting better, the tools are getting better. I can do more with AI these days than I could a year ago. All of that is true. But to say that all kinds of technical work can now be done with near perfection, I wouldn’t personally agree with that statement.”

I agree with Addy, but I also know how it feels when you see the future crashing in and no one is paying attention. At O’Reilly, we started working with the web when there were only 200 websites. In 1993, we built GNN, the first web portal, and the web’s first advertising. In 1994, we did the first large-scale market research on the potential of advertising as the web’s future business model. We went around lobbying phone companies to adopt the web and (a few years later) for bookstores to pay attention to the rise of Amazon, and nobody listened. I’m a big believer in “something is happening” moments. But I’m also very aware that it always takes longer than it appears.

Both things can be true. The direction and magnitude of this shift are real. The models keep getting better. The harnesses keep getting better. But we still have to figure out new kinds of businesses and new kinds of workflows. AI won’t be a tsunami that wipes everything away overnight.

Addy and I will be cohosting the O’Reilly AI Codecon: Software Craftsmanship in the Age of AI on March 26, where we’ll go much deeper on orchestration, agent coordination, and the new skills developers need. We’d love to see you there. Sign up for free here.

And if you’re interested in presenting at AI Codecon, our CFP is open through this Friday, February 20. Check out what we’re looking for and submit your proposal here.

Feeling productive vs. being productive

There was a great line from a post by Will Manidis called “Tool Shaped Objects” that I shared during our conversation: “The market for feeling productive is orders of magnitude larger than the market for being productive.” The essay is about things that feel amazing to build and use but aren’t necessarily doing the work that needs to be done.

Addy picked up on this immediately. “There is a difference between feeling busy and being productive,” he said. “You can have 100 agents working in the background and feel like you’re being productive. And then someone asks, What did you get built? How much money is it making you?”

This isn’t to dismiss anyone who’s genuinely productive running lots of agents. Some people are. But a healthy skepticism about your own productivity is worth maintaining, especially when the tools make it so easy to feel like you’re moving fast.

Planning is the new coding

Addy talked about how the balance of his time on a task has shifted significantly. “I might spend 30, 40% of the time a task takes just to actually write out what exactly is it that I want,” he said. What are the constraints? What are the success criteria? What’s the architecture? What libraries and UI components should be used?

All of that work to get clarity before you start code generation leads to much-higher-quality outcomes from AI. As Addy put it, “LLMs are very good at grounding things in the lowest common denominator. If there are patterns in the training data that are popular, they’re going to use those unless you tell them otherwise.” If your team has established best practices, codify them in Markdown files or MCP tools so the agent can use them.

I connected the planning phase to something larger about taste. Think about Steve Jobs. He wasn’t a coder. He was a master of knowing what good looked like and driving those who worked with him to achieve it. In this new world, that skill matters enormously. You’re going to be like Jobs telling his engineers “no, no, not that” and giving them a vision of what’s beautiful and powerful. Except now some of those engineers are agents. So management skill, communication skill, and taste are becoming core technical competencies.

Code review is getting harder

One thing Addy flagged that doesn’t get enough attention: “Increasingly teams feel like they’re being thrashed with all of these PRs that are AI generated. People don’t necessarily understand everything that’s in there. And you have to balance increased velocity expectations with ‘What is a quality bar?’ because someone’s going to have to maintain this.”

Knowing your quality bar matters. What are the cases where you’re comfortable merging an AI-generated change? Maybe it’s small and well-compartmentalized and has solid test coverage. And what are the cases where you absolutely need deep human review? Getting clear on that distinction is one of the most practical things a team can do right now.

Yes, young people should still go into software

We got a question about whether students should still pursue software engineering. Addy’s answer was emphatic: “There has never been a better time to get into software engineering if you are someone that is comfortable with learning. You do not necessarily have the burden of decades of knowing how things have historically been built. You can approach this with a very fresh set of eyes.” New entrants can go agent first. They can get deep into orchestration patterns and model trade-offs without having to unlearn old habits. And that’s a real advantage when interviewing at companies that need people who already know how to work this way.

The more important point is that in the early days of a new technology, people basically try to make the old things over again. The really big opportunities come when we figure out what was previously impossible that we can now do. If AI is as powerful as it appears to be, the opportunity isn’t to make companies more efficient at the same old work. It’s to solve entirely new problems and build entirely new kinds of products.

I’m 71 years old and 45 years into this industry, and this is the most excited I’ve ever been. More than the early web, more than open source. The future is being reinvented, and the people who start using these tools now get to be part of inventing it.

The token cost question

Addy had a funny and honest admission: “There were weeks when I would look at my bill for how much I was using in tokens and just be shocked. I don’t know that the productivity gains were actually worthwhile.”

His advice: experiment. Get a sense of what your typical tasks cost with multiple agents. Extrapolate. Ask yourself whether you’d still find it valuable at that price. Some people spend hundreds or even thousands a month on tokens and feel it’s worthwhile because the alternative was hiring a contractor. Others are spending that much and mostly feeling busy. As Addy said, “Don’t feel like you have to be spending a huge amount of money to not miss out on productivity wins.”

I’d add that we’re in a period where these costs are massively subsidized. The model companies are covering inference costs to get you locked in. Take advantage of that while it lasts. But also recognize that a lot of efficiency work is yet to be done. Just as JavaScript frameworks replaced everyone hand-coding UIs, we’ll get frameworks and tools that make agent workflows much more token-efficient than they are today.

2028 predictions are already here

One of the most striking things Addy shared was that a group in the AI coding community that he is part of had put together predictions for what software engineering would look like by 2028. “We recently revisited that list, and I was kind of shocked to discover that almost everything on that list is already possible today,” he said. “But how quickly the rest of the ecosystem adopts these things is on a longer trajectory than what is possible.”

That gap between capability and adoption is where most of the interesting work will happen over the next few years. The technology is running ahead of our ability to absorb it. Figuring out how to close that gap, in your team, your company, and your own practice, is the real job right now.

Agents writing code for agents

Near the end we answered another great audience question: Will agents eventually produce source code that’s optimized for other agents to read, not humans? Addy said yes. There are already platform teams having conversations about whether to build for an agent-first world where human readability becomes a secondary concern.

I have a historical parallel for this. I wrote the manual for the first C compiler on the Mac, and I worked closely with the developer who was hand-tuning the compiler output at the machine code level. That was about 30 years ago. We stopped doing that. And I’m quite confident there will be a similar moment with AI-generated code where humans mostly just let it go and trust the output. There will be special cases where people dive in for absolute performance or correctness. But they’ll be rare.

That transition won’t happen overnight. But the direction seems pretty clear. You can help to invent the future now, or spend time later trying to catch up with those who do.

This conversation was part of my ongoing series of discussions with innovators, Live with Tim O’Reilly. You can explore past episodes on the O’Reilly learning platform.

For most of the history of software engineering, we’ve built systems around a simple and comforting assumption: Given the same input, a program will produce the same output. When something went wrong, it was usually because of a bug, a misconfiguration, or a dependency that wasn’t behaving as advertised. Our tools, testing strategies, and even our mental models evolved around that expectation of determinism.

AI quietly breaks that assumption.

As large language models and AI services make their way into production systems, they often arrive through familiar shapes. There’s an API endpoint, a request payload, and a response body. Latency, retries, and timeouts all look manageable. From an architectural distance, it feels natural to treat these systems like libraries or external services.

In practice, that familiarity is misleading. AI systems behave less like deterministic components and more like nondeterministic collaborators. The same prompt can produce different outputs, small changes in context can lead to disproportionate shifts in results, and even retries can change behavior in ways that are difficult to reason about. These characteristics aren’t bugs; they’re inherent to how these systems work. The real problem is that our architectures often pretend otherwise. Instead of asking how to integrate AI as just another dependency, we need to ask how to design systems around components that do not guarantee stable outputs. Framing AI as a nondeterministic dependency turns out to be far more useful than treating it like a smarter API.

One of the first places where this mismatch becomes visible is retries. In deterministic systems, retries are usually safe. If a request fails due to a transient issue, retrying increases the chance of success without changing the outcome. With AI systems, retries don’t simply repeat the same computation. They generate new outputs. A retry might fix a problem, but it can just as easily introduce a different one. In some cases, retries quietly amplify failure rather than mitigate it, all while appearing to succeed.

Testing reveals a similar breakdown in assumptions. Our existing testing strategies depend on repeatability. Unit tests validate exact outputs. Integration tests verify known behaviors. With AI in the loop, those strategies quickly lose their effectiveness. You can test that a response is syntactically valid or conforms to certain constraints, but asserting that it is “correct” becomes far more subjective. Matters get even more complicated as models evolve over time. A test that passed yesterday may fail tomorrow without any code changes, leaving teams unsure whether the system regressed or simply changed.

Observability introduces an even subtler challenge. Traditional monitoring excels at detecting loud failures. Error rates spike. Latency increases. Requests fail. AI-related failures are often quieter. The system responds. Downstream services continue. Dashboards stay green. Yet the output is incomplete, misleading, or subtly wrong in context. These “acceptable but wrong” outcomes are far more damaging than outright errors because they erode trust gradually and are difficult to detect automatically.

Once teams accept nondeterminism as a first-class concern, design priorities begin to shift. Instead of trying to eliminate variability, the focus moves toward containing it. That often means isolating AI-driven functionality behind clear boundaries, limiting where AI outputs can influence critical logic, and introducing explicit validation or review points where ambiguity matters. The goal isn’t to force deterministic behavior from an inherently probabilistic system but to prevent that variability from leaking into parts of the system that aren’t designed to handle it.

This shift also changes how we think about correctness. Rather than asking whether an output is correct, teams often need to ask whether it is acceptable for a given context. That reframing can be uncomfortable, especially for engineers accustomed to precise specifications, but it reflects reality more accurately. Acceptability can be constrained, measured, and improved over time, even if it can’t be perfectly guaranteed.

Observability needs to evolve alongside this shift. Infrastructure-level metrics are still necessary, but they’re no longer sufficient. Teams need visibility into outputs themselves: how they change over time, how they vary across contexts, and how those variations correlate with downstream outcomes. This doesn’t mean logging everything, but it does mean designing signals that surface drift before users notice it. Qualitative degradation often appears long before traditional alerts fire, if anyone is paying attention.

One of the hardest lessons teams learn is that AI systems don’t offer guarantees in the way traditional software does. What they offer instead is probability. In response, successful systems rely less on guarantees and more on guardrails. Guardrails constrain behavior, limit blast radius, and provide escape hatches when things go wrong. They don’t promise correctness, but they make failure survivable. Fallback paths, conservative defaults, and human-in-the-loop workflows become architectural features rather than afterthoughts.

For architects and senior engineers, this represents a subtle but important shift in responsibility. The challenge isn’t choosing the right model or crafting the perfect prompt. It’s reshaping expectations, both within engineering teams and across the organization. That often means pushing back on the idea that AI can simply replace deterministic logic, and being explicit about where uncertainty exists and how the system handles it.

If I were starting again today, there are a few things I would do earlier. I would document explicitly where nondeterminism exists in the system and how it’s managed rather than letting it remain implicit. I would invest sooner in output-focused observability, even if the signals felt imperfect at first. And I would spend more time helping teams unlearn assumptions that no longer hold, because the hardest bugs to fix are the ones rooted in outdated mental models.

AI isn’t just another dependency. It challenges some of the most deeply ingrained assumptions in software engineering. Treating it as a nondeterministic dependency doesn’t solve every problem, but it provides a far more honest foundation for system design. It encourages architectures that expect variation, tolerate ambiguity, and fail gracefully.

That shift in thinking may be the most important architectural change AI brings, not because the technology is magical but because it forces us to confront the limits of determinism we’ve relied on for decades.

Over the past six months, I’ve watched the same pattern repeat across enterprise AI teams. A2A and ACP light up the room during architecture reviews—the protocols are elegant, the demos impressive. Three weeks into production, someone asks: “Wait, which agent authorized that $50,000 vendor payment at 2 am?“ The excitement shifts to concern.

Here’s the paradox: Agent2Agent (A2A) and the Agent Communication Protocol (ACP) are so effective at eliminating integration friction that they’ve removed the natural “brakes“ that used to force governance conversations. We’ve solved the plumbing problem brilliantly. In doing so, we’ve created a new class of integration debt—one where organizations borrow speed today at the cost of accountability tomorrow.

The technical protocols are solid. The organizational protocols are missing. We’re rapidly moving from the “Can these systems connect?“ phase to the “Who authorized this agent to liquidate a position at 3 am?“ phase. In practice, that creates a governance gap: Our ability to connect agents is outpacing our ability to control what they commit us to.

To see why that shift is happening so fast, it helps to look at how the underlying “agent stack“ is evolving. We’re seeing the emergence of a three-tier structure that quietly replaces traditional API-led connectivity:

This stack makes multi-agent workflows a configuration problem instead of a custom engineering project. That is exactly why the risk surface is expanding faster than most CISOs realize.

Think of it this way: A2A is the handshake between agents (who talks to whom, about what tasks). ACP is the briefing document they exchange (what context, history, and goals move in that conversation). MCP is the toolbox each agent has access to locally. Once you see the stack this way, you also see the next problem: We’ve solved API sprawl and quietly replaced it with something harder to see—agent sprawl, and with it, a widening governance gap.

Most enterprises already struggle to govern hundreds of SaaS applications. One analysis puts the average at more than 370 SaaS apps per organization. Agent protocols do not reduce this complexity; they route around it. In the API era, humans filed tickets to trigger system actions. In the A2A era, agents use “Agent Cards“ to discover each other and negotiate on top of those systems. ACP allows these agents to trade rich context—meaning a conversation starting in customer support can flow into fulfillment and partner logistics with zero human handoffs. What used to be API sprawl is becoming dozens of semiautonomous processes acting on behalf of your company across infrastructure you do not fully control. The friction of manual integration used to act as a natural brake on risk; A2A has removed that brake.

That governance gap doesn’t usually show up as a single catastrophic failure. It shows up as a series of small, confusing incidents where everything looks “green“ in the dashboards but the business outcome is wrong. The protocol documentation focuses on encryption and handshakes but ignores the emergent failure modes of autonomous collaboration. These are not bugs in the protocols; they’re signs that the surrounding architecture has not caught up with the level of autonomy the protocols enable.

Policy drift: A refund policy encoded in a service agent may technically interoperate with a partner’s collections agent via A2A, but their business logic may be diametrically opposed. When something goes wrong, nobody owns the end-to-end behavior.

Context oversharing: A team might expand an ACP schema to include “User Sentiment“ for better personalization, unaware that this data now propagates to every downstream third-party agent in the chain. What started as local enrichment becomes distributed exposure.

The determinism trap: Unlike REST APIs, agents are nondeterministic. An agent’s refund policy logic might change when its underlying model is updated from GPT-4 to GPT-4.5, even though the A2A Agent Card declares identical capabilities. The workflow “works“—until it doesn’t, and there’s no version trace to debug. This creates what I call “ghost breaks“: failures that don’t show up in traditional observability because the interface contract looks unchanged.

Taken together, these aren’t edge cases. They’re what happens when we give agents more autonomy without upgrading the rules of engagement between them. These failure modes have a common root cause: The technical capability to collaborate across agents has outrun the organization’s ability to say where that collaboration is appropriate, and under what constraints.

That’s why we need something on top of the protocols themselves: an explicit “Agent Treaty“ layer. If the protocol is the language, the treaty is the constitution. Governance must move from “side documentation“ to “policy as code.“

Want Radar delivered straight to your inbox? Join us on Substack. Sign up here.

Traditional governance treats policy violations as failures to prevent. An antifragile approach treats them as signals to exploit. When an agent makes a commitment that violates a business constraint, the system should capture that event, trace the causal chain, and feed it back into both the agent’s training and the treaty ruleset. Over time, the governance layer gets smarter, not just stricter.

Define treaty-level constraints: Don’t just authorize a connection; authorize a scope. Which ACP fields is an agent allowed to share? Which A2A operations are “read only“ versus “legally binding“? Which categories of decisions require human escalation?

Version the behavior, not just the schema: Treat Agent Cards as first-class product surfaces. If the underlying model changes, the version must bump, triggering a rereview of the treaty. This is not bureaucratic overhead—it’s the only way to maintain accountability in a system where autonomous agents make commitments on behalf of your organization.

Cross-organizational traceability: We need observability traces that don’t just show latency but show intent: Which agent made this commitment, under which policy? And who is the human owner? This is particularly critical when workflows span organizational boundaries and partner ecosystems.

Designing that treaty layer isn’t just a tooling problem. It changes who needs to be in the room and how they think about the system. The hardest constraint isn’t the code; it’s the people. We’re entering a world where engineers must reason about multi-agent game theory and policy interactions, not just SDK integration. Risk teams must audit “machine-to-machine commitments“ that may never be rendered in human language. Product managers must own agent ecosystems where a change in one agent’s reward function or context schema shifts behavior across an entire partner network. Compliance and audit functions need new tools and mental models to review autonomous workflows that execute at machine speed. In many organizations, those skills sit in different silos, and A2A/ACP adoption is proceeding faster than the cross-functional structures needed to manage them.

All of this might sound abstract until you look at where enterprises are in their adoption curve. Three converging trends are making this urgent: Protocol maturity means A2A, ACP, and MCP specifications have stabilized enough that enterprises are moving beyond pilots to production deployments. Multi-agent orchestration is shifting from single agents to agent ecosystems and workflows that span teams, departments, and organizations. And silent autonomy is blurring the line between “tool assistance“ and “autonomous decision-making“—often without explicit organizational acknowledgment. We’re moving from integration (making things talk) to orchestration (making things act), but our monitoring tools still only measure the talk. The next 18 months will determine whether enterprises get ahead of this or we see a wave of high-profile failures that force retroactive governance.

The risk is not that A2A and ACP are unsafe; it’s that they are too effective. For teams piloting these protocols, stop focusing on the “happy path“ of connectivity. Instead, pick one multi-agent workflow and instrument it as a critical product:

Map the context flow: Every ACP field must have a “purpose limitation“ tag. Document which agents see which fields, and which business or regulatory requirements justify that visibility. This isn’t an inventory exercise; it’s a way to surface hidden data dependencies.

Audit the commitments: Identify every A2A interaction that represents a financial or legal commitment—especially ones that don’t route through human approval. Ask, “If this agent’s behavior changed overnight, who would notice? Who is accountable?“

Code the treaty: Prototype a “gatekeeper“ agent that enforces business constraints on top of the raw protocol traffic. This isn’t about blocking agents; it’s about making policy visible and enforceable at runtime. Start minimal: One policy, one workflow, one success metric.

Instrument for learning: Capture which agents collaborate, which policies they invoke, and which contexts they share. Treat this as telemetry, not just audit logs. Feed patterns back into governance reviews quarterly.

If this works, you now have a repeatable pattern for scaling agent deployments without sacrificing accountability. If it breaks, you’ve learned something critical about your architecture before it breaks in production. If you can get one workflow to behave this way—governed, observable, and learn-as-you-go—you have a template for the rest of your agent ecosystem.

If the last decade was about treating APIs as products, the next one will be about treating autonomous workflows as policies encoded in traffic between agents. The protocols are ready. Your org chart is not. The bridge between the two is the Agent Treaty—start building it before your agents start signing deals without you. The good news: You don’t need to redesign your entire organization. You need to add one critical layer—the Agent Treaty—that makes policy machine-enforceable, observable, and learnable. You need engineers who think about composition and game theory, not just connection. And you need to treat agent deployments as products, not infrastructure.

The sooner you start, the sooner that governance gap closes.