For most of the history of software engineering, we’ve built systems around a simple and comforting assumption: Given the same input, a program will produce the same output. When something went wrong, it was usually because of a bug, a misconfiguration, or a dependency that wasn’t behaving as advertised. Our tools, testing strategies, and even our mental models evolved around that expectation of determinism.

AI quietly breaks that assumption.

As large language models and AI services make their way into production systems, they often arrive through familiar shapes. There’s an API endpoint, a request payload, and a response body. Latency, retries, and timeouts all look manageable. From an architectural distance, it feels natural to treat these systems like libraries or external services.

In practice, that familiarity is misleading. AI systems behave less like deterministic components and more like nondeterministic collaborators. The same prompt can produce different outputs, small changes in context can lead to disproportionate shifts in results, and even retries can change behavior in ways that are difficult to reason about. These characteristics aren’t bugs; they’re inherent to how these systems work. The real problem is that our architectures often pretend otherwise. Instead of asking how to integrate AI as just another dependency, we need to ask how to design systems around components that do not guarantee stable outputs. Framing AI as a nondeterministic dependency turns out to be far more useful than treating it like a smarter API.

One of the first places where this mismatch becomes visible is retries. In deterministic systems, retries are usually safe. If a request fails due to a transient issue, retrying increases the chance of success without changing the outcome. With AI systems, retries don’t simply repeat the same computation. They generate new outputs. A retry might fix a problem, but it can just as easily introduce a different one. In some cases, retries quietly amplify failure rather than mitigate it, all while appearing to succeed.

Testing reveals a similar breakdown in assumptions. Our existing testing strategies depend on repeatability. Unit tests validate exact outputs. Integration tests verify known behaviors. With AI in the loop, those strategies quickly lose their effectiveness. You can test that a response is syntactically valid or conforms to certain constraints, but asserting that it is “correct” becomes far more subjective. Matters get even more complicated as models evolve over time. A test that passed yesterday may fail tomorrow without any code changes, leaving teams unsure whether the system regressed or simply changed.

Observability introduces an even subtler challenge. Traditional monitoring excels at detecting loud failures. Error rates spike. Latency increases. Requests fail. AI-related failures are often quieter. The system responds. Downstream services continue. Dashboards stay green. Yet the output is incomplete, misleading, or subtly wrong in context. These “acceptable but wrong” outcomes are far more damaging than outright errors because they erode trust gradually and are difficult to detect automatically.

Once teams accept nondeterminism as a first-class concern, design priorities begin to shift. Instead of trying to eliminate variability, the focus moves toward containing it. That often means isolating AI-driven functionality behind clear boundaries, limiting where AI outputs can influence critical logic, and introducing explicit validation or review points where ambiguity matters. The goal isn’t to force deterministic behavior from an inherently probabilistic system but to prevent that variability from leaking into parts of the system that aren’t designed to handle it.

This shift also changes how we think about correctness. Rather than asking whether an output is correct, teams often need to ask whether it is acceptable for a given context. That reframing can be uncomfortable, especially for engineers accustomed to precise specifications, but it reflects reality more accurately. Acceptability can be constrained, measured, and improved over time, even if it can’t be perfectly guaranteed.

Observability needs to evolve alongside this shift. Infrastructure-level metrics are still necessary, but they’re no longer sufficient. Teams need visibility into outputs themselves: how they change over time, how they vary across contexts, and how those variations correlate with downstream outcomes. This doesn’t mean logging everything, but it does mean designing signals that surface drift before users notice it. Qualitative degradation often appears long before traditional alerts fire, if anyone is paying attention.

One of the hardest lessons teams learn is that AI systems don’t offer guarantees in the way traditional software does. What they offer instead is probability. In response, successful systems rely less on guarantees and more on guardrails. Guardrails constrain behavior, limit blast radius, and provide escape hatches when things go wrong. They don’t promise correctness, but they make failure survivable. Fallback paths, conservative defaults, and human-in-the-loop workflows become architectural features rather than afterthoughts.

For architects and senior engineers, this represents a subtle but important shift in responsibility. The challenge isn’t choosing the right model or crafting the perfect prompt. It’s reshaping expectations, both within engineering teams and across the organization. That often means pushing back on the idea that AI can simply replace deterministic logic, and being explicit about where uncertainty exists and how the system handles it.

If I were starting again today, there are a few things I would do earlier. I would document explicitly where nondeterminism exists in the system and how it’s managed rather than letting it remain implicit. I would invest sooner in output-focused observability, even if the signals felt imperfect at first. And I would spend more time helping teams unlearn assumptions that no longer hold, because the hardest bugs to fix are the ones rooted in outdated mental models.

AI isn’t just another dependency. It challenges some of the most deeply ingrained assumptions in software engineering. Treating it as a nondeterministic dependency doesn’t solve every problem, but it provides a far more honest foundation for system design. It encourages architectures that expect variation, tolerate ambiguity, and fail gracefully.

That shift in thinking may be the most important architectural change AI brings, not because the technology is magical but because it forces us to confront the limits of determinism we’ve relied on for decades.

Over the past six months, I’ve watched the same pattern repeat across enterprise AI teams. A2A and ACP light up the room during architecture reviews—the protocols are elegant, the demos impressive. Three weeks into production, someone asks: “Wait, which agent authorized that $50,000 vendor payment at 2 am?“ The excitement shifts to concern.

Here’s the paradox: Agent2Agent (A2A) and the Agent Communication Protocol (ACP) are so effective at eliminating integration friction that they’ve removed the natural “brakes“ that used to force governance conversations. We’ve solved the plumbing problem brilliantly. In doing so, we’ve created a new class of integration debt—one where organizations borrow speed today at the cost of accountability tomorrow.

The technical protocols are solid. The organizational protocols are missing. We’re rapidly moving from the “Can these systems connect?“ phase to the “Who authorized this agent to liquidate a position at 3 am?“ phase. In practice, that creates a governance gap: Our ability to connect agents is outpacing our ability to control what they commit us to.

To see why that shift is happening so fast, it helps to look at how the underlying “agent stack“ is evolving. We’re seeing the emergence of a three-tier structure that quietly replaces traditional API-led connectivity:

This stack makes multi-agent workflows a configuration problem instead of a custom engineering project. That is exactly why the risk surface is expanding faster than most CISOs realize.

Think of it this way: A2A is the handshake between agents (who talks to whom, about what tasks). ACP is the briefing document they exchange (what context, history, and goals move in that conversation). MCP is the toolbox each agent has access to locally. Once you see the stack this way, you also see the next problem: We’ve solved API sprawl and quietly replaced it with something harder to see—agent sprawl, and with it, a widening governance gap.

Most enterprises already struggle to govern hundreds of SaaS applications. One analysis puts the average at more than 370 SaaS apps per organization. Agent protocols do not reduce this complexity; they route around it. In the API era, humans filed tickets to trigger system actions. In the A2A era, agents use “Agent Cards“ to discover each other and negotiate on top of those systems. ACP allows these agents to trade rich context—meaning a conversation starting in customer support can flow into fulfillment and partner logistics with zero human handoffs. What used to be API sprawl is becoming dozens of semiautonomous processes acting on behalf of your company across infrastructure you do not fully control. The friction of manual integration used to act as a natural brake on risk; A2A has removed that brake.

That governance gap doesn’t usually show up as a single catastrophic failure. It shows up as a series of small, confusing incidents where everything looks “green“ in the dashboards but the business outcome is wrong. The protocol documentation focuses on encryption and handshakes but ignores the emergent failure modes of autonomous collaboration. These are not bugs in the protocols; they’re signs that the surrounding architecture has not caught up with the level of autonomy the protocols enable.

Policy drift: A refund policy encoded in a service agent may technically interoperate with a partner’s collections agent via A2A, but their business logic may be diametrically opposed. When something goes wrong, nobody owns the end-to-end behavior.

Context oversharing: A team might expand an ACP schema to include “User Sentiment“ for better personalization, unaware that this data now propagates to every downstream third-party agent in the chain. What started as local enrichment becomes distributed exposure.

The determinism trap: Unlike REST APIs, agents are nondeterministic. An agent’s refund policy logic might change when its underlying model is updated from GPT-4 to GPT-4.5, even though the A2A Agent Card declares identical capabilities. The workflow “works“—until it doesn’t, and there’s no version trace to debug. This creates what I call “ghost breaks“: failures that don’t show up in traditional observability because the interface contract looks unchanged.

Taken together, these aren’t edge cases. They’re what happens when we give agents more autonomy without upgrading the rules of engagement between them. These failure modes have a common root cause: The technical capability to collaborate across agents has outrun the organization’s ability to say where that collaboration is appropriate, and under what constraints.

That’s why we need something on top of the protocols themselves: an explicit “Agent Treaty“ layer. If the protocol is the language, the treaty is the constitution. Governance must move from “side documentation“ to “policy as code.“

Want Radar delivered straight to your inbox? Join us on Substack. Sign up here.

Traditional governance treats policy violations as failures to prevent. An antifragile approach treats them as signals to exploit. When an agent makes a commitment that violates a business constraint, the system should capture that event, trace the causal chain, and feed it back into both the agent’s training and the treaty ruleset. Over time, the governance layer gets smarter, not just stricter.

Define treaty-level constraints: Don’t just authorize a connection; authorize a scope. Which ACP fields is an agent allowed to share? Which A2A operations are “read only“ versus “legally binding“? Which categories of decisions require human escalation?

Version the behavior, not just the schema: Treat Agent Cards as first-class product surfaces. If the underlying model changes, the version must bump, triggering a rereview of the treaty. This is not bureaucratic overhead—it’s the only way to maintain accountability in a system where autonomous agents make commitments on behalf of your organization.

Cross-organizational traceability: We need observability traces that don’t just show latency but show intent: Which agent made this commitment, under which policy? And who is the human owner? This is particularly critical when workflows span organizational boundaries and partner ecosystems.

Designing that treaty layer isn’t just a tooling problem. It changes who needs to be in the room and how they think about the system. The hardest constraint isn’t the code; it’s the people. We’re entering a world where engineers must reason about multi-agent game theory and policy interactions, not just SDK integration. Risk teams must audit “machine-to-machine commitments“ that may never be rendered in human language. Product managers must own agent ecosystems where a change in one agent’s reward function or context schema shifts behavior across an entire partner network. Compliance and audit functions need new tools and mental models to review autonomous workflows that execute at machine speed. In many organizations, those skills sit in different silos, and A2A/ACP adoption is proceeding faster than the cross-functional structures needed to manage them.

All of this might sound abstract until you look at where enterprises are in their adoption curve. Three converging trends are making this urgent: Protocol maturity means A2A, ACP, and MCP specifications have stabilized enough that enterprises are moving beyond pilots to production deployments. Multi-agent orchestration is shifting from single agents to agent ecosystems and workflows that span teams, departments, and organizations. And silent autonomy is blurring the line between “tool assistance“ and “autonomous decision-making“—often without explicit organizational acknowledgment. We’re moving from integration (making things talk) to orchestration (making things act), but our monitoring tools still only measure the talk. The next 18 months will determine whether enterprises get ahead of this or we see a wave of high-profile failures that force retroactive governance.

The risk is not that A2A and ACP are unsafe; it’s that they are too effective. For teams piloting these protocols, stop focusing on the “happy path“ of connectivity. Instead, pick one multi-agent workflow and instrument it as a critical product:

Map the context flow: Every ACP field must have a “purpose limitation“ tag. Document which agents see which fields, and which business or regulatory requirements justify that visibility. This isn’t an inventory exercise; it’s a way to surface hidden data dependencies.

Audit the commitments: Identify every A2A interaction that represents a financial or legal commitment—especially ones that don’t route through human approval. Ask, “If this agent’s behavior changed overnight, who would notice? Who is accountable?“

Code the treaty: Prototype a “gatekeeper“ agent that enforces business constraints on top of the raw protocol traffic. This isn’t about blocking agents; it’s about making policy visible and enforceable at runtime. Start minimal: One policy, one workflow, one success metric.

Instrument for learning: Capture which agents collaborate, which policies they invoke, and which contexts they share. Treat this as telemetry, not just audit logs. Feed patterns back into governance reviews quarterly.

If this works, you now have a repeatable pattern for scaling agent deployments without sacrificing accountability. If it breaks, you’ve learned something critical about your architecture before it breaks in production. If you can get one workflow to behave this way—governed, observable, and learn-as-you-go—you have a template for the rest of your agent ecosystem.

If the last decade was about treating APIs as products, the next one will be about treating autonomous workflows as policies encoded in traffic between agents. The protocols are ready. Your org chart is not. The bridge between the two is the Agent Treaty—start building it before your agents start signing deals without you. The good news: You don’t need to redesign your entire organization. You need to add one critical layer—the Agent Treaty—that makes policy machine-enforceable, observable, and learnable. You need engineers who think about composition and game theory, not just connection. And you need to treat agent deployments as products, not infrastructure.

The sooner you start, the sooner that governance gap closes.

AI coding assistants have quickly moved from novelty to necessity, where up to 90% of software engineers use some kind of AI for coding. But a new paradigm is emerging in software development—one where engineers leverage fleets of autonomous coding agents. In this agentic future, the role of the software engineer is evolving from implementer to manager, or in other words, from coder to conductor and ultimately orchestrator.

Over time, developers will increasingly guide AI agents to build the right code and coordinate multiple agents working in concert. This write-up explores the distinction between conductors and orchestrators in AI-assisted coding, defines these roles, and examines how today’s cutting-edge tools embody each approach. Senior engineers may start to see the writing on the wall: Our jobs are shifting from “How do I code this?” to “How do I get the right code built?”—a subtle but profound change.

What’s the tl;dr of an orchestrator tool? It supports multi-agent workflows where you can run many agents in parallel without them interfering with each other. But let’s talk terminology first.

The Conductor: Guiding a Single AI Agent

In the context of AI coding, acting as a conductor means working closely with a single AI agent on a specific task, much like a conductor guiding a soloist through a performance.

The engineer remains in the loop at each step, dynamically steering the agent’s behavior, tweaking prompts, intervening when needed, and iterating in real time. This is the logical extension of the “AI pair programmer” model many developers are already familiar with. With conductor-style workflows, coding happens in a synchronous, interactive session between human and AI, typically in your IDE or CLI.

Key characteristics: A conductor keeps a tight feedback loop with one agent, verifying or modifying each suggestion, much as a driver navigates with a GPS. The AI helps write code, but the developer still performs many manual steps—creating branches, running tests, writing commit messages, etc.—and ultimately decides which suggestions to accept.

Crucially, most of this interaction is ephemeral: Once code is written and the session ends, the AI’s role is done and any context or decisions not captured in code may be lost. This mode is powerful for focused tasks and allows fine-grained control, but it doesn’t fully exploit what multiple AIs could do in parallel.

Modern tools as conductors

Several current AI coding tools exemplify the conductor pattern:

• Claude Code (Anthropic): Anthropic’s Claude model offers a coding assistant mode (accessible via a CLI tool or editor integration) where the developer converses with Claude to generate or modify code. For example, with the Claude Code CLI, you navigate your project in a shell and ask Claude to implement a function or refactor code, and it prints diffs or file updates for you to approve. You remain the conductor: You trigger each action and review the output immediately. While Claude Code has features to handle long-running tasks and tools, in the basic usage it’s essentially a smart codeveloper working step-by-step under human direction.
• Gemini CLI (Google): A command-line assistant powered by Google’s Gemini model, used for planning and coding with a very large context window. An engineer can prompt Gemini CLI to analyze a codebase or draft a solution plan, then iterate on results interactively. The human directs each step and Gemini responds within the CLI session. It’s a one-at-a-time collaborator, not running off to make code changes on its own (at least in this conductor mode).
• Cursor (editor AI assistant): The Cursor editor (a specialized AI-augmented IDE) can operate in an inline or chat mode where you ask it questions or to write a snippet, and it immediately performs those edits or gives answers within your coding session. Again, you guide it one request at a time. Cursor’s strength as a conductor is its deep context integration—it indexes your whole codebase so the AI can answer questions about any part of it. But the hallmark is that you, the developer, initiate and oversee each change in real time.
• VS Code, Cline, Roo Code (in-IDE chat): Similar to above, other coding agents also fall into this category. They suggest code or even multistep fixes, but always under continuous human guidance.

This conductor-style AI assistance has already boosted productivity significantly. It feels like having a junior engineer or pair programmer always by your side. However, it’s inherently one-agent-at-a-time and synchronous. To truly leverage AI at scale, we need to go beyond being a single-agent conductor. This is where the orchestrator role comes in.

The Orchestrator: Managing a Fleet of Agents

If a conductor works with one AI “musician,” an orchestrator oversees the entire symphony of multiple AI agents working in parallel on different parts of a project. The orchestrator sets high-level goals, defines tasks, and lets a team of autonomous coding agents independently carry out the implementation details.

Instead of micromanaging every function or bug fix, the human focuses on coordination, quality control, and integration of the agents’ outputs. In practical terms, this often means an engineer can assign tasks to AI agents (e.g., via issues or prompts) and have those agents asynchronously produce code changes—often as ready-to-review pull requests. The engineer’s job becomes reviewing, giving feedback, and merging the results rather than writing all the code personally.

This asynchronous, parallel workflow is a fundamental shift. It moves AI assistance from the foreground to the background. While you attend to higher-level design or other work, your “AI team” is coding in the background. When they’re done, they hand you completed work (with tests, docs, etc.) for review. It’s akin to being a project tech lead delegating tasks to multiple devs and later reviewing their pull requests, except the “devs” are AI agents.

Modern tools as orchestrators

Over just the past year, several tools have emerged that embody this orchestrator paradigm:

• GitHub Copilot coding agent (Microsoft): This upgrade to Copilot transforms it from an in-editor assistant into an autonomous background developer. (I cover it in this video.) You can assign a GitHub issue to Copilot’s agent or invoke it via the VS Code agents panel, telling it (for example) “Implement feature X” or “Fix bug Y.” Copilot then spins up an ephemeral dev environment via GitHub Actions, checks out your repo, creates a new branch, and begins coding. It can run tests, linters, even spin up the app if needed, all without human babysitting. When finished, it opens a pull request with the changes, complete with a description and meaningful commit messages. It then asks for your review.
  
  You, the human orchestrator, review the PR (perhaps using Copilot’s AI-assisted code review to get an initial analysis). If changes are needed, you can leave comments like “@copilot please update the unit tests for edge case Z,” and the agent will iterate on the PR. This is asynchronous, autonomous code generation in action. Notably, Copilot automates the tedious bookkeeping—branch creation, committing, opening PRs, etc.—which used to cost developers time. All the grunt work around writing code (aside from the design itself) is handled, allowing developers to focus on reviewing and guiding at a high level. GitHub’s agent effectively lets one engineer supervise many “AI juniors” working in parallel across different issues (and you can even create multiple specialized agents for different task types).

• Jules, Google’s coding agent: Jules is an autonomous coding agent. Jules is “not a copilot, not a code-completion sidekick, but an autonomous agent that reads your code, understands your intent, and gets to work.” Integrated with Google Cloud and GitHub, Jules lets you connect a repository and then ask it to perform tasks much as you would a developer on your team. Under the hood, Jules clones your entire codebase into a secure cloud VM and analyzes it with a powerful model. You might tell Jules “Add user authentication to our app” or “Upgrade this project to the latest Node.js and fix any compatibility issues.” It will formulate a plan, present it to you for approval, and once you approve, execute the changes asynchronously. It makes commits on a new branch and can even open a pull request for you to merge. Jules handles writing new code, updating tests, bumping dependencies, etc., all while you could be doing something else.
  
  Crucially, Jules provides transparency and control: It shows you its proposed plan and reasoning before making changes, and allows you to intervene or modify instructions at any point (a feature Google calls “user steerability”). This is akin to giving an AI intern the spec and watching over their shoulder less frequently—you trust them to get it mostly right, but you still verify the final diff. Jules also boasts unique touches like audio changelogs (it generates spoken summaries of code changes) and the ability to run multiple tasks concurrently in the cloud. In short, Google’s Jules demonstrates the orchestrator model: You define the task, Jules does the heavy lifting asynchronously, and you oversee the result.

• OpenAI Codex (cloud agent): OpenAI introduced a new cloud-based Codex agent to complement ChatGPT. This evolved Codex (different from the 2021 Codex model) is described as “a cloud-based software engineering agent that can work on many tasks in parallel.” It’s available as part of ChatGPT Plus/Pro under the name OpenAI Codex and via an npm CLI (npm i -g @openai/codex). With the Codex CLI or its VS Code/Cursor extensions, you can delegate tasks to OpenAI’s agent similar to Copilot or Jules. For instance, from your terminal you might say, “Hey Codex, implement dark mode for the settings page.” Codex then launches into your repository, edits the necessary files, perhaps runs your test suite, and when done, presents the diff for you to merge. It operates in an isolated sandbox for safety, running each task in a container with your repo and environment.
  
  Like others, OpenAI’s Codex agent integrates with developer workflows: You can even kick off tasks from a ChatGPT mobile app on your phone and get notified when the agent is done. OpenAI emphasizes seamless switching “between real-time collaboration and async delegation” with Codex. In practice, this means you have the flexibility to use it in conductor mode (pair-programming in your IDE) or orchestrator mode (hand off a background task to the cloud agent). Codex can also be invited into your Slack channels—teammates can assign tasks to @Codex in Slack, and it will pull context from the conversation and your repo to execute them. It’s a vision of ubiquitous AI assistance, where coding tasks can be delegated from anywhere. Early users report that Codex can autonomously identify and fix bugs, or generate significant features, given a well-scoped prompt. All of this again aligns with the orchestrator workflow: The human defines the goal; the AI agent autonomously delivers a solution.

• Anthropic Claude Code (for web): Anthropic has offered Claude as an AI chatbot for a while, and their Claude Code CLI has been a favorite for interactive coding. Anthropic took the next step by launching Claude Code for web, effectively a hosted version of their coding agent. Using Claude Code for web, you point it at your GitHub repo (with configurable sandbox permissions) and give it a task. The agent then runs in Anthropic’s managed container, just like the CLI version, but now you can trigger it from a web interface or even a mobile app. It queues up multiple prompts and steps, executes them, and when done, pushes a branch to your repo (and can open a PR). Essentially, Anthropic took their single-agent Claude Code and made it an orchestratable service in the cloud. They even provided a “teleport” feature to transfer the session to your local environment if you want to take over manually.
  
  The rationale for this web version aligns with orchestrator benefits: convenience and scale. You don’t need to run long jobs on your machine; Anthropic’s cloud handles the heavy lifting, with filesystem and network isolation for safety. Claude Code for web acknowledges that autonomy with safety is key—by sandboxing the agent, they reduce the need for constant permission prompts, letting the agent operate more freely (less babysitting by the user). In effect, Anthropic has made it easier to use Claude as an autonomous coding worker you launch on demand.

• Cursor background agents: tl;dr Cursor 2.0 has a multi-agent interface more focused around agents rather than files. Cursor 2 expands its background agents feature into a full-fledged orchestration layer for developers. Beyond serving as an interactive assistant, Cursor 2 lets you spawn autonomous background agents that operate asynchronously in a managed cloud workspace. When you delegate a task, Cursor 2’s agents now clone your GitHub repository, spin up an ephemeral environment, and check out an isolated branch where they execute work end-to-end. These agents can handle the entire development loop—from editing and running code to installing dependencies, executing tests, running builds, and even searching the web or referencing documentation to resolve issues. Once complete, they push commits and open a detailed pull request summarizing their work.
  
  Cursor 2 introduces multi-agent orchestration, allowing several background agents to run concurrently across different tasks—for instance, one refining UI components while another optimizes backend performance or fixes tests. Each agent’s activity is visible through a real-time dashboard that can be accessed from desktop or mobile, enabling you to monitor progress, issue follow-ups, or intervene manually if needed. This new system effectively treats each agent as part of an on-demand AI workforce, coordinated through the developer’s high-level intent. Cursor 2’s focus on parallel, asynchronous execution dramatically amplifies a single engineer’s throughput—fully realizing the orchestrator model where humans oversee a fleet of cooperative AI developers rather than a single assistant.

• Agent orchestration platforms: Beyond individual product offerings, there are also emerging platforms and open source projects aimed at orchestrating multiple agents. For instance, Conductor by Melty Labs (despite its name!) is actually an orchestration tool that lets you deploy and manage multiple Claude Code agents on your own machine in parallel. With Conductor, each agent gets its own isolated Git worktree to avoid conflicts, and you can see a dashboard of all agents (“who’s working on what”) and review their code as they progress. The idea is to make running a small swarm of coding agents as easy as running one. Similarly, Claude Squad is a popular open source terminal app that essentially multiplexes Anthropic’s Claude—it can spawn several Claude Code instances working concurrently in separate tmux panes, allowing you to give each a different task and thus code “10x faster” by parallelizing. These orchestration tools underscore the trend: Developers want to coordinate multiple AI coding agents and have them collaborate or divide work. Even Microsoft’s Azure AI services are enabling this: At Build 2025 they announced tools for developers to “orchestrate multiple specialized agents to handle complex tasks,” with SDKs supporting agent-to-agent communication so your fleet of agents can talk to each other and share context. All of this infrastructure is being built to support the orchestrator engineer, who might eventually oversee dozens of AI processes tackling different parts of the software development lifecycle.

I found Conductor to make the most sense to me. It was a perfect balance of talking to an agent and seeing my changes in a pane next to it. Its Github integration feels seamless; e.g. after merging PR, it immediately showed a task as “Merged” and provided an “Archive” button.
—Juriy Zaytsev, Staff SWE, LinkedIn

He also tried Magnet:

The idea of tying tasks to a Kanban board is interesting and makes sense. As such, Magnet feels very product-centric.

Conductor versus Orchestrator—Differences

Many engineers will continue to engage in conductor-style workflows (single agent, interactive) even as orchestrator patterns mature. The two modes will coexist.

It’s clear that “conductor” and “orchestrator” aren’t just fancy terms; they describe a genuine shift in how we work with AI.

• Scope of control: A conductor operates at the micro level, guiding one agent through a single task or a narrow problem. An orchestrator operates at the macro level, defining broader tasks and objectives for multiple agents or for a powerful single agent that can handle multistep projects. The conductor asks, “How do I solve this function or bug with the AI’s help?” The orchestrator asks, “What set of tasks can I delegate to AI agents today to move this project forward?”
• Degree of autonomy: In conductor mode, the AI’s autonomy is low—it waits for user prompts each step of the way. In orchestrator mode, we give the AI high autonomy—it might plan and execute dozens of steps internally (writing code, running tests, adjusting its approach) before needing human feedback. A GitHub Copilot agent or Jules will try to complete a feature from start to finish once assigned, whereas Copilot’s IDE suggestions only go line-by-line as you type.
• Synchronous vs asynchronous: Conductor interactions are typically synchronous—you prompt; AI responds within seconds; you immediately integrate or iterate. It’s a real-time loop. Orchestrator interactions are asynchronous—you might dispatch an agent and check back minutes or hours later when it’s done (somewhat like kicking off a long CI job). This means orchestrators must handle waiting, context-switching, and possibly managing multiple things concurrently, which is a different workflow rhythm for developers.
• Artifacts and traceability: A subtle but important difference: Orchestrator workflows produce persistent artifacts like branches, commits, and pull requests that are preserved in version control. The agent’s work is fully recorded (and often linked to an issue/ticket), which improves traceability and collaboration. With conductor-style (IDE chat, etc.), unless the developer manually commits intermediate changes, a lot of the AI’s involvement isn’t explicitly documented. In essence, orchestrators leave a paper trail (or rather a Git trail) that others on the team can see or even trigger themselves. This can help bring AI into team processes more naturally.
• Human effort profile: For a conductor, the human is actively engaged nearly 100% of the time the AI is working—reviewing each output, refining prompts, etc. It’s interactive work. For an orchestrator, the human’s effort is front-loaded (writing a good task description or spec for the agent, setting up the right context) and back-loaded (reviewing the final code and testing it), but not much is needed in the middle. This means one orchestrator can manage more total work in parallel than would ever be possible by working with one AI at a time. Essentially, orchestrators leverage automation at scale, trading off fine-grained control for breadth of throughput.

To illustrate, consider a common scenario: adding a new feature that touches frontend and backend and requires new tests. As a conductor, you might open your AI chat and implement the backend logic with the AI’s help, then separately implement the frontend, then ask it to generate some tests—doing each step sequentially with you in the loop throughout. As an orchestrator, you could assign the backend implementation to one agent (Agent A), the frontend UI changes to another (Agent B), and test creation to a third (Agent C). You give each a prompt or an issue description, then step back and let them work concurrently.

After a short time, you get perhaps three PRs: one for backend, one for frontend, one for tests. Your job then is to review and integrate them (and maybe have Agent C adjust tests if Agents A/B’s code changed during integration). In effect, you managed a mini “AI team” to deliver the feature. This example highlights how orchestrators think in terms of task distribution and integration, whereas conductors focus on step-by-step implementation.

It’s worth noting that these roles are fluid, not rigid categories. A single developer might act as a conductor in one moment and an orchestrator the next. For example, you might kick off an asynchronous agent to handle one task (orchestrator mode) while you personally work with another AI on a tricky algorithm in the meantime (conductor mode). Tools are also blurring lines: As OpenAI’s Codex marketing suggests, you can seamlessly switch between collaborating in real-time and delegating async tasks. So, think of “conductor” versus “orchestrator” as two ends of a spectrum of AI-assisted development, with many hybrid workflows in between.

Why Orchestrators Matter

Experts are suggesting that this shift to orchestration could be one of the biggest leaps in programming productivity we’ve ever seen. Consider the historical trends: We went from writing assembly to using high-level languages, then to using frameworks and libraries, and recently to leveraging AI for autocompletion. Each step abstracted away more low-level work. Autonomous coding agents are the next abstraction layer. Instead of manually coding every piece, you describe what you need at a higher level and let multiple agents build it.

As orchestrator-style agents ramp up, we could imagine even larger percentages of code being drafted by AIs. What does a software team look like when AI agents generate, say, 80% or 90% of the code, and humans provide the 10% critical guidance and oversight? Many believe it doesn’t mean replacing developers—it means augmenting developers to build better software. We may witness an explosion of productivity where a small team of engineers, effectively managing dozens of agent processes, can accomplish what once took an army of programmers months. (Note: I continue to believe the code review loop where we’ll continue to focus our human skills is going to need work if all this code is not to be slop.)

One intriguing possibility is that every engineer becomes, to some degree, a manager of AI developers. It’s a bit like everyone having a personal team of interns or junior engineers. Your effectiveness will depend on how well you can break down tasks, communicate requirements to AI, and verify the results. Human judgment will remain vital: deciding what to build, ensuring correctness, handling ambiguity, and injecting creativity or domain knowledge where AI might fall short. In other words, the skillset of an orchestrator—good planning, prompt engineering, validation, and oversight—is going to be in high demand. Far from making engineers obsolete, these agents could elevate engineers into more strategic, supervisory roles on projects.

Toward an “AI Team” of Specialists

Today’s coding agents mostly tackle implementation: write code, fix code, write tests, etc. But the vision doesn’t stop there. Imagine a full software development pipeline where multiple specialized AI agents handle different phases of the lifecycle, coordinated by a human orchestrator. This is already on the horizon. Researchers and companies have floated architectures where, for example, you have:

• A planning agent that analyzes feature requests or bug reports and breaks them into specific tasks
• A coding agent (or several) that implements the tasks in code
• A testing agent that generates and runs tests to verify the changes
• A code review agent that checks the pull requests for quality and standards compliance
• A documentation agent that updates README or docs to reflect the changes
• Possibly a deployment/monitoring agent that can roll out the change and watch for issues in production.

In this scenario, the human engineer’s role becomes one of oversight and orchestration across the whole flow: You might initiate the process with a high-level goal (e.g., “Add support for payment via cryptocurrency in our app”); the planning agent turns that into subtasks; coding agents implement each subtask asynchronously; the testing agent and review agent catch problems or polish the code; and finally everything gets merged and deployed under watch of monitoring agents.

The human would step in to approve plans, resolve any conflicts or questions the agents raise, and give final approval to deploy. This is essentially an “AI swarm” tackling software development end to end, with the engineer as the conductor of the orchestra.

While this might sound futuristic, we see early signs. Microsoft’s Azure AI Foundry now provides building blocks for multi-agent workflows and agent orchestration in enterprise settings, implicitly supporting the idea that multiple agents will collaborate on complex, multistep tasks. Internal experiments at tech companies have agents creating pull requests that other agent reviewers automatically critique, forming an AI/AI interaction with a human in the loop at the end. In open source communities, people have chained tools like Claude Squad (parallel coders) with additional scripts that integrate their outputs. And the conversation has started about standards like the Model Context Protocol (MCP) for agents sharing state and communicating results to each other.

I’ve noted before that “specialized agents for Design, Implementation, Test, and Monitoring could work together to develop, launch, and land features in complex environments”—with developers onboarding these AI agents to their team and guiding/overseeing their execution. In such a setup, agents would “coordinate with other agents autonomously, request human feedback, reviews and approvals” at key points, and otherwise handle the busywork among themselves. The goal is a central platform where we can deploy specialized agents across the workflow, without humans micromanaging each individual step—instead, the human oversees the entire operation with full context.

This could transform how software projects are managed: more like running an automated assembly line where engineers ensure quality and direction rather than handcrafting each component on the line.

Challenges and the Human Role in Orchestration

Does this mean programming becomes a push-button activity where you sit back and let the AI factory run? Not quite—and likely never entirely. There are significant challenges and open questions with the orchestrator model:

• Quality control and trust: Orchestrating multiple agents means you’re not eyeballing every single change as it’s made. Bugs or design flaws might slip through if you solely rely on AI. Human oversight remains critical as the final failsafe. Indeed, current tools explicitly require the human to review the AI’s pull requests before merging. The relationship is often compared to managing a team of junior developers: They can get a lot done, but you wouldn’t ship their code without review. The orchestrator engineer must be vigilant about checking the AI’s work, writing good test cases, and having monitoring in place. AI agents can make mistakes or produce logically correct but undesirable solutions (for instance, implementing a feature in a convoluted way). Part of the orchestration skillset is knowing when to intervene versus when to trust the agent’s plan. As the CTO of Stack Overflow wrote, “Developers maintain expertise to evaluate AI outputs” and will need new “trust models” for this collaboration.
• Coordination and conflict: When multiple agents work on a shared codebase, coordination issues arise—much like multiple developers can conflict if they touch the same files. We need strategies to prevent merge conflicts or duplicated work. Current solutions use workspace isolation (each agent works on its own Git branch or separate environment) and clear task separation. For example, one agent per task, and tasks designed to minimize overlap. Some orchestrator tools can even automatically merge changes or rebase agent branches, but usually it falls to the human to integrate. Ensuring agents don’t step on each others’ toes is an active area of development. It’s conceivable that in the future agents might negotiate with each other (via something like agent-to-agent communication protocols) to avoid conflicts, but today the orchestrator sets the boundaries.
• Context, shared state, and handoffs: Coding workflows are rich in state: repository structure, dependencies, build systems, test suites, style guidelines, team practices, legacy code, branching strategies, etc. Multi-agent orchestration demands shared context, memory, and smooth transitions. But in enterprise settings, context sharing across agents is nontrivial. Without a unified “workflow orchestration layer,” each agent can become a silo, working well in its domain but failing to mesh. In a coding-engineering team this may translate into: One agent creates a feature branch; another one runs unit tests; another merges into master—if the first agent doesn’t tag metadata the second is expecting, you get breakdowns.
• Prompting and specifications: Ironically, as the AI handles more coding, the human’s “coding” moves up a level to writing specifications and prompts. The quality of an agent’s output is highly dependent on how well you specify the task. Vague instructions lead to subpar results or agents going astray. Best practices that have emerged include writing mini design docs or acceptance criteria for the agents—essentially treating them like contractors who need a clear definition of done. This is why we’re seeing ideas like spec-driven development for AI: You feed the agent a detailed spec of what to build, so it can execute predictably. Engineers will need to hone their ability to describe problems and desired solutions unambiguously. Paradoxically, it’s a very old-school skill (writing good specs and tests) made newly important in the AI era. As agents improve, prompts might get simpler (“write me a mobile app for X and Y with these features”) and yet yield more complex results, but we’re not quite at the point of the AI intuiting everything unsaid. For now, orchestrators must be excellent communicators to their digital workforce.
• Tooling and debugging: With a human developer, if something goes wrong, they can debug in real time. With autonomous agents, if something goes wrong (say the agent gets stuck on a problem or produces a failing PR), the orchestrator has to debug the situation: Was it a bad prompt? Did the agent misinterpret the spec? Do we roll back and try again or step in and fix it manually? New tools are being added to help here: For instance, checkpointing and rollback commands let you undo an agent’s changes if it went down a wrong path. Monitoring dashboards can show if an agent is taking too long or has errors. But effectively, orchestrators might at times have to drop down to conductor mode to fix an issue, then go back to orchestration. This interplay will improve as agents get more robust, but it highlights that orchestrating isn’t just “fire and forget”—it requires active monitoring. AI observability tools (tracking cost, performance, accuracy of agents) are likely to become part of the developer’s toolkit.
• Ethics and responsibility: Another angle—if an AI agent writes most of the code, who is responsible for license compliance, security vulnerabilities, or bias in that code? Ultimately the human orchestrator (or their organization) carries responsibility. This means orchestrators should incorporate practices like security scanning of AI-generated code and verifying dependencies. Interestingly, some agents like Copilot and Jules include built-in safeguards: They won’t introduce known vulnerable versions of libraries, for instance, and can be directed to run security audits. But at the end of the day, “trust, but verify” is the mantra. The human remains accountable for what ships, so orchestrators will need to ensure AI contributions meet the team’s quality and ethical standards.

In summary, the rise of orchestrator-style development doesn’t remove the human from the loop—it changes the human’s position in the loop. We move from being the one turning the wrench to the one designing and supervising the machine that turns the wrench. It’s a higher-leverage position, but also one that demands broader awareness.

Developers who adapt to being effective conductors and orchestrators of AI will likely be even more valuable in this new landscape.

Conclusion: Is Every Engineer a Maestro?

Will every engineer become an orchestrator of multiple coding agents? It’s a provocative question, but trends suggest we’re headed that way for a large class of programming tasks. The day-to-day reality of a software engineer in the late 2020s could involve less heads-down coding and more high-level supervision of code that’s mostly written by AIs.

Today we’re already seeing early adopters treating AI agents as teammates—for example, some developers report delegating 10+ pull requests per day to AI, effectively treating the agent as an independent teammate rather than a smart autocomplete. Those developers free themselves to focus on system design, tricky algorithms, or simply coordinating even more work.

That said, the transition won’t happen overnight for everyone. Junior developers might start as “AI conductors,” getting comfortable working with a single agent before they take on orchestrating many. Seasoned engineers are more likely to early-adopt orchestrator workflows, since they have the experience to architect tasks and evaluate outcomes. In many ways, it mirrors career growth: Junior engineers implement (now with AI help); senior engineers design and integrate (soon with AI agent teams).

The tools we discussed—from GitHub’s coding agent to Google’s Jules to OpenAI’s Codex—are rapidly lowering the barrier to try this approach, so expect it to go mainstream quickly. The hyperbole aside, there’s truth that these capabilities can dramatically amplify what an individual developer can do.

So, will we all be orchestrators? Probably to some extent—yes. We’ll still write code, especially for novel or complex pieces that defy simple specification. But much of the boilerplate, routine patterns, and even a lot of sophisticated glue code could be offloaded to AI. The role of “software engineer” may evolve to emphasize product thinking, architecture, and validation, with the actual coding being a largely automated act. In this envisioned future, asking an engineer to crank out thousands of lines of mundane code by hand would feel as inefficient as asking a modern accountant to calculate ledgers with pencil and paper. Instead, the engineer would delegate that to their AI agents and focus on the creative and critical-thinking aspects around it.

BTW, yes, there’s plenty to be cautious about. We need to ensure these agents don’t introduce more problems than they solve. And the developer experience of orchestrating multiple agents is still maturing—it can be clunky at times. But the trajectory is clear. Just as continuous integration and automated testing became standard practice, continuous delegation to AI could become a normal part of the development process. The engineers who master both modes—knowing when to be a precise conductor and when to scale up as an orchestrator—will be in the best position to leverage this “agentic” world.

One thing is certain: The way we build software in the next 5–10 years will look quite different from the last 10. I want to stress that not all or most code will be agent-driven within a year or two, but that’s a direction we’re heading in. The keyboard isn’t going away, but alongside our keystrokes we’ll be issuing high-level instructions to swarms of intelligent helpers. In the end, the human element remains irreplaceable: It’s our judgment, creativity, and understanding of real-world needs that guides these AI agents toward meaningful outcomes.

The future of coding isn’t AI or human, it’s AI and human—with humans at the helm as conductors and orchestrators, directing a powerful ensemble to achieve our software ambitions.

Synthetic data has been around for a long time, decades even. But as KPMG’s Fabiana Clemente points out, “That doesn’t mean there aren’t a lot of misconceptions.” Fabiana sat down with Ben to clarify some of the current applications of synthetic data and new directions the field is taking—working with offshore teams when privacy controls just don’t allow you to share actual datasets, improving fraud detection, building simulation models of the physical world, enabling multi-agent architectures. The takeaway? Whether your data’s synthetic or from the real world, success often comes down to the processes you’ve established to build data solutions. Watch now.

About the Generative AI in the Real World podcast: In 2023, ChatGPT put AI on everyone’s agenda. In 2026, the challenge will be turning those agendas into reality. In Generative AI in the Real World, Ben Lorica interviews leaders who are building with AI. Learn from their experience to help put AI to work in your enterprise.

Check out other episodes of this podcast on the O’Reilly learning platform or follow us on YouTube, Spotify, Apple, or wherever you get your podcasts.

Transcript

This transcript was created with the help of AI and has been lightly edited for clarity.

00.47
All right. Today we have Fabiana Clemente, senior director and distinguished engineer at KPMG. Fabiana, welcome to the podcast. 

00.57
Thank you. It’s a pleasure to be here. 

01.00
Our main topic today is synthetic data. We’ll try to focus on that, but obviously we may get derailed here and there. I think it’s fair to say at this point most listeners have heard of this notion of synthetic data. Some have probably even tried to generate their own or used a tool. But obviously you’re much more hands-on and much more active on a day-to-day basis when it comes to synthetic data. So maybe we’ll start, Fabiana, if you can describe the top two to three use cases where synthetic data seems to work right now.

01.46
Yeah that’s a good start. And yes, it’s true that a lot of users have already heard of synthetic data before. That does not mean there aren’t a lot of misconceptions. But we can delve into that a bit later on. 

But in a nutshell, understanding that synthetic data is the concept of any data that is not collected from real-world events, we can think of a different set and spectrum of use cases and applications, and we can go from the low-hanging fruit of test data management—data that will allow you to test systems—all the way to more intelligent use cases where you need to help the development of AI agents, and in between. You can think of synthetic data as a privacy-preserving way for you to have access to data.

So it’s a large and broad scope, and the scope is not served by all means by the same technology. Of course, it will vary depending on your application use case and what you want and expect to gain from synthetic data generation.

02.56
When you talk about AI applications, most people think of things like coding and programming and maybe customer support, things like that. What would be the equivalent for synthetic data? What are the most cited examples? If you were to give a talk, and you’re pressed to give examples where synthetic data is being used, what would be the top two most common reasons for using [it]?

03.34
Yeah, the three ones that I mentioned are the most common. So one of them is, “OK, I have a real dataset. I want to try to share this with my offshore team, but I can’t.” So the data can’t leave the country, but I still want to keep some level of structure, but also correlations. So you go for synthetic data instead. And here you use synthetic replicas, which is a type of synthetic data.

Or you are developing your own AI agents, and you are looking into improving your training, your evals. And then you leverage synthetic data to construct the whole system and change the epistemics around your AI agents. So I would say those two are fundamentally different, but they are true applications on how synthetic data can help nowadays.

04.32
You’ve been working on synthetic data for a while. What’s one or two examples where synthetic data solved a problem and it actually surprised you?

04.47
Surprised me? I wouldn’t say it surprised me, but definitely it’s probably the best way to leverage it. One of them—I just mentioned it—was really to enable how offshore teams would have access to a dataset that is similar and in this case, develop analytic solutions on top, for example. And that one is. . . Usually you think about how companies are restricted to share data with external entities. But you don’t think sometimes [about] how an external entity can still be the same company, just in a different country.

05.37
On the other hand, I would say that I also have seen cases where synthetic data did help a lot in improving the results of fraud detection, which, to an extent, is something that is not obvious that [that] will be a good path for a good way for you to improve your results when it comes to fraud detection.

06.05
So for teams that don’t have a lot of experience with synthetic data, what are, let’s say, the two most common mistakes? 

06.15
Oh, that’s a good one. Yeah. I would say that the biggest mistake I’ve seen is perhaps oversimplifying the complexity of synthetic data. And I’m not saying synthetic data complexity in a bad way. But as in anything that leverages data, you need planning. You need to think about “What do you want to get as an outcome?” So even if you are just building a test dataset to test the software application, you need to plan “What use cases do you really want to cover on the synthetic data?”

And usually people have this expectation that synthetic data is just “Click on a button. It’ll do exactly everything I want—it’s simple and it’s just dummy. So it’s very easy to do.” That, I’ll say, is one of the biggest mistakes I have observed. 

07.17
And the second one, I would say, is not understanding [that] there are different methodologies and different types of synthetic data that you can leverage, and being able to select the correct one for their objectives. And these are two fundamental [concepts]. They are not technical, if you ask me. They are really around requirements, and understanding the technology that you want to leverage. 

07.46
Is it fair to say that, I guess, historically, a few years ago, synthetic data—my impression at least, and I guess this was before ChatGPT—tended to be around computer vision, images, those kinds of things. So these days, [what are the] data modalities basically across the board everyone is using trying to do synthetic data? I mean, even people in robotics are doing synthetic data at this point. But what’s the dominant type of data that people are. . .?

08.28
I would say that the first data type that leveraged synthetic data was actually structured data way before text or images, if you think about [it]. We have been doing that for more than 50 years probably regardless. And I do think that images did evolve quite interestingly in the last 10 years, probably, I would say, as well as text. 

And I would say that nowadays, if you think about it, text is probably the type of synthetic data that is dominating the market. That doesn’t mean the space of synthetic data for text is well-defined or well-structured, because now anyone today considers synthetic data is just. . . An issue of oversimplifying: The outcome of an LLM can be considered synthetic data, but that does not mean it’s well-structured or is actually being correctly used and leveraged for what they are doing.

But definitely text is dominating nowadays. 

09.45
So without synthetic data, normally what you would do is say, “OK, I want to build a model; here’s some historical data or in the case of finance, here’s historical trades and financial data.” And then I’ll build the model and test the model out and then deploy to production. But obviously things can go wrong even in the scenario I painted. You can have drift—so the real world changes and then what you built your model on is no longer the same. Or you may have ended up kind of. . . The sample you created your model from was biased and so on and so forth.

Obviously the same problems will occur with synthetic data. So what are some of the common technical problems? I guess is the question for synthetic data. 

10.50
I wouldn’t say that it’s a technical problem from synthetic data. It’s a technical problem from data in general. What you just described is definitely a fundamental problem of how the processes around building data solutions are defined. 

11.00
But it could be the case, Fabiana, that your data is perfectly fine, but your synthetic data tool was bad. And so then the data says the synthetic data generated was bad. 

11.21
No, I wouldn’t say. . . And again, that goes exactly [back] to my initial point: You also can end up with good data and end up with a crappy model. And that’s a you problem. That’s a problem of you not understanding how models behave. 

11.42
But surely, just like models and model building tools, there are synthetic generation tools that are better than others. So I guess what should people look for in terms of what tools they’re using? 

11.59
It depends a lot on the use case on the end application, right?

12.04
Yeah. That’s a reasonable answer. 

12.07
And it’s an answer that nobody likes to hear. But for me that’s the true answer: It depends. And you need to be aware of what you want, in order to search for the right parameters and functionalities that you are looking for.

12.27
But basically, synthetic data becomes a part of the workflow, just like real data, right? So what you would do in order to harden whatever model or analytics that you’re building with real data, you would apply the same hardening steps, if you’re using synthetic data. 

12.52
100%. And I think it’s very important that you have what they would call a governance process around what you consider is a synthetic dataset that is ready for you to leverage. 

If there are evaluation metrics that you should put in place, those evaluation metrics will depend on the type of data that you are leveraging but also on the use case that you are building. And those processes are really important. You should make sure that the people there are leveraging synthetic data and also well trained on it. Because as you said, yes, training a model [on] synthetic data can lead to potential mistakes that you don’t want to propagate. And those mistakes usually stem exactly from the lack of processes of governance on how to generate synthetic [data], when to generate it, from where, and from what, for what. . . And having those metrics and that insurance I think it’s essential for companies to adopt on a daily basis a synthetic data generation method. 

14.04
With the rise of foundation models and generative AI, you know a few of the trends: There are things like agents, multimodality, reasoning. So let’s take them one at a time. So agents. . . Obviously, agents is a broad topic, but at the simplest level, you have an agent that does one thing well, but even that one thing may involve multiple steps, could involve tool callings and things like this. Are people starting to use synthetic data as part of their agent building process? 

14.52
I wouldn’t generalize to everyone across the industry, but I would say that we have evidence that some companies are definitely adopting [synthetic data]. Meta, OpenAI. . .

15:12
So it sounds like really advanced companies.

15.15
Yes, exactly. And I was about to say that. Even xAI, they are all leveraging synthetic data and and all of them are betting on leveraging synthetic data to enable a different structured exploration of the knowledge spaces. 

Exactly what you said, an AI agent or a set or a multi-agent system will require reasoning, a multistep kind of framework. And usually your knowledge base is not structur[ed that] way, or it’s less structured if you go and check. So synthetic data is actually one of the pieces that is helping on having those knowledge spaces well-structured in a way that they can optimize the outcome from agents for example, or even to change how models actually acquire the understanding.

16.15
So in the traditional way we used to think about building an AI system, as we collect the data, we build the model, we have an output. . . A lot of those more sophisticated companies are actually already thinking a different way, right? The AI, especially the agents, will need to learn or to be developed in a different way, where you have an hypothesis, you want to cover that hypothesis with your data, you want to model, you want to evaluate that hypothesis and make sure that your systems are updated.

And that’s where synthetic data is actually helping in changing. And this is what we call the acceleration through epistemic development, where synthetic data is the main tool to achieve that. But this is how we know, “Are we understanding the general way how sophisticated companies are using it?” I wouldn’t dare to say that everyone in the industry is using it that way.

17.15
Yeah, yeah, yeah. So one of the more interesting things in this area is this emerging body of practice around agent optimization. And the key insight there is that you can boost your agent a lot by just rewiring the agent graph without upgrading your model. So now you’ve got a bunch of open source projects ranging from TextGrad, the DSPy, OpenEvolve, GEPA. . .all designed to do a lot of these things.

And I would imagine, even as you’re optimizing your agent, you’re gonna want to run this agent through a bunch of scenarios that don’t exist in your dataset—and could involve even edge cases. And now that these agents are actually, as we discussed, doing a bunch of things, using a bunch of tools—that space is kind of broad, and I doubt that you would have that historical data handy anyway—you would need to have tools that would allow you to, with confidence, know that you’ve optimized this agent properly and that it’s ready to at least be rolled out, even in a limited way. 

18.50
Exactly, exactly. What you just described is exactly this need of a change of paradigm, right? We used to think that we need to learn by exposure, by learning historical data. We definitely now need to have our systems learning by construction and be able to test it right away. And that’s where I think the synthetic data is actually a very good (and a needed) accelerator. And I’m just glad that AI agents brought that perspective because. . . This perspective already existed. It was just harder to conceptualize and see the value, because it’s very abstract.

19.32
If you think of all the agents at least on the business side, right, so server side, the coding agents, actually a lot of these business agents are coming out of China. Since I spent a lot of time in China in the past, I’ve been talking to a bunch of people there, and I guess, the reason that the Chinese companies are moving to the West is it’s much easier to charge people in the West than in China.

So for whatever reason, they’re here; they’re building these tools that will automate a bunch of things. Right. So the canonical example would be, “Create a PowerPoint presentation based on the following specs and blah, blah, blah.” But if you can imagine these business process agents becoming more and more complex, hitting more and more tools, it’s just impossible to think that you would have all of that historical data handy anyway, so you would really need a way to simulate the behavior of these agents.

20.45
And one question I have, Fabiana, is one of the things that you keep reading about and I guess is generally true of millennials is chatbots becoming kind of true friends or companions or even romantic partners. 

It got me thinking. So if that’s happening, in order to harden this chatbot, you would need to simulate data where the chatbot is now starting to detect emotion, emotional response—you know, not just not just plain text, but there’s got to be, as you’re testing these chatbots, you have to inject all sorts of emotional scenarios, because now it’s like acting like a friend of someone. So have you heard of emotion being part of synthetic data generation somehow? 

21.52
Not really. And I’m probably a bit more skeptical when it comes to emotion. I understand your point. It depends on what you consider emotion. 

22.05
I’m skeptical too. I’m not sure if it’s happening. I’m just speculating that because the interaction is becoming emotional to some degree, there must be some people attempting to generate data that has an emotional dimension somehow. I’m just making this up, by the way. 

22.30
Yeah, yeah, yeah. [laughs] No, I bet it’s a possibility and I’m not surprised if someone was doing that. Emotions have been like one of the focuses of AI. We always heard about sentiment analysis, that always happens. So I wouldn’t be surprised. I’m not aware [of any] myself. But as I told you, I’m really skeptical that even synthetic data could be helpful on that side.

Perhaps you can create better boundaries, if that makes sense. But still, there’s always a limited capability of these models to really understand beyond syntax. And that’s where I still stand. Even if someone told me I was able to get some better results, I [would think] that those better results were achieved in a very specific, narrowed kind of situation. Though. . .

Well, we have heard the stories of people [who] are very happy with bots, that they never felt more companionship than [with] the bots they have right now. So there’s a lot of nuance there. [laughs]

23.51
One of the things that brought synthetic data back in the headlines maybe 12 or 18 months ago was there was so suddenly a lot of talk about “We’re running out of data. All these models are being trained on internet data, but everyone has basically vacuumed all of that data. So then now we have to distinguish our model or make our models even better.” 

Obviously scaling laws have multiple dimensions. There’s compute; there’s data. But since data is running out, we need synthetic data, right? On the other hand, though, a lot of people raised the possibility that AI trained on AI data is going to lead to some sort of model collapse. So what have you heard recently in terms of the concerns around. . . 

You know, obviously “There’s no such thing as free lunch. . .” So every kind of thing you use has potential disadvantages. So this disadvantage that people bring up, Fabiana, [is] if you’re able to train models on synthetic data then that’s going to degrade the model over time, because basically it’s like a loop, right? The model’s capability of generating synthetic data is limited by the model itself. So therefore, you know… 

25.42
And that’s under the assumption that the synthetic data that we are talking about is generated by the LLMs. We can’t forget that there’s way more about synthetic data. There are simulations, and simulations [have been] used for quite some time with very good results. They were used for the studies of COVID vaccination. It is used every day with weather, and they work. But of course there’s a limitation. I agree there’s no free lunch. I wouldn’t say it degrades the capability of the model, but I would definitely say a plateau.

Because unless you are doing assumptions based on what you know, and you just know that there is no collected data but this actually happens. . . But unless you know new behaviors, the fact that we are generating the same data from around the same behaviors, you will achieve a plateau. But also I think that’s one of the things that regardless narrative AIs like LLMs will always have a problem with. They always are dependent on having seen a lot of data.

And we know that that plateau will eventually be achieved. And then we have a totally different problem. How mathematically can we solve this bottleneck? And on that side, I don’t think synthetic data will be the answer anymore. 

27.32
What we just discussed there focuses mainly on LLMs and foundation models involving text. But one area that people seem particularly excited about these days are foundation models for the physical world, primarily robotics. So in that world, it seems like there’s two general approaches that people are doing. One is [to] actually collect data, but obviously they don’t have the same internet scale data that you’ll have for LLMs.

Secondly, you generate data by having humans do a task, and you just capture it on video and that’s how you collect data. And then the third approach is simulation. So basically now that you’ve collected human data, maybe you can have simulations to expand the amount of data you have. The critics say that simulations are fine, but there’s still a gap between [the] simulation [and] real data.

I mean these are you know, people like Rodney Brooks—one of the granddaddies of robotics. So it seems like, in certain areas like that, synthetic data may still need work, no?

29.12
I wouldn’t say “may still need work,” but I would say that definitely needs to be more explored. It’s more on that side. Because I know companies that work on specifically synthetic data for robotics, and they are having very good results.

And I understand that a lot of people. . .

29.39
We have to have them talk to Rodney. [laughs]

29.41
Perhaps. Because we have to be pragmatic. You want to develop robots and solutions for automation. But data collection is expensive, time-consuming. And it’s very hard to get all the movements that you want to capture collected just by nature. 

Having said that, simulation is great. Synthetic data can help in, you know, building a bridge between the real data and the simulations. In some cases, it won’t cover 100%, but it will cover perhaps 80% to 90%. And sometimes it’s better to just have 80% of the cases than having the 20% covered by real data. I think here it’s more a pragmatic approach, and [in] real-world scenarios, a lot of times the 80% are very good. Excellent actually. 

30.42
So in closing, going back to the topic of agents, obviously, people tend to get ahead of themselves—people are still working on single agents to do very narrow tasks. But then on the other hand, there’s already a lot of talk about multi-agents, and obviously multi-agents introduce a lot more complexity, for one, particularly if the agents are communicating. So there’s just communication challenges between those agents. What are some of the new tools that you’re hearing about that target specifically multi-agents or the scale that agents have introduced to synthetic data?

31.34
Not new tools, actually. But of course, we have been actively working on—and a lot of the vendors in synthetic data that already work with this type of data are exploring—covering new scenarios and new features. A lot of these agents are relying, for example, on document processing. So there are new solutions for document generation, which is highly helpful.

One of the things that I also like is, for example, in market research, there’re all these synthetic personas required nowadays to accelerate hypothesis testing—learning speeds, for example, which is very interesting. Or there are solutions being developed, but to help with reasoning structure for bots. So those are, I wouldn’t say specifically tools that are coming out, but are definitely solutions that are being developed targeting the needs and requirements to test for multi-agent architectures.

32.46
Yeah. It seems like there’s. . . Like there’s a group out of Meta that—I don’t know how real this is, but they released a paper basically even uses Ray for scale and orchestration and specifically, [to] increase throughput mainly to generate synthetic data for multi-agent scenarios. I’m not sure. It seems like according to the paper, they’re actually using this, but I’m not sure if anyone else is using it. 

33.41
Yeah, but that’s. . . The companies will use a different way. Right? That’s an architecture solution for a problem they have. They want to augment the throughput, test the system loads. And that will be a decision for the different engineering teams on how to apply synthetic data generation. 

Testing throughput, testing systems capabilities, well, we have been using synthetic data that way for decades now. It’s just a change of paradigm. And by the way, it’s not really a change because if we think about multi-agents, just as we think about microservices from the 2010s, it’s the same concept; it’s the same needs. It’s just a shift in terms of tools.

Just because instead of being applied to just software engineering, you are actually applying this to AI-driven solutions. So I see a lot of change in that area, on tooling, even, for example, authentication for agents—we are seeing a lot of solutions exactly for that. But it’s not something specific to synthetic data. It’s more on the broader sense of architectural solutions to deliver multi-agent systems.

35.01
Yeah. And also it seems like it fits into the natural tooling that’s happening in multimodal data and data for generative AI in general in that you need high throughput, but you also need efficient utilization of a lot of resources between GPUs and CPUs and fine-grained utilization, because, basically, these are precious computing resources.

And with that, thank you, Fabiana. 

35.37
Thank you, Ben. Thank you for having me. This was a pleasure.