The following article originally appeared on Medium and is being republished here with the author’s permission.

As companies adopt AI tools, a lot of time is spent on thinking about AI policies from a security, compliance, or even cost-focused angle. But many leaders are neglecting to address how their teams should work with AI in the context of the team as a whole. This creates a lot of unresolved tension, and it’s time for leaders to step up and set some guidelines not just for how to use AI in an “approved” sense but how to use it respectfully.

When I say respectfully, I am not talking about the baseline appropriate workplace behavior (bullying, abuse, harassment, etc.). Instead, I’m concerned that many of us haven’t considered that the ways AI can make an individual more productive (literally enabling them to produce more outputs) can have an overall negative impact on the team’s productivity. Leaders can’t just sit around and expect that their teams will know that they can’t just produce slop and send it to others; if you haven’t set up a thorough policy yet, here are some suggestions on what to cover.

Elements of respectful AI use

Don’t ask someone to read/review what you haven’t read or reviewed yourself.

This is one of the most common frustrations I hear amongst people working on AI-heavy teams. Whether it’s code that the owner didn’t really bother to understand before submitting for review or documents that they generated and didn’t bother to read, too often people try to steal productivity from their colleagues by streamlining their production of work while asking their colleagues to do all of the quality control themselves. It’s great to have a loop of AI code generation → AI code review → AI fixes → final human review, but if the person prompting the AI doesn’t bother to review that code first, they’re putting a huge validation tax onto their teammate, who has to trust both that you prompted well AND that the AI understood the context and problem well enough to get a sustainable solution.

Documents are an even bigger temptation than code, because AI is so verbose and most of us hate writing and editing. It’s easy to get into a loop where you ask the AI some questions, skim the answers, output a document and send it to others. I’m guilty of this myself! But what makes sense when you’re skimming one answer at a time may not make for a good overall document, and there is a big difference between answering individual questions and writing for a human reader. In particular, the context that you have in your own head as you are talking to the AI may not come out at all in the document; if you don’t bother to read it thoroughly before sending it out, you won’t catch the gap in framing.

Even worse, sometimes people don’t even understand what the document they prompted is trying to say. Can you describe this document, and have a conversation about the concepts it presents with others and why it makes sense? If not, you have no business sending it along without at minimum the huge caveat “This is AI-generated and I still don’t really understand this space, please help me.”

Many people have reached the point where they won’t read something a person didn’t bother to write themselves, and who can blame them when so many don’t even bother to read their output before sending it on?

Shorter is better.

Part of the annoyance of reviewing AI-generated work is that the AI can be painfully long-winded. AI code often looks like tutorial code, with much more verbosity than human developers would bother with. Add in the temptation to one-shot big changes rather than thinking about how to break the code down into pieces, and you can end up with stacks of thousand line pull requests. The documents AI produces are so thorough that something that should be 3 pages turns into 10 or 20. And for those who have fully embraced AI for all of their text-based interactions, you start to see the LLM-generated wall of text chat messages or emails.

This is, frankly, just rude. It goes hand in hand with not bothering to review your own work, but even if for some reason you convince yourself that you really did read and edit that giant PR/document/message, you’re still asking so much more of the audience than you probably put into the exercise in the first place. When it comes to code, I encourage you to honestly ask yourself: If this broke at 3:00am and none of the AI tools were working, would you be able to look at the PR context and the change and debug it? If not, it is probably too much. When it comes to a big document, at a minimum, have you at least summarized the important points up-front? If someone is just going to ask an AI to summarize the document themselves, you should probably do more work to provide that value before handing it off.

Finally, if you’re writing long-winded emails or chat messages with AI-assistance in order to painstakingly try to explain something, perhaps you actually need to have a meeting or call instead. Increasingly long text exchanges have always been a sign that people need to stop and talk face-to-face, and AI logorrhea hasn’t changed that.

AI is not an excuse to turn off your brain, or your heart.

Signs we’ve switched off our brains and our hearts include: not reviewing the AI-generated work, not taking the time to do human editing, not breaking the changes down into chunks, and avoiding real conversations through AI-mediated text exchange. This guidance is about respectful use of AI because if you have empathy for your colleagues and respect for their time and skills, you will show them the courtesy of giving them work that you are proud of, that you stand behind, that you have thought through and can explain. The AI may have produced a lot of the output, but you thought about all of the pieces that needed to be done, and used the extra productivity to make something better: more reliable, simpler, well tested, whatever. If you find yourself not thinking at all and just mindlessly prompting, accepting output, and moving forward, it’s a warning sign that something is wrong. Perhaps take some advice from Vicki Boykis on adding friction to your development process (or whatever the equivalent is of your day-to-day work).

Framing these guidelines

If you decide to do this, one final tip from me: Assuming your company has some sort of company values, it’s always a good idea to call back to these values when you create policies and guidelines like this. It’s one thing to abstractly say that shorter is better, but if you can tie that to a value for your company, it will resonate more strongly. As an example, if I were at Amazon I might consider tying “shorter is better” to the leadership principle Invent and Simplify. And since shorter is better and this is already too long, I leave you here.

Enjoy this post? You might like my books The Manager’s Path and Platform Engineering: A Guide for Technical, Product, and People Leaders.

The practice of tokenmaxxing appears to be dying out, even before I had a chance to write about it. Good riddance. Burning tokens to create the appearance of productivity was fated to last only until the accountants learned about it, and the strictest of all accountants is one’s personal checkbook. What got many developers thinking about the cost of AI was the change in GitHub Copilot’s usage charges. The cost of Copilot went from a monthly fee with unlimited use to a monthly fee that purchased a limited number of credits, which are used to pay the AI provider of your choice. One credit is equivalent to US$0.01; when you’ve used up your credits, you can upgrade your account or pay for additional credits as you go.

The question isn’t why this didn’t happen earlier; it’s why this happened now. Tokenmaxxing is both the creation and victim of two large-scale trends in AI. First, starting with OpenAI, the major AI providers were all playing a blitzscaling game that prioritized user growth over profitability. Giving AI services away for free got you more users, and in the long run, scalers would figure out how to make money from end-user fees, selling user data, or advertising. This process inevitably ends in enshittification, and is still very much the road we’re on.

Second, token usage exploded late in 2025. The appearance of “reasoning models,” which use tokens to maintain an internal dialog in the course of solving a problem, increased the number of tokens used to respond to each prompt. Reasoning tokens are a model’s conversation with itself about possible responses to the prompt, and are often more numerous than the prompt and response themselves. Whether or not users see the reasoning process (often they don’t), reasoning tokens add to the bill. They are frequently counted as “output tokens” because they are generated by the model, and are more expensive than input tokens.

The appearance of agents also multiplied the rate at which users consumed tokens. In May, 2025, Simon Willison quoted Anthropic’s Hannah Moran’s definition of an agent: “Agents are models using tools in a loop.” The Tredence blog writes: “The agent loop is a repeating cycle in which the AI reads the current data, thinks through what it means, chooses an action, carries it out, checks what happens and starts over.” If you’ve ever watched Claude Code, OpenClaw, or any other agent work, a single request can become many calls to a model, each one using hundreds of tokens, if not thousands. In addition to the current request, one agent-generated invocation can contain the task’s entire accumulated context and relevant documents. Between reasoning tokens and agents, token usage goes up by a factor of hundreds.

The increase in token usage might not be an issue if it results in problems being solved and tasks completed more effectively. But it collides with the loss-leader pricing of the blitzscalers; their willingness to operate at a loss to gain control of a market has limits. Regardless of whether the number of AI users is increasing, the amount of computation, and therefore cost, per user grows as the use of agents increases. Reasoning models increased token usage; agents compounded the problem; and that led to price increases.¹ Microsoft/GitHub doesn’t want to pay Copilot customers’ AI bills. We haven’t yet seen across-the-board price increases from the AI providers themselves. But we have seen GitHub’s token credits, and we have seen Anthropic and OpenAI price more capable models significantly higher than older or less capable models. Fable is twice as expensive as Opus 4.8, and while some writers have called this pricing “fantastic,” that’s probably because they were expecting an even greater increase. While Fable can delegate tasks to Anthropic’s less expensive models, most early users observe that with Fable, token use goes up rather than down. Anthropic’s switch to token-based billing for its agent SDK (currently on hold) is another signal that the days of inexpensive AI are coming to an end. OpenAI’s story is similar: GPT 5.5 costs twice as much GPT 5.4 per million tokens.

It’s also important to take capacity into account. Huge data centers have been in the news, but those data centers haven’t been built yet. More important, the electrical infrastructure needed to support those data centers—transmission lines, generators—hasn’t been built either, and that’s not an investment over which AI companies have much control. They can build their own power generation facilities on a data center campus, but that’s a huge investment in technologies that they’re not familiar with. And even if you generate power locally, you need other kinds of infrastructure: rail for coal, pipelines for gas. This isn’t (yet) an essay about data center power consumption and its consequences, but it is another factor that limits increased token usage. We’ve seen Anthropic’s outages blamed on capacity, and Anthropic has responded by leasing unused data center capacity from SpaceX. But the other way to respond to increased demand that can’t be met by current capacity is to increase prices, limiting customers to those who can afford to pay. That increase is being noticed by managers, accountants, and independent developers.

Token optimization and accountability are the inevitable consequence of upward pressure on token price. One way to build accountability is through better governance, a route Bennie Haelen describes in “The Subsidy Ended: What Tool-Using Agents Actually Cost.” Better governance is achieved through building an observability layer that lets you see exactly what the agents and models are doing. With a well-designed observability layer, you can see whether the data sent to the model is growing with each invocation, whether the model is using appropriate tools, whether tools are being called repeatedly, and a lot of other information that will tell you whether your agent is running efficiently.

Another piece of token accountability is understanding which models are running your agent’s requests. General-purpose reasoning models range from expensive high-performance models like Claude Fable or Opus 4.8 to models like Gemma 4 26B that can run on a well-equipped laptop, and some models that are even smaller. While it’s tempting to say “I need the best; I’ll run Opus 4.8 or Fable with maximum reasoning,” most requests don’t require that level of reasoning or expense. Agents will be able to decide what model is best for processing every request. Fable can delegate, and we expect other frontier providers to follow as models incorporate agent capabilities. And there’s an active world of open models outside of the frontier AI providers. Vicki Boykis writes that models running locally now work almost as well as frontier models. Tools like OpenRouter give you a model-independent way of routing requests to different models, including open models that run locally. OpenRouter can be integrated with OpenClaw, Claude Code, Cursor, Codex, and other agents to provide intelligent routing.

Tokenmaxxing is dying. It will no doubt take time for its vestiges to die away, and there will always be developers who think they can game the path to a promotion, along with managers who insist on being “all in” with AI. But spending tokens responsibly is now the norm, whether you pay with your own checkbook or a company account. Token optimization will only become more important as per-token charges increase. They undoubtedly will.

Footnotes

1. Some articles make the strange claim that tokens have gotten cheaper by up to 98%. GPT-5.5 suggests that these writers are considering the work that can be done per token. That comparison may be worthwhile, though it’s unclear how to compare GPT-3 with 5.5 or Fable meaningfully. For this article, a token is a token. ︎

In late 2025, the security community stopped treating indirect prompt injection as a theoretical risk. It had spent two years as a tidy lab demonstration; then production systems started getting hit. The OWASP Top 10 for LLM applications now ranks prompt injection as the number-one risk, NIST has called indirect injection generative AI’s greatest security flaw, and academic researchers showed that a single poisoned email could coerce a model into exfiltrating SSH keys in up to 80% of trials, with zero user interaction. The attack needs no malicious binary, no phishing clicks, and no anomalous login. The agent simply reads content and takes action, exactly as designed, and the content was written by an attacker.

The most instructive example is ForcedLeak. In September 2025, researchers at Noma disclosed a critical vulnerability chain (CVSS 9.4) in Salesforce’s Agentforce platform: An attacker embedded malicious instructions in the description field of a routine Web-to-Lead form. The text sat harmlessly in the CRM until an employee later asked the AI agent to process that lead, at which point the agent dutifully executed both the legitimate query and the attacker’s hidden payload, exfiltrating sensitive CRM data to an external server. The detail that should keep you up at night is that the exfiltration destination was a domain still on Salesforce’s trusted allowlist, one that had expired and which the researchers re-registered for about five dollars. Every security control saw legitimate traffic to a trusted domain. Nothing looked wrong.

If your instinct reading that is “we filter for prompt injection,” you’re defending the wrong perimeter. Input filtering is necessary but nowhere near sufficient. The uncomfortable truth is that the injection isn’t the breach; the action is. And almost everything we call “AI security” is aimed at the wrong half of that sentence.

The defense everyone is building

Ask most enterprise AI teams how they secure their agents, and you’ll hear a consistent answer: They sanitize inputs. They harden system prompts with elaborate instructions to ignore conflicting directives. They run classifiers over incoming content to flag adversarial patterns. Some have adopted the more sophisticated training-time defenses the frontier labs have published—instruction hierarchies that teach a model to assign differential trust to different sources and reinforcement-learning approaches that harden models against injection in agentic contexts.

All of this is good work, and none of it should be abandoned. But notice what every one of these techniques shares. They all try to stop the model from being fooled. They assume that if we make the model robust enough at the input layer, the system is safe. That assumption is the vulnerability.

We’ve spent two years trying to make the model unfoolable. The systems that survive contact with production assume it will be fooled anyway.

Why the input layer is the wrong perimeter

Prompt injection isn’t a bug a future model will lack. It’s a structural property of how language models work. The model consumes a single undifferentiated stream of tokens at the moment of inference. Your instructions, the retrieved document, the tool output, and the web page just fetched are indistinguishable channels collapsed into one context. There’s no hardware-enforced boundary between “trusted instruction” and “untrusted data” the way there is between kernel space and user space in an operating system.

This is why the attack surface explodes the moment an agent becomes agentic. A chatbot that only talks is a contained risk. An agent that retrieves from the open web, reads email, queries databases, and calls APIs ingests adversarial content from a dozen sources on every turn, and any one of them can carry an instruction. Researchers cataloging real agent ecosystems have already found hundreds of malicious third-party extensions performing data exfiltration and silent injection without any user awareness. These aren’t laboratory curiosities. They’re the production environment.

So, if you can’t guarantee the model will never be fooled—and you can’t—then architecture that depends on it never being fooled is built on sand. You need a second principle, one distributed systems engineers have understood for decades.

Verify, then trust

The principle is simple to state and hard to retrofit: An agent’s proposed action should be validated against an external, deterministic policy before it executes, regardless of why the agent proposed it. The validator doesn’t ask whether the instruction that produced the action was legitimate. It doesn’t try to detect the injection. It asks a different and far more answerable question: Is this action, on its face, permitted?

This inverts the burden. Detecting a cleverly disguised malicious instruction is open-ended because the adversary gets to be arbitrarily creative. Checking whether a wire transfer exceeds a hard dollar limit is a closed problem with a definite answer. We move the security decision from where the attacker has infinite freedom to where they have almost none.

Crucially, the check must be deterministic code, not another model asking, “Does this look dangerous?” The moment you ask a second LLM to adjudicate, you’ve reintroduced the exact same vulnerability one layer down. The enforcement layer is boring, auditable conventional software, and that’s the point.

Here’s what it looks like in practice. An agent managing procurement proposes an action, and a runtime contract evaluates it before anything reaches a real API:

# agent_contract.yaml
 agent_id: "procurement_executor_07"
 role: "EXECUTOR"
 policy:
   approve_invoice:
        max_amount_usd: 50000
        allowed_vendors: from_approved_registry
        require_human_above_usd: 10000

 # Runtime, on a proposed action:
 ACTION   approve_invoice(vendor='Acme', amount=1200000)
 REJECTED policy violation: max_amount_usd
        proposed 1,200,000 / limit 50,000
        action discarded, human notified, no API call made

The injected instruction at 2:14am never matters here. The agent can be perfectly, catastrophically fooled, and the wire transfer still doesn’t happen, all because a simple deterministic check stood between the model’s output and the outside world, and the proposed action failed it.

This only works if the action arrives structured, which makes structure a precondition.

The contract inspects approve_invoice (vendor, amount) cleanly only because the action is already typed. If the agent emits prose, “please approve the Acme invoice,” something has to parse it, and the only thing that parses open language is another LLM, so the indeterminacy walks back in. That dictates the design.

A consequential action must cross the boundary as a typed tool call, never as free text. Where the input is unavoidably natural—an email saying, “Wire them their balance” for example—let the model extract a structured value but never let its extraction be self-authorizing. The model proposes the amount; the gate still checks it against the limit, the vendor registry, and the actual balance in the system of record, not the number the email asserted. Extraction is probabilistic, while validation stays deterministic.

A few decisions are pure judgment with no schema, such as “Is this email phishing?” There the model stays in the loop. You bound the consequences instead, with reversibility and human review above a threshold. Contracts protect parameterizable actions, and unparameterizable judgments fall back to containment.

The architecture this implies

Once you accept that the action layer is where security lives, three design commitments follow, and they map almost directly onto principles that hardened distributed systems years ago.

Least privilege for agents, scoped to the action, not the agent. The naive version assumes you can predict what an agent will do and provision it accordingly. For a specialized agent you can: One that only summarizes has no business holding a credential that moves money. But the agents people actually reach for are general. In a single session, I might ask a coding agent to summarize a file, write code, execute it, and query company data—four tasks with four risk profiles, none of which are enumerated in advance. Static least privilege collapses the moment one identity spans that range.

The fix is to make privilege a property of the action, not the agent. The agent holds no dangerous capability by standing grant; it requests narrow, transient elevation per action, which the same deterministic gate approves or denies. Reading a document is auto-approved; querying the warehouse is not. The dangerous credential exists only for the instant the action is permitted, then evaporates. One caveat: This governs what an agent may reach but not what the code it writes then does. Executing code can be gated as a capability, but what executes still needs containment, sandboxing, and egress control, because generativity is a different problem from access.

Zero trust for machine identities. Every action an agent takes should be authenticated and authorized as if it came from an untrusted actor, because, functionally, it might be acting on an attacker’s instructions. The proliferation of agents has expanded the attack surface faster than most identity systems were designed to handle, and treating agent traffic as inherently trusted because it originates inside your own system is precisely the mistake.

Capability contracts at the boundary. Every consequential action passes through a deterministic gate that encodes what is allowed, dollar limits, rate limits, allowlisted destinations, mandatory human review thresholds. The contract is version-controlled, auditable, and lives entirely outside the model.

The trap of normalized deviance

The quieter organizational danger is the slow accumulation of false confidence from connecting insecure agents to real systems and watching nothing bad happen. . .for a while. Researchers have warned about indirect injections for years, but most deployments have gotten away with it. Each uneventful day makes the next risky connection feel safer. This is the normalization of deviance. Every system that eventually failed catastrophically felt the same way: fine, fine, fine, until it wasn’t.

The teams that will weather the coming wave of agent incidents aren’t the ones with the cleverest input filters. They’re the ones who assumed compromise from the start and built the boring enforcement layer anyway, the ones who decided that an agent’s autonomy ends precisely at the point where it tries to do something irreversible.

Where to start on Monday

You don’t need to rearchitect everything. Start by inventorying the actions your agents can take, and sort them by blast radius: What’s the worst thing that happens if this action fires when it shouldn’t? For every high-blast-radius action, write a deterministic contract that gates it and put a human in the loop above a threshold you can defend to your risk team. Then, and only then, keep hardening your inputs.

Prompt injection won’t be solved at the input layer, because it can’t be. But it can be rendered survivable at the action layer, where deterministic code gets the final word. The model’s job is to be useful. Your architecture’s job is to make sure that when the model fails—or worse, when it has been turned against you—the failure stops at the gate.

Harper Carroll came to AI education through a CS background at Stanford, machine learning engineering at Meta, and a brief stint at a small GPU compute startup in late 2023, where she noticed that almost no one understood how to fine-tune open source models. She started writing and teaching to help drive signups for the startup’s platform. Her first guide, posted right after Mistral 7B was released, when she had about 50 followers, got 50,000 views. In March 2024, a video explaining the difference between AI and machine learning got 5 million views, with 1 in 20 viewers following her afterward. She now has more than 500,000 followers across multiple platforms and is a full-time AI educator.

We covered fine-tuning versus prompting, what it actually means to learn to code in 2025, and what the AI field gets wrong when it talks to the public.

Understanding the world with math

We started with Harper’s own AI learning journey, and it contained a wonderful insight. She grew up loving math and came to computer science at Stanford because algorithms seemed like wonderful math puzzles. Eventually she realized that AI is “understand[ing] the world around us with math.” Text-based LLMs are only one branch. The field as a whole is “the math of the world.” That seems like a deep intuition that all of us need to internalize.

AI as a medium

A study that circulated last year found that people who used AI to write essays showed reduced brain activity compared to people who write unaided. The reaction in many quarters was alarm. People said, “We’re outsourcing cognition and our brains will atrophy.” Harper’s smart response was that those users must have given the AI a one-sentence prompt and accepted whatever came back.

As she put it, that’s the equivalent of just telling Alexa to order you the most popular book this week. Of course less brain activity is being measured! Contrast that with the difference between shopping for a book by browsing and searching at Amazon versus driving to a physical bookstore. There’s certainly a difference, but it isn’t outsourcing cognition. It’s saving time, and that time might well be spent on other demanding cognitive tasks.

My framing is that AI is a medium, the way language is a medium, or photography. Anyone can take a photograph or write a book. The words available to every writer are the same; what differs is what they do with them, just as some photographers do something with it that others can’t. The same is true of software. There’s a line in Aaron Sorkin’s movie The Social Network where the Zuckerberg character says about the Winklevosses, “If you guys were the inventors of Facebook, you’d have invented Facebook.” An idea and its execution aren’t the same thing. One person gives AI a prompt and the output is bad. Another builds a process around AI and the output is great. What you bring to the medium is what determines the result. Harper agreed.

Fine-tuning is like psychedelics for AI

I’ve been trying to figure out how we can use AI for writing and editing at O’Reilly. We want skills and workflows that accelerate our productivity but don’t produce copy that reads as whatever the base model sounds like when nobody’s putting in any effort.

Takeaway posts like this one are a great use case for AI-assisted writing. As source material we have a transcript, with the actual conversation between the participants (or in the case of one of our online conferences, their presentations). We want a structured summary that captures the high points and suggests possible clips for social media. I (or whomever is using this AI-assisted workflow) can then rewrite, rearrange, elaborate, or delete from that first draft. It might not be as good as a draft written from scratch, but quite frankly, it’s far better than the alternative, which is no summary at all. I just don’t have time to write them all unaided.

When I’m writing an article, I generate a similar “transcript” by recording myself talking about the ideas I’m wrestling with and trying to put into the world. Then I ask Claude to put it together into something a bit more structured.

I’ve been improving Claude’s ability to produce prose that we can use by rewriting its output, showing it the differences, and then asking it to construct a skill that captures what it’s learned. Over time, it’s gotten closer and closer to something that I’m comfortable with, and I’m now generalizing that into a system that learns any author’s voice, respects the various conventions of the target content type (which can be very different across books, articles and blog posts, social media, and marketing materials like back cover copy and course descriptions), and applies editing suggestions from my favorite books on good writing, including Strunk and White and On Writing Well by William Zinsser.

Harper attacked the same problem from a different angle. She built a dataset of roughly 1,000 of her Instagram captions, video transcripts, and X posts, then fed them to Claude as context and asked it to write in her style. Unfortunately, the output tested 100% AI by a detection tool, even with 1,000 examples of her real voice in the prompt. She then fine-tuned an open source Llama model on the same data. The fine-tuned output tested 100% human. She gave a compelling demo at South by Southwest showing how easy this is to do. It took her about 20 minutes.

After Harper said that prompting doesn’t shift the output distribution the way fine-tuning does, I told her the story about the French writer Marcel Proust that I first used in my conversation with Steve Wilson, which I picked up from Alain de Botton’s How Proust Can Change Your Life. A friend comes to visit the bedridden Proust, and making polite conversation begins to tell him about the train trip to Paris. “More slowly,” Proust replies. This cycle repeats several times until the friend is telling him small details like the old man feeding pigeons on the steps of the station.

Harper got it, and broke it down more slowly in her inimitable way. Here’s why in-context prompting fails where fine-tuning succeeds:

Basically AI models are these massive mathematical equations, and the parameters are variables when you’re training, and then they become constants in those equations when you’re running inference . . .So what you’re doing when you’re training the model is you’re learning how to map, by adjusting those constants when they’re variables during training,. . .input to desired output.

Once the model is deployed, the probability distribution over output tokens is fixed. You can put 1,000 examples in a prompt and ask the model to pattern-match, but you’re asking it to do that with frozen weights. The surface behavior bends a little, but the underlying distribution doesn’t shift. Fine-tuning lets you actually modify the weights and how the model wants to write.

Her suggested approach for building the training dataset is to take your own writing, have AI rewrite it with its characteristic tics, then train with the AI version as input and your original as the target output. You’re teaching the model to undo the tells.

Should people still learn to code?

We also spent time on the inevitable question of whether people should still learn to code. We both agree they should, but not necessarily like they used to, by learning the detailed syntax of a programming language, then by trial and error as they painfully learn how hard it is to get the desired behavior.

Harper’s take (which I also agree with) is that vibe coding has lowered the floor. People who could never afford to hire someone to build a product can now do so themselves. But it has also raised the ceiling, because people who actually understand systems can build vastly more sophisticated things with the same tools, which takes us back to the case for AI as a medium.

Perhaps more importantly to the question of how much coding you should learn, experienced developers will also see failure modes that pure vibe coders miss. Harper gave an example that came from watching a friend using an agent tool that had, at some point, started storing its data in a Word document and using it as a makeshift database, probably because the session started with a Word doc. It was extremely slow and extremely inefficient. An engineer sees the problem immediately. A vibe coder might run that system for months before noticing something is wrong.

So yes, you should learn enough about coding to understand what’s happening. The art of teaching programming to the next generation will be developing useful projects that also highlight underlying concepts of software architecture and engineering.

Intuition as differentiator

Silicon Valley runs heavily on logic and on the idea that good decisions come from better data, more rigorous analysis, and sharper models. In this environment, intuition can get dismissed as something “soft and fuzzy,” Harper noted. And that’s the wrong mindset for AI.

AI is getting better and better at exactly the things the logical axis does well, but intuition remains a challenge because it often contradicts what the data says. Good intuition “goes against the input,” to use Harper’s phrase. A model that’s been trained to recognize patterns in data will, almost by definition, struggle with making decisions that run counter to those patterns. Just as skills-informed judgment supercharges AI-assisted engineers, intuition could be a uniquely human skill for a long time. Elevating it as a concern might bring the industry more of an attitude of humility towards ourselves and our place in the world.

What the field gets wrong

I closed by asking Harper what the AI field most consistently gets wrong in how it talks to the public. She said that too much of the public-facing discourse leads with fear, of job displacement, of rapidly approaching AGI, and of a rocky transition that requires a universal basic income to cushion the blow. She’s not calling those impossible futures, but she thinks they’re the wrong introduction to the technology.

A lot of companies are using AI to ask how to do the same things at lower cost. The better question is how to raise ambitions. AI doesn’t just scale individual capabilities. It scales what organizations can attempt. But for it to work out that way, everybody has to actually learn AI. We can’t have AI haves and have-nots. That means lower-cost models, serious open source investment, and companies that don’t just become serfs to the major platforms.

Harper has been making this point for a while, to audiences ranging from engineers to people who’ve never written a line of code. “There is not really much to fear right now,” she says. “AI is this incredible productivity tool.” The people who will struggle, in her view, are the ones who refuse to engage with it at all.

At O’Reilly, we’ve been working on a version of the same narrative at an organizational level. The fear-first narrative produces avoidance, and avoidance is the one thing that will actually leave someone behind. So we’re building a corporate AI transformation practice that starts with people’s existing jobs, and figures out how to “mix in” AI to make them more impactful. We’re learning how to teach both the humans and the agents at the same time to make them more productive together.

On July 9, I’ll be speaking with Trail of Bits cofounder and CEO Dan Guido about the playbook his company used to go AI native, which he first outlined at this year’s [un]prompted. He’ll give a version of the same talk, then take about 40 minutes of audience questions on what worked, what didn’t, and what is still unsolved. I hope you join us to find out what’s changed since [un]prompted and where the playbook is heading next. Register here; it’s free and open to all.

The following article originally appeared on Angie Jones’s LinkedIn page and is being republished here with the author’s permission.

I’m fascinated by the concept of agent memory. LLMs are stateless by design, meaning they have no memory or awareness of past interactions. Each prompt you send to an LLM is treated as a completely isolated event.

When you have a continuous chat with an AI agent, it feels like the AI remembers previous messages. However, the interface itself is faking it. Behind the scenes, your agent takes the entire conversation history and resends all of it to the LLM as one giant, combined prompt.

Companies, researchers, and even indie devs are all trying to crack agent memory. Because once an agent can remember, the entire interaction changes. It can build on what it learned, adapt to the user, resume work after a restart, and develop a sense of continuity.

Recently, I spent time with Richmond Alake, who has been in the trenches working on agent memory at Oracle.

We talked about the different kinds of memory, why memory is harder than it sounds, and what it takes to build a memory system that is actually useful in production.

That conversation made something very clear to me. When people say, “agent memory,” they often mean very different things.

So let’s unpack the various types of memory.

Conversational memory

Conversational memory is the one most people think of first. It stores the messages exchanged between the user and the assistant.

This makes sense. If I ask, “What did I say was the ultimate goal of this task?” the agent needs access to the conversation in order to answer. Without that history, every turn starts from zero.

But this is also where many memory systems go wrong.

The most common first attempt is to keep appending prior messages to the prompt. For example:

User: I’m building a customer support agent.

Assistant: Great, what should it do?

User: It should look up past tickets and draft replies.

Assistant: Got it.

User: Also, I prefer Python and FastAPI.

Then on the next call, we send all of that back to the model along with the new question.

This works for a short conversation, but the agent only “remembers” because we keep reminding it. This is not really memory engineering.

Eventually, the conversation gets too long and the model receives a giant blob of context where some details are important, some are stale, and some are completely irrelevant. The agent may technically have the information, but that doesn’t mean it can use it well.

So yes, conversation history is a valid and important type of memory. But it shouldn’t be the whole memory strategy. Real agent memory requires deciding what should be stored, where it should be stored, how it should be retrieved, and when it should be summarized, forgotten, or compressed.

Semantic memory

Semantic memory stores durable facts.

These are things that should outlive the exact conversation where they were learned:

• The user prefers Python over TypeScript for backend work.
• The customer support agent needs access to past tickets.
• The production system handles 50,000 queries per day.

This is different from conversational memory because the exact wording and sequence are less important. What matters is the meaning.

If the agent needs to recall what stack the user is using, it should retrieve the memory even if the user never says those exact words again.

Vector search is useful for this. The memory can be embedded and retrieved by semantic similarity.

The benefit is that the agent doesn’t need to replay the full conversation. It can retrieve the few durable facts that are relevant to the current request.

Episodic memory

Episodic memory stores events.

This is the “what happened” layer of memory:

• The agent searched the web for recent API gateway patterns.
• The agent generated a draft response for ticket #4821.
• The workflow failed at the compliance review step.

Episodic memory is especially useful for debugging, auditing, and long-running workflows.

For example, if an agent makes a decision, I may want to know what happened right before that decision (e.g., What tools did it call? What data did it retrieve?).

This type of memory often benefits from structured storage.

For example:

Find all failed tool calls from the mortgage approval workflow in the last 24 hours.

That is a database query problem, not just a vector search problem.

Procedural memory

Procedural memory is about how to do things.

For example:

• When investigating a failed deployment, check logs first, then recent config changes, then dependency updates.
• When drafting a customer support reply, include the ticket summary, likely cause, recommended fix, and next step.
• When creating a database-aware agent, scan table comments, column comments, constraints, and recent workload patterns.

This is the kind of memory that helps an agent improve its process. That’s powerful because agents are often asked to operate in messy real-world environments. With procedural memory, it can reuse proven approaches.

The value extends beyond just knowing things to actually knowing how to proceed.

Entity memory

Entity memory stores facts about specific people, accounts, projects, systems, tickets, or objects.

For example:

• Angie prefers practical examples over abstract explanations.
• Customer Acme Corp has strict data residency requirements.
• Ticket #4821 is related to a billing reconciliation issue.

Entity memory matters because many agent tasks are scoped around a particular thing.

If I ask, “What do we know about Acme Corp?” I don’t want every memory in the system. I want memories attached to that customer.

This is also where memory safety becomes important.

Agents should not accidentally mix memories between users, customers, or projects. A memory system needs strong scoping so one user’s context does not leak into another user’s response.

Working memory

Working memory is the short-term scratchpad for the current task.

This is where the agent keeps temporary information while reasoning through a problem.

Working memory is usually not meant to last forever. It’s useful during the task, but it may not deserve to become durable memory.

If an agent stores every temporary thought as long-term memory, the memory store gets noisy very quickly. The agent may later retrieve half-baked assumptions as if they were facts, which is dangerous.

Not everything the agent observes or thinks should be remembered permanently.

Summary memory

Summary memory is one many agent users are familiar with. It deals with the problem of context windows being limited.

Even with large context models, you can’t keep appending forever. At some point, you need to compress.

Summary memory stores a compact version of a longer thread or context window. The original details can still live in the thread, but the prompt gets a smaller representation.

For example, instead of sending 80 turns of conversation, the agent might send:

The user is building a SaaS customer support agent. They prefer Python and FastAPI, deploy on OCI, and want the agent to retrieve past tickets before drafting replies. They are currently evaluating memory strategies for production usage.

Why memory is hard for agents

At first, memory sounds straightforward: store things, retrieve them later.

But the hard part is judgment, not storage.

What should be remembered? If the user says, “I usually prefer Python,” that’s probably worth remembering. If they say, “Let’s try Python for this one experiment,” maybe not. The agent needs to distinguish durable details from temporary context.

When should memory be updated? People change their minds, and systems and requirements change. If a user used to prefer FastAPI but now works mostly in Java, should the old memory be deleted, overwritten, or kept with a timestamp? A memory system needs a correction strategy.

How much memory should be retrieved? Retrieving too little means the agent misses important context. Retrieving too much means the prompt becomes noisy. This balance matters as more context isn’t always better.

How do we prevent memory leaks? If memories are shared across users, agents, or tenants, scoping is critical. The agent should only retrieve memories it’s allowed to use. This is especially important in enterprise systems where agents may operate across many customers, teams, or workflows.

How do we know whether memory helped? Memory should improve the agent’s behavior. It should reduce repeated questions, improve continuity, lower token usage, and help the agent produce more relevant responses. If memory just adds complexity without improving outcomes, it isn’t doing its job.

How Oracle is approaching agent memory

Richmond was gracious enough to share how Oracle is tackling this with the Oracle AI Agent Memory Package (OAMP), built on top of Oracle AI Database 26ai.

Yes, an AI database! Think of it as a database that can store and query the kinds of data AI applications need, not just rows and columns. That includes embeddings and JSON documents along with text search and regular SQL. These live together in the database, so an agent does not have to bounce between separate systems just to gather context.

The idea is to make Oracle AI Database the memory core for agents. Instead of stitching together a vector database, a relational database, a document store, and custom thread management, OAMP provides agent-friendly memory primitives on top of a database that already supports multiple data access patterns.

At a high level, OAMP gives you:

• Users and agents to scope memory ownership
• Memories for durable facts and extracted knowledge
• Threads for conversation history and continuity
• Context cards for compact, prompt-ready memory retrieval
• Summaries for long-running conversations
• Vector search for semantic recall
• Database-backed persistence so memory survives restarts

This matters because, again, agent memory is not only a vector search problem. Some memory needs semantic retrieval. Some need ordered reads or exact SQL filtering. A database-backed memory system gives you room to support all of those patterns.

Here’s a small example of what that looks like in code:

from oracleagentmemory.core import OracleAgentMemory

from oracleagentmemory.core.llms import Llm

client = OracleAgentMemory(

    connection=connection,

    embedder="text-embedding-3-small",

    llm=Llm("gpt-5.5"),

    extract_memories=True,

    schema_policy="create_if_necessary",

)

client.add_user(

    "angie",

    "Developer exploring agent memory patterns."

)

client.add_agent(

    "memory-demo-agent",

    "Assistant that demonstrates Oracle AI Agent Memory."

)

client.add_memory(

    "Angie is fascinated by agent memory and prefers practical examples over abstract explanations.",

    user_id="angie",

    agent_id="memory-demo-agent",

)

There are a few important ideas packed into this snippet.

The OracleAgentMemory client is the bridge between the agent application and Oracle AI Database. The database connection tells OAMP where memory lives. The embedder tells it how to turn memory text into vectors for semantic retrieval. The LLM enables automatic memory extraction and summary generation. And schema_policy="create_if_necessary" lets OAMP manage the underlying memory schema instead of making every application reinvent it.

The user and agent registration may look like simple setup code, but it’s actually part of the memory model. Memories need ownership. In a real system, you don’t want one user’s preferences showing up in another user’s session, and you don’t want memories written by one agent casually mixed with another agent’s context. The user ID and agent ID give the memory layer a way to scope what gets stored and retrieved.

The add_memory() call stores a durable fact. This is a piece of information the agent may need later, even if the exact conversation has moved on.

Given this, we can now recall memories.

results = client.search(

    "how should I explain this topic to Angie?",

    user_id="angie",

    max_results=3,

)

This search() call shows the part that makes semantic memory useful. The query doesn’t have to match the stored sentence exactly. We stored that I prefer practical examples, but we searched for how to explain something to me. Those are different words but related in meaning. That’s the point.

Threads and context cards

Durable memories are only part of the picture. Agents also need conversation continuity.

With OAMP, a thread can represent a real work session, such as an agent helping investigate a production issue:

from oracleagentmemory.apis.thread import Message

thread = client.create_thread(

    user_id="angie",

    agent_id="support-triage-agent",

)

thread.add_messages([

    Message(

        role="user",

        content="Customer Acme Corp is seeing intermittent checkout failures after the latest deployment.",

    ),

    Message(

        role="assistant",

        content="I'll check recent deployment notes, related incidents, and payment service logs.",

    ),

    Message(

        role="user",

        content="Focus on the payment gateway first. We saw similar timeout errors last quarter.",

    ),

])

This is much closer to how memory shows up in real agent applications. The useful context is not just that messages were exchanged. It’s that this thread is about Acme Corp, checkout failures, a recent deployment, the payment gateway, and a related incident from last quarter.

When it’s time to call the model, instead of passing the entire raw thread, you can ask for a context card:

card = thread.get_context_card()

The context card gives the agent a compact block of relevant memory to use in the next prompt.

Conceptually, the prompt becomes:

System: You are a helpful assistant. Use the provided memory context.

Memory context: [context card]

User: What did we decide earlier?

This is a much cleaner pattern than appending every message forever.

Automatic memory extraction

OAMP can also extract memories from conversation.

For example, if the user says:

I prefer Python over TypeScript for backend work. I usually deploy FastAPI apps on OCI behind an API gateway.

The memory system can extract durable facts such as:

The user prefers Python over TypeScript for backend work.

The user deploys FastAPI applications on Oracle Cloud Infrastructure behind an API gateway.

That means the application does not have to manually call add_memory() for every useful fact.

A smart thread can be configured like this:

thread = client.create_thread(

    user_id="angie",

    agent_id="memory-demo-agent",

    memory_extraction_frequency=2,

    memory_extraction_window=4,

    enable_context_summary=True,

    context_summary_update_frequency=2,

)

This tells the system to periodically inspect recent messages, extract durable memories, and maintain a running summary.

Here is where agent memory starts to feel more like a living part of the agent architecture vs just a data structure.

Teaching an agent about a database

One of the most interesting examples Richmond and I discussed was using memory to teach an agent about a database.

Imagine an enterprise data agent that needs to answer questions about a schema it has never seen before. Instead of fine-tuning a model, the agent can scan the database catalog and store what it learns as memory.

It might inspect:

• ALL_TABLES for table names and row counts
• ALL_TAB_COLUMNS for column names and types
• ALL_TAB_COMMENTS for human-written table descriptions
• ALL_COL_COMMENTS for column descriptions
• ALL_CONSTRAINTS for primary keys and foreign keys
• V$SQL for recent workload patterns

Then it can convert those technical details into natural-language memories.

For example:

Table SUPPLYCHAIN.VESSELS stores individual ships owned or operated by carriers. It includes vessel identifiers, carrier relationships, and operational metadata.

Now when a user asks:

Where would I find information about ships and carriers?

The agent can retrieve the relevant schema memory by meaning.

This is a beautiful pattern because it avoids one of the common traps with agents expecting the model to already know your private system.

It doesn’t. And that’s okay.

You can teach it by turning your system’s metadata into memory.

The more I learn about agent memory, the more I believe this will be one of the defining pieces of agent architecture.

Tool calling lets agents act. Planning lets agents decide what to do. Memory lets agents build continuity.

With memory, we can start designing agents that feel less like one-off prompt responders and more like persistent collaborators.

Of course, this also raises the bar. Memory has to be scoped, auditable, correctable, and intentionally retrieved. Bad memory is worse than no memory. So the challenge is not simply giving agents memory but giving them the right memory architecture.

Oracle’s OAMP approach is one way to make that system concrete: users, agents, memories, threads, context cards, summaries, and database-backed retrieval.

And while the implementation details matter, the bigger idea is that if we want agents to be useful beyond a single prompt, they need a way to remember.

Not everything. But enough to carry context forward.