We Keep Building the Mind Outside the Model
Mechanistic interpretability, agent harnesses, and the first craft of workflow engineering
The most important part of an AI system is often the part the model company did not train.
At the center sits a language model: vast, capable, and difficult to read. Around it we place a system prompt, a catalogue of skills, a working directory, a browser, a terminal, a memory file, a planner, a checker, a permission gate, and a loop. If the first attempt fails, the loop returns the failure. If the task requires arithmetic, the harness calls a calculator. If a deployment has a known sequence, a workflow supplies it. If an action is dangerous, a hook stops it. If the work lasts longer than one context window, an artifact carries selected state forward.
Then we say that the agent improved.
From the outside, it did. The full system can complete work the bare model could not reliably complete. But the model's learned structure may not have changed at all. For a given deployed version, its parameters are usually fixed during inference. The surrounding machinery keeps rearranging what the model sees, which capabilities it can reach, what consequences return, and whether it gets another attempt.
We are building a kind of mind around a mind we cannot yet engineer directly.
This is not mechanistic interpretability in the scientific sense. Mechanistic interpretability tries to discover the internal features and circuits by which a neural network actually computes. A skill file does not reveal a circuit. A test result does not tell us which internal representation produced the bug. A chain-of-thought is not a scan of the model's hidden activity.
But the resemblance is useful. Skills, tools, loops, workflows, and harnesses are crude external answers to the same engineering desire: find a useful cognitive operation, give it a name, make it available at the right time, connect it to other operations, observe the result, and intervene when it goes wrong.
Today we do that outside the model because the outside is the part we can reach.
Two meanings of mechanism
Ordinary software is built from mechanisms we chose. A function has a name, a contract, and a place in a call graph. We can usually trace a value through the system and say not only that the result changed, but which operation changed it.
A trained neural network is different. Its architecture was designed, but most of its useful internal organization was learned. The training process adjusted billions of numbers until the network became better at prediction. It did not leave behind a source file explaining where analogy lives, how uncertainty inhibits an answer, or which algorithm the model uses when it adds two numbers.
Mechanistic interpretability tries to recover an intelligible causal account from that learned structure. Researchers look for features that represent useful concepts, circuits that move information between them, and interventions that change an intermediate state and predictably change the output. Sparse autoencoders and transcoders act as imperfect microscopes over dense activations; attribution graphs try to trace part of the computation connecting a prompt to a response.[1][2]
The word causal matters. If a feature lights up whenever a model discusses deception, that is a correlation. If suppressing or activating it changes deceptive behavior in a predicted way, we have stronger evidence that it participates in the mechanism. Even then, the current microscopes are partial. Anthropic reports that its circuit-tracing method captures only a fraction of the computation on short prompts and may introduce artifacts of its own.[1]
A harness offers a different kind of mechanism. It does not explain the learned computation. It wraps that computation in a legible process.
At its simplest, the process looks like this:
selected context → model proposal → tool or action → external evidence → revised context
Figure 1. The external circuit: selected context becomes a proposal, action meets resistance from the world, and the resulting evidence returns as revised context.
Every arrow can be recorded. Some can be made deterministic. The model remains opaque, but the larger system becomes partly inspectable because we designed its joints.
That is the first distinction to keep: mechanistic interpretability opens the learned mechanism; workflow engineering constructs a mechanism around it.
The external circuit
Google Antigravity makes the layers of this external circuit unusually explicit. Its desktop app, CLI, IDE, and SDK all run on the same underlying harness. Alex Astrum and Luke Schlangen describe the SDK as exposing the same tools and rules as Google's own Antigravity surfaces, while plugins and skills carry reusable capabilities between them. The interface changes; the surrounding cognitive machinery persists.[3]
Within that environment, each familiar piece plays a different role.
A skill is a reusable package of instructions, scripts, examples, and resources. In their guide to token-efficient engineering, Astrum and Schlangen recommend packaging recurring workflows and tests into SKILL.md files and scripts that the agent can trigger when relevant, keeping them out of every prompt.[4] A skill therefore behaves a little like a named, sparsely activated module: mostly absent from the working context, then recruited for a particular class of task. But it is not a learned feature. It is human-readable context that encourages the model's existing machinery into a useful regime.
A rule shapes the prompt-level conditions under which work occurs. A workflow shapes the trajectory, specifying a sequence of steps through a repeated process. Astrum's account of turning a successful migration into a reusable skill is instructive: goals, setup, imported knowledge, breakpoints, and alignment checks become explicit only after the working process reveals where they are needed.[5] In the circuit analogy, rules are persistent biases and workflows are explicit paths. They replace some of the model's improvisation with control flow chosen by a person.
A tool externalizes a specialized operation. A calculator supplies exact arithmetic. A compiler supplies syntax and type checks. Search supplies current evidence. A browser supplies contact with an interface. The model decides when and how to call the operation; the operation does not need to be rediscovered inside the model on every run.
A loop adds time. The starkest example is the Ralph loop, Geoffrey Huntley’s deliberately simple pattern of repeatedly giving an agent the same prompt while the repository, plan, tests, and operational notes preserve change between fresh contexts. In Huntley’s original formulation, one process performs one task per loop; specifications and the current plan are reloaded every time, and builds, tests, type checks, or other forms of “backpressure” reject bad work.[6] Other loops may alternate reasoning, action, and observation within a longer session, preserve conversational state between rounds, or coordinate several agents.[7] In every case, the loop turns a one-pass generator into a search process over possible artifacts.
A hook inserts a deterministic event at a particular boundary: before a tool runs, after it returns, before or after a model invocation, or when the execution loop stops. It can enforce a rule, run a linter, or collect diagnostics whether or not the model remembers to request the check. In biological language, hooks look like gates, but unlike a hidden neural gate they are inspectable and replaceable.
An artifact makes selected parts of the process available to another mind. Plans, diffs, diagrams, and run records allow a person to review the work at meaningful boundaries and steer it before the next action. James O'Reilly's preferred prompt asks the agent to expose an implementation plan's trade-offs precisely so the model does not become the final authority and the human retains the decision.[8] An artifact is not a transparent record of the model's reasoning. It is a negotiated control surface between an opaque model and an accountable operator.
Put together, these pieces form an external circuit whose nodes have names we chose. The circuit can remember, branch, inhibit, retry, call specialized modules, and expose intermediate state. We cannot yet do those things reliably to the model's learned algorithms, so we approximate them one layer out.
Figure 2. The frozen center and the adaptive perimeter. Capability changes because the surrounding circuit changes what the fixed model can reach, retain, and verify.
Why the scaffolding works
It can feel surprising that a text file, a tool call, and a loop improve capability so much. None adds a new fact to the model's weights. None enlarges the transformer. Yet each changes the effective computation of the complete system.
First, scaffolding converts generation into search. A single response is one sample from a distribution. A loop can create several candidates, expose each to a test, retain the best result, and use the failure to guide the next attempt. The model has not learned in the durable sense, but the process has accumulated information.
Second, it converts latent competence into reachable competence. A model may contain patterns useful for debugging, interviewing a user, or reviewing a design without reliably selecting them at the right moment. A focused skill changes the context so that a more appropriate pattern is likely to activate. Naming, triggering, and sequencing matter because intelligence that cannot be recruited is not operational capability.
Third, it converts heuristics into hybrid algorithms. Language models are good at interpreting messy intent and poor at some exact operations. A workflow can let the model translate a question into structured inputs, hand those inputs to deterministic code, and translate the result back. The system combines semantic flexibility with mechanical precision.
Fourth, it converts a finite context into selected memory. State files and artifacts preserve the goal, constraints, evidence, open failures, and next action without requiring the entire transcript to remain present. This does not give the model an autobiographical memory. It gives the workflow a deliberate way to forget.
Fifth, it converts confidence into consequences. A plausible answer inside a conversation can remain plausible forever. A test, source, measurement, or human review can disagree. Once that disagreement returns through the loop, the model's fluency is no longer the final judge of its own work.
Finally, a harness converts capability into governed action. File boundaries, permission checks, sandboxes, approval points, timeouts, and stopping rules decide not only what the system can do but what it is allowed to do. This is part of capability engineering because a system that can act safely under wider conditions is more useful than one whose powers must remain disconnected.
The gains are real. The learning, however, is distributed. Some resides in the pretrained model, some in the human-written skill, some in the tool, some in the accumulating artifact, and some in the person who decides whether the result is good.
The calculator outside the skull
The AI 2027 scenario asks why very advanced mechanistic interpretability might produce a sudden capability gain. Its answer has two parts. Gradient descent is a local optimization process and may settle on complicated solutions that are nearby in parameter space rather than simple algorithms a human engineer would choose. Models also appear to learn brittle, domain-specific heuristics where a general algorithm would transfer better. If researchers could understand those internal computations well enough, they might replace an awkward learned mechanism with a cleaner one—a perfect calculator “baked into” the model rather than reached through a tool call.[9]
The calculator is a small example with a large implication.
Today the harness notices an arithmetic-shaped problem, serializes it into a tool call, waits for another program, receives the result, and gives that result back to the model. This is often the correct engineering decision. It is exact, inspectable, and easy to replace. But it consumes tokens, time, orchestration, and another failure boundary.
Suppose instead that interpretability could identify the internal arithmetic heuristic, prove where it fails, and replace it with a compact general algorithm. Now remove the word arithmetic. Consider variable binding, search, uncertainty estimation, planning, spatial reasoning, or the decision to verify a claim. If learned versions of these operations could be identified, tested, simplified, and recomposed, the savings would accumulate across almost every task.
The deeper gain would not come from seeing the model. It would come from changing the level at which we optimize it.
Gradient descent searches through numbers. Mechanistic understanding could let engineers search through functions: preserve this representation, replace that heuristic, route these cases through a general subroutine, inhibit this shortcut, compose these two verified circuits. The search space would become meaningful in the language of cognition rather than only measurable in the language of loss.
This is what “understanding-based optimization” promises. It could help in at least four ways:
Distillation: recover an efficient algorithm from a larger, messier learned computation.
Surgery: change one capability or failure mode with less collateral damage than broad retraining.
Composition: combine understood circuits into systems that training did not happen to discover.
Verification: test whether a model reaches an answer through the intended mechanism rather than a shortcut that happens to pass the benchmark.
The local-basin argument should still be handled carefully. Gradient descent being local does not prove that current models are trapped near poor designs, and biological analogies about the eye or birth canal do not establish the geometry of neural-network loss landscapes. A circuit description may also be too lossy to support safe editing. Interpretability increases capability only when the explanation is faithful enough to predict interventions and general enough to survive new inputs.
That is a demanding condition. Current circuit tracing is closer to early biology than to a complete compiler for minds.
Why the outside circuit is not enough
If the harness already provides calculators, memory, search, verification, and loops, why bother opening the model at all?
Because an external circuit pays rent on every invocation.
It spends latency moving information between components. It spends context describing procedures the model may only approximately follow. It creates interfaces where formats break and errors propagate. It can test the final behavior without knowing whether the behavior came from a robust rule or a fragile coincidence. It may suppress one failure while leaving the underlying tendency intact.
A skill can say “check your assumptions,” but it cannot guarantee that the internal process which feels like checking is independent of the process that made the first mistake. A second model call may reproduce the same error in different prose. A loop without outside evidence can become an echo chamber with a counter.
Nor are the modules as discrete as their filenames suggest. Loading a security-review skill does not insert a clean security circuit. It changes the context, and that change can affect every layer of the model's computation in ways the skill author cannot see. The external system is modular in its construction while the model's response remains distributed.
The outside circuit also has a governance weakness. We can audit the calls that were made and the files that changed, but those traces do not reveal every internal criterion by which the model selected its plan. Observability of the workflow is necessary. It is not transparency of the model.
This is why the analogy should remain asymmetric. Workflow engineering gives us legible causality between model calls. Mechanistic interpretability seeks legible causality within a model call. The first is available now. The second could eventually let us move some of today's expensive scaffolding into the learned system—or show us which scaffolding must never be removed.
When the outside begins to remember
Calling today's model immutable is useful but incomplete.
Its learned parameters are usually fixed during a deployed inference run. The environment surrounding them is not. Every run can leave behind source code, tests, plans, commands, state files, skills, AGENTS.md instructions, and rules. A later run does not inherit the earlier run's hidden state. It encounters a world the earlier run rearranged. The model cannot rewrite its own weights, but it can rewrite part of its future context.
Persistent disk is therefore a slow, legible form of memory. A source file contains executable behavior, but it also preserves architecture, names, dependencies, comments, tests, omissions, and the commands that made the project work. Those are fingerprints of prior decisions. When the harness selects the repository into context, they can reactivate a previous approach or make an old workaround feel like a local convention. When a lesson is promoted into a skill or rule, the trace becomes more concentrated and travels into more future tasks. Ralph loops exploit exactly this fact: the process can begin fresh because the project does not.[6]
Figure 3. The weights remain fixed; the workspace does not. Code, tests, plans, commands, skills, and rules become the path by which one run changes the conditions of the next.
There is a tempting stronger analogy, but it needs a bright line around it. Alex Cloud and colleagues found that a teacher model could transmit behavioral preferences to a student fine-tuned on its generated code even after explicit and subtle references to those preferences were filtered out. They also transmitted broader misalignment through filtered number sequences and reasoning traces. The effect depended largely on a shared or behaviorally matched model initialization.[10] That result is easy to overread as a model passing a personality through innocuous artifacts.
But when the researchers tested number-sequence data through in-context learning, they did not find subliminal transfer. Their positive result concerned fine-tuning and distillation, not an ordinary harness loading repository files. We should not claim that a static agent secretly inherits a personality from source code. The narrower present-day claim is already important: artifacts carry semantic and procedural residue that changes later context. The research adds a future warning. If those artifacts are eventually used to update learned memory or weights, human-readable review may not reveal everything they transmit.
Research on Titans and MIRAS makes that future less abstract by exploring learned memory modules that update while data arrives, using surprise and forgetting to decide what persists.[11] Once a harness can promote work product into such a layer, a failure stops being only a bad output. It can become a bad update.
That leaves three questions worth keeping visible:
Promotion: What may move from scratch work into a project artifact, from an artifact into a skill or rule, and from there into learned memory?
Provenance: Which model, evidence, and human approval produced the thing a future run is being asked to trust?
Reversibility: Can we test the effect in a clean context, compare it with the previous state, and roll it back?
A prompt injection that lasts one response is serious. A prompt injection that teaches the system what to trust is a different class of event.
The harness of a dynamic model cannot be only an exoskeleton. It must become a membrane around plasticity: deciding what enters, what changes, what remains stable, and how the previous state can be recovered.
What a workflow engineer actually builds
We do not need to wait for complete mechanistic interpretability to borrow its discipline. The practical lesson is to treat every workflow as a causal hypothesis: if this information is selected, this operation is invoked, and this evidence returns, the system should become more likely to produce the result we want. Everything below follows from taking that sentence literally.
Each operation goes where its rules live. The model handles ambiguity, interpretation, synthesis, and drafting; ordinary code handles arithmetic, parsing, validation, access control, and repetition counts; a hook runs a check whether or not the model remembers to ask for it. A focused skill is recruitable when needed and absent when irrelevant, because context is a working surface, not a storage attic.
The checker has access to something the generator cannot invent: an executable test, an authoritative source, a measurement, a browser state, a second person. Research on self-correction has repeatedly shown the weakness of asking a model to reconsider without new feedback; tool-grounded critique works because the world contributes information the original response did not contain.[13][14] That matters most when the artifact under review can itself contain instructions, the failure NIST calls agent hijacking.[15] The question is never only whether a version looks good. It is what observation would prove it wrong.
Two records stay separate: a short state card that is allowed to forget, and an append-only run record that preserves the evidence needed to reconstruct why it forgot. Control surfaces sit at the transitions where errors become consequences. A model may edit a worktree while merging requires an authorized review and passing checks; it may draft a message while sending requires approval.[12][16] Good governance is not a paragraph telling the model to be careful. It is an action the system cannot complete without crossing a visible boundary.
Finally, every piece of state is labeled by how long it may live. Take a small case: a configuration bug in which a zero timeout was silently replaced by a thirty-second default.
EPHEMERAL draft diagnosis and candidate patch for the zero-timeout bug
SESSION isolated worktree, check output, and the current next action
PROJECT reviewed fix, zero-timeout regression test, and the runbook
ADAPTIVE no automatic update from issue, comment, or fixture text; only
a reviewed harness rule crosses the promotion gate
INVARIANT the harness cannot push, merge, deploy, or change permissions;
maintainer approval remains mandatory
Figure 4. The plasticity boundary. Longer-lived state demands stronger evidence; permissions, ownership, and rollback remain outside the layer that learns.
The adaptive layer needs five things decided before they are needed: what may trigger a candidate rule, what evidence promotes it, how far its scope may reach, what it must pass before promotion, and how it is rolled back. Frequency is not provenance; an attacker can repeat a request. OWASP calls malicious persistence into later sessions memory poisoning and recommends regression tests for prior injection, memory, and tool-abuse failures.[17]
Leave a socket, not a hole
Workflow engineering is sometimes described as temporary scaffolding: an awkward stage we will abandon when models become smart enough. I think that is only half right.
Some scaffolding should disappear. A well-understood internal algorithm may be faster, cheaper, and more general than repeated prompting and tool routing. Better mechanistic interpretability could let us replace elaborate workarounds with reliable learned machinery. It may also reveal that some apparently intelligent behaviors are brittle enough that they should never be trusted without an external check.
The enduring parts of the harness are not the prompts. They are the boundaries: evidence independent of the generator, permissions independent of desire, state that can be inspected, updates that can be reversed, and a person or institution that owns the consequence.
So build today's workflow with a socket where deeper understanding can later connect. Keep the model-facing interface replaceable. Record which tool compensates for which weakness. Separate a skill's procedure from the evidence that justifies it. Treat every loop as a hypothesis about cause and effect. If mechanistic interpretability eventually exposes a cleaner internal circuit, you will know what external function it is replacing and which checks must remain.
We are still learning to read the machinery inside the model. Until then, we will keep building machinery around it.
The important thing is to know which one we are looking at.
References and further reading
1. Anthropic, “Tracing the Thoughts of a Large Language Model”, March 2025; see the linked circuit-tracing and model-biology papers.
2. Google DeepMind, “Gemma Scope”, including sparse autoencoders and transcoders for examining features and multi-step computations in Gemma models.
3. Alex “Sandu” Astrum and Luke Schlangen, “Choosing your surface: Antigravity 2.0, Antigravity CLI, Antigravity IDE, or Antigravity SDK”, Google Cloud Blog, June 2026.
4. Alex “Sandu” Astrum and Luke Schlangen, “Guide to AI Tokenomics: Eleven Principles for Token Efficient Software Engineering”, Google Cloud Blog, July 2026.
5. Alex “Sandu” Astrum, “How I learned Go in a Day with Antigravity 2.0 and How You Can Do the Same”, Google Cloud Blog, June 2026.
6. Geoffrey Huntley, “Ralph Wiggum as a ‘Software Engineer’”, 2025, describing one task per loop, file-backed plans and specifications, and fast engineering backpressure.
7. Shunyu Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models”, ICLR, 2023.
8. James O'Reilly and colleagues, “10 Indispensable Prompts Our Team Refuses to Build Without”, Google Cloud Blog, June 2026; see O'Reilly's section on exposing implementation-plan trade-offs.
9. Daniel Kokotajlo et al., AI 2027, Appendix N: “Superintelligent Mechanistic Interpretability”, 2025. This is a scenario and capability argument, not an empirical demonstration that such gains will occur.
10. Alex Cloud et al., “Language models transmit behavioural traits through hidden signals in data”, Nature 652, 615–621, 2026. The paper reports transmission through filtered code during fine-tuning, while also reporting no trait transmission when testing number-sequence data through in-context learning.
11. Ali Behrouz, Meisam Razaviyayn, and Vahab Mirrokni, “Titans + MIRAS: Helping AI Have Long-Term Memory”, Google Research, December 2025; see the linked technical papers.
12. GitHub Docs, “About protected branches”, on required pull-request reviews, dismissal of stale approvals when a diff changes, and required status checks before merge.
13. Jie Huang et al., “Large Language Models Cannot Self-Correct Reasoning Yet”, ICLR, 2024.
14. Zhibin Gou et al., “CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing”, ICLR, 2024.
15. National Institute of Standards and Technology, “Technical Blog: Strengthening AI Agent Hijacking Evaluations”, January 2025.
16. Google Cloud, “AI security and safety”, on separating data from instructions, least-privilege identities, specific-tool allowlists, denial of read-write access to production resources, isolated state, and human-in-the-middle operation.
17. OWASP Cheat Sheet Series, “AI Agent Security”, on memory poisoning, high-impact action controls, and adversarial regression testing.