In June 2026, Peter Steinberger reported that his system spent $1,305,088.81 over 30 days and processed more than 603 billion tokens across roughly 100 Codex instances, with a team of three people running it. Not a benchmark, not a weekend experiment. A real production workload, big enough to expose every weakness an agent can have.
Most of the reaction went straight to the obvious question: which model powered it? Fair place to start. Better models write cleaner code, follow instructions more reliably, and recover better when a task goes sideways. If you're building coding agents, picking the strongest model available feels like the decision that matters most.
But at $1.3 million across 100 Codex instances, there's a more interesting question.
What was the harness?
Almost nobody asked it, even though it's the question that explains how the system around the model keeps long-running work on track. It's also what supports hundreds of parallel agents at once and what. It's also what supports hundreds of parallel agents at a go and what keeps producing useful results after thousands of tool calls and decisions. None of that is a property of the model alone. It's the job of the production harness and the discipline of building one well now has a name: agent harness engineering.
Frontier models keep pushing what's possible in autonomous problem solving and that progress is real. But production systems fail for reasons benchmarks rarely capture. Once an agent is running over long horizons, calling tools, coordinating with other agents, model selection stops being the whole decision. It becomes one part of a larger system, and everything from here on assumes you're building a production agentic system, not running a one-off agent taskeverything from here on assumes you're building a production agentic system, not running a one-off agent task.
TL;DR
Model choice isn't what determines whether an AI agent ships, the harness around it is.
Two studies published this year measure different constraints, not competing claims, and together they show harness quality moving output independent of the model.
Teams shipping reliably converge on the same pattern: state lives outside the model, verification is separate from generation and context loads before the first prompt.
The real question isn't which model to use. It's whether the harness around it is doing enough of the work.
What the AI agent harness does that the model cannot
A model is responsible for reasoning, but a production AI agent harness has a different job. It's the runtime around the model that coordinates the work needed to turn the model's reasoning into reliable execution. Depending on the system, that includes assembling instructions and context, managing tool access and execution, maintaining persistent state and integrating with capabilities such as model routing, verification and observability. Those responsibilities sit outside the model because they're systems engineering concerns, not reasoning tasks.
Looking at it this way explains why the system around a model keeps growing. Teams rarely add better context management, a sandbox, richer observability or stronger guardrails because a framework recommends them. Those capabilities usually appear after the same production issue keeps recurring, and the fix becomes permanent instead of something an engineer remembers to do manually.
Every harness component exists because something failed first
None of this got built as one piece. It works the same way separating a production system into layers does for application code: each component owns one job, so a failure in one place doesn't take down everything downstream with it. A system prompt failing doesn't corrupt the sandbox. A sandbox breach doesn't erase your observability logs. That separation is the entire point, and it's why harness designers keep converging on similar architectures regardless of which model or framework they use.
Thinking about it this way shifts the conversation from features to harness-level failure modes. Instead of asking what a component does, ask what breaks when it isn't there. That's usually the fastest way to audit an agent system, since every missing layer leaves behind a recognizable pattern.
| Harness component | Purpose | If it's missing, you'll notice... | What production teams use |
|---|---|---|---|
| System prompt | Defines the agent's role, operating rules, constraints, and persistent instructions. | The same prompt produces different behavior across runs, even though nothing appears to have changed. | Versioned, testable prompts treated as code rather than one-off instructions typed into a chat window. |
| Tools and tool execution | Connects the agent to APIs, terminals, databases, web search, web interaction, and external services. The harness decides when to use a tool instead of asking the model to reason about the answer directly. | The agent confidently says it completed a task, but nothing actually happened because the tool call failed, was never executed, or returned unexpected results. | Tool calls that return structured, verifiable results, with retries, tool call offloading, and explicit failure states instead of silent ones. |
| Sandbox | Runs agent-generated code inside an isolated operating environment for safe code execution. | You're reluctant to let the agent execute code because one mistake could modify your local machine, shared infrastructure, or production systems. | A sandboxed environment that enforces network isolation, limits permissions, and isolates execution from production infrastructure. |
| Filesystem and durable storage | Preserves files, artifacts, and intermediate work across sessions instead of relying entirely on the prompt. | Long-running tasks restart from scratch, generated files disappear, or the agent loses work between sessions. | State written to disk or durable storage and reloaded between sessions instead of relying on increasingly large or multiple context windows. |
| Memory and context management | Preserves relevant information while preventing unnecessary history from filling the context window. | The agent forgets earlier decisions, repeats completed work, or gradually loses sight of the original objective as the context window fills. | Compaction, summarization, just-in-time context injection, and offloading large tool outputs to storage instead of keeping everything in the context window. |
| Feedback loops and self-verification | Evaluates outputs and agent behavior before moving to the next step. | Small agent failures go unnoticed until several steps later, when fixing them is far more expensive than catching them early. | An independent verification step, often another agent or a predefined test suite, that checks work before it's marked complete. |
| Guardrails and human-in-the-loop controls | Applies approvals, policies, and safety checks to sensitive operations. | You don't trust the agent with production systems because there's no safe point to review, approve, or stop risky actions. | A defined approval flow for irreversible actions, with the harness pausing instead of guessing. |
| Observability and logging | Captures execution traces, decisions, tool calls, and execution history. | An agent fails in production, and all you know is that it failed. There's no trace showing which decision, tool call, or interaction caused the problem. | Full execution traces for every agent run, with cost and latency metrics attached so failures can be reconstructed after the fact. |
The pattern isn't subtle once you look for it. Context rot isn't a prompt engineering problem; it's a context management problem and long-horizon execution failures don't happen because the model suddenly gets worse at reasoning. They happen because the agent context around it wasn't built to sustain autonomous work over time. The same logic holds when agents call the wrong tool repeatedly, lose intermediate work or become impossible to debug after something breaks. These are harness problems with harness solutions.
Once you're looking at agents through that lens, the question changes. Instead of asking whether a different model would have performed better, you start asking which part of the system let the failure happen in the first place. That's the more useful question to sit with, because every missing component leaves behind a specific, documentable failure mode.
The evidence the model-first camp has to explain
If you assume better models are the reason agents perform better, then improving the model should explain most of the gains you see in production. That's a reasonable assumption. But it's difficult to square with two of the most interesting pieces of evidence published this year.
METR measures the ceiling
Start with an evaluation of coding agents from Model Evaluation and Threat Research (METR). Claude Code outperformed a simple Reason-Act (ReAct) loop 50.7 percent of the time. That result deserves your attention because it wasn't comparing raw models in isolation; it evaluated complete agent systems performing the same software engineering tasks. The same research also found Codex underperforming Triframe, another generic scaffold METR uses, winning only 14.5 percent of the time. If you build a purpose-built harness and it loses to a generic one, that's not a model problem. It's a harness-design problem, and it's some of the clearest evidence you'll find that the system wrapped around a model can hurt as easily as it can help.
METR's research also identifies a constraint you'll run into eventually: as tasks become longer and require more sustained reasoning, today's coding agents become less reliable. They lose coherence, recover less effectively from mistakes and eventually struggle to complete work that stretches over extended periods of execution. Every agent you build eventually reaches a practical time horizon, where reliability begins to decline as task duration increases.
Anthropic measures quality within the ceiling
Anthropic's engineering postmortem investigates a different variable, one worth separating from the ceiling above. It asks what changes when the production harness changes while the underlying model stays the same.
The investigation began after users reported that Claude Code's coding quality had noticeably declined. Anthropic traced the regressions to three changes outside the model itself: lowering the default reasoning effort to reduce latency, a context-management caching bug that repeatedly discarded prior reasoning after idle sessions and a system prompt change intended to reduce verbosity that unexpectedly reduced coding quality. Throughout the investigation, the underlying model never changed.
Two constraints, one production system
At first glance, the two studies appear to point in different directions. If you assume improving the production harness leads to better results, you might wonder why longer tasks still cause agents to fail. If you accept that today's models have a practical time-horizon ceiling, you might wonder how changing the harness can noticeably improve or degrade performance without changing the model.
The answer is that the studies are measuring different things. METR looks at how reliably coding agents complete tasks as they get longer and more demanding. Anthropic looks at how changes to the harness affect output quality while the model stays the same. They're answering different questions, which is why the findings complement rather than contradict each other.
A side-by-side comparison makes the distinction easier to see.
| Question | METR | Anthropic |
|---|---|---|
| What is being measured? | How reliably coding agents perform as tasks become longer and more complex. | How changes to the production harness affect output quality while keeping the same model. |
| What changed? | The complete agent system being evaluated. | Reasoning effort, context management, and system instructions, while keeping the underlying model constant. |
| What does it tell us? | Today's coding agents have a practical time-horizon ceiling. | Production harness engineering directly affects output quality within that ceiling. |
Read together, these findings describe orthogonal constraints. If you're trying to find where your production bottleneck actually lives, this is why it matters: one measures the limits of long-horizon execution, the other measures the quality of execution within those limits.
Anthropic's postmortem strengthens the argument because the underlying model remained the same throughout the investigation. So when your production harness changes while the model stays constant, it's difficult to argue that your production performance is determined by model capability alone.
The takeaway isn't that models have stopped mattering to you. Better models can extend what your agents are capable of attempting. Better production harnesses determine how reliably those capabilities become your production outcomes. Today's models still suffer from context loss, execution drift and long-running coordination failures that only your surrounding system can address.
What the teams shipping in production actually built
Those findings point to the same conclusion: if reliability breaks outside the model, the harness has to own the fix. You don't solve reliability by switching models alone, not once you're shipping agents into production. You move critical responsibilities out of the model and into the harness instead. A model is good at generating the next action, but production teams don't rely on it to preserve state across long-running work, validate its own output or reconstruct an unfamiliar codebase before it starts making changes. Those responsibilities move into the surrounding system instead.
State lives outside the model
Don't ask the model to remember everything. Store task state, intermediate results, execution history and artifacts independently. That way, work can continue when an agent stops, retries or hands off to another agent. This also makes collaboration practical: instead of passing an ever-growing context window between agents, each one reads the current state, contributes its work and writes the result back.
Feedback loops are separate from generation
Keep evaluation outside the generation loop entirely. Generating code and verifying it are different jobs. The model produces an answer. The harness checks whether it compiles, passes tests, satisfies project rules or introduces regressions before deciding whether execution continues, retries or stops.
Context management starts before the first prompt
If you're prototyping, you might let the agent begin with a task and discover the rest as it goes. A production harness does the opposite: it assembles the context the agent needs before work begins. That can include project rules, repository structure, dependency information or prior task state, depending on the system. Starting with the right context reduces unnecessary rediscovery, limits context rot and keeps the agent coherent over long-running tasks. In multi-agent systems, the harness also propagates dependency changes automatically, so every agent starts from the same understanding.
That last point matters more than it sounds like it should, and AgentField's own postmortem shows you exactly why it's not optional. A pull request built by more than 30 parallel agents passed every test it was given, and it still broke in production, because a downstream agent had mocked a dependency that an upstream agent never actually exported. Every individual agent did its job correctly. Nothing in the harness gave them visibility into what other agents were doing. That's not a model failure, and better reasoning wouldn't have caught it. It's a harness failure, and it gets more expensive, not less, the more parallel agents you add without also building the layer that lets them see each other.
Forward-deployed engineering addresses this directly, since putting delivery ownership close to the work tends to surface exactly this class of problem before it ships rather than after. One team's approach to that model shows what harness ownership looks like in practice, and it connects to a broader point about scaling AI development past prototype speed: informal iteration works right up until it doesn't, and durability needs structure you can hand off.
What happens before the first agent action
This startup sequence isn't a framework feature. It's an architectural decision. By initializing state, rules, context and verification before work begins, the harness removes problems the model would otherwise have to solve mid-execution.
The common pattern isn't a better model. It's a harness that prepares the work before the model ever begins it, and you still get to care about model quality without expecting it to solve problems that belong to the harness.
What the reader does next
Start by auditing your harness against the eight production components. The missing or weakest one is almost always your bottleneck. Then move agent state outside the execution loop using a single state file with Done, In Progress and Next sections that the agent reads at the start of a session and updates before it ends. Before scaling parallel agents, address context rot by implementing context compaction and moving large tool outputs into the filesystem instead of leaving them in the context window. Finally, preload project rules, repository context and dependency relationships, so every session starts with working knowledge instead of rediscovering the codebase from scratch.
If you're building AI agents for production, the better question is whether the harness around the model is doing enough of the work.