The model is not the system: why powerful AI still needs a harness

Every new generation of models invites the same fantasy: now the technology has become so capable that the rest of the engineering has become detail. The demo understands the intent, writes code, uses tools, and appears to complete a task end-to-end. The question quickly changes from “does it work?” to “how much can we automate?”

It is in this interval that promising projects become fragile systems.

The model produces possibilities. The system needs to transform a possibility into an accepted outcome, in the right environment, under explicit permissions, with known cost, verification and accountability. A better model can raise the ceiling. It does not alone define what should be done, what data it can access, when it needs to stop or what counts as done.

This layer around the model is the engineering harness.

The category error

Comparing models helps you choose a capability: reasoning, code, vision, context, latency, or cost. But a company does not buy abstract capacity. It needs a change to be delivered, an analysis to be reliable, a service to be resolved or a decision to be made within limits.

Between the request and the result there are questions that a model benchmark does not answer:

• who defined the intention and acceptance criteria;
• what sources, repositories, tools and environments can be used;
• which actions are reversible and which require confirmation;
• how a long run recovers context after failing;
• who checks behavior, safety and impact;
• what evidence links the result to the version, control, and approver;
• how a failure becomes learning, and not just a new attempt.

When these questions are implicit, the prompt becomes policy, the model becomes a security boundary and a convincing answer becomes synonymous with delivery. There are three responsibilities that the model should not accumulate.

Harness is the agent's work system

In FORGE, the harness is organized into six planes. They don't have to be six services or six products; there need to be six visible responsibilities.

1. Intention

Transforms desire into objective, scope, restrictions and definition of doneness. “Improve checkout” is not a specification. It is an opening for the system to discover the wrong problem very quickly.

2. Policy

Defines authority: what can be read, written, executed, published or sent; what data is prohibited; when the task should stop; and which decision remains human. Policy is not a recommendation in the prompt. It is a rule that the environment and tools can impose.

3. Execution

Provides tools, sandbox, session identity, durable memory, and resumption mechanism. Work must survive a process crash and remain assignable. Anthropic engineering research on managed agents describes the separation between harness, sandbox, and session logging; The sources analyzed by FORGE argue that history, orchestration policy, and isolation must be explicit boundaries, and that model or channel cannot become the security boundary. [HARNESS31-C2]

4. Verification

Test different properties with different controls. Unit testing, API contract, behavior eval, secret scanner, architectural review, and actual navigation are not repeated vows of the same thing. Each control needs to state what it observes and what it does not prove.

5. Evidence

Ties the result to its origin: commit, contract version, report, hash, source, executor, verifier, validity and residual. “Passed” without subject and without artifact is just a sentence.

6. Feedback

Converts correction, incident and exception into durable improvement. The loop only closes when the failure produces a test, a guardrail, a decision, a context update, or an explicit limit.

A simple example: changing an authentication flow

Imagine asking an agent: “add social login”. A capable model can locate files, generate the integration, and produce a screen. Still, the result may fail for reasons outside of code generation:

• the chosen provider does not meet the product identity framework;
• a secret ended up in the browser bundle;
• the callback accepts an improper origin;
• there is no tenant isolation test;
• mobile experience does not cover error and recovery;
• the documentation states that the function is ready before deployment;
• no one approved the attack surface change.

The harness changes the job. Intent requires scenarios and criteria. The policy restricts secrecy and permission. Execution occurs in an isolated branch and environment. Verification combines contract, testing, security and actual flow. The evidence links everything to the same commit. Human approval occurs where the risk becomes external or difficult to reverse.

The model remains essential. It just stopped being confused with the entire product.

More layers don't mean a better system

There is also the opposite error: responding to every failure with more agents, more reviewers and more loops. Complexity can improve an outcome while making cost, latency, and diagnosis worse.

In a long-term application development experiment, Anthropic reported better results with planner, generator and evaluator, but with materially greater cost and duration. This result pertains to the task, model, and design tested. He supports the need for evals and ablation on a case-by-case basis; not a rule that every task needs a multi-agent squad. [HARNESS31-C3]

A component should remain in the harness because it proves measurable value or controls real risk. With each leap in model capacity, it is worth asking:

• this step still avoids a failure class;
• the same control can be made simpler;
• the gain compensates for cost and latency;
• the verifier is actually independent of the executor;
• removal worsens results in realistic evals.

The harness is adaptive architecture. It is not a permanent collection of compensations for limitations of an old version of the model.

Tool is not authority

An agent can operate on the endpoint, in the browser, in Slack, in an API, or in a portal. These channels are integration surfaces. They should not grant, for convenience, authority over the entire organization.

The sources examined by FORGE point to managed and scoped shared capabilities, maintaining web, chat and future interfaces as replaceable channels. They do not define a universal authorization model; deny-by-default, revocation, and testing remain required on each product. [HARNESS31-C4]

This distinction helps to switch models without rebuilding security. The roster can change depending on task, cost and risk. The identity it runs, the resources it accesses, and the publishing gates remain stable.

What business people need to demand

An AI decision becomes more mature when it moves beyond “what is the best model?” and responds:

1. Which outcome will be accepted and by whom?
2. What baseline exists today for time, quality, cost and risk?
3. Where does the AI suggest, execute, verify, or need to stop?
4. What data and actions are out of scope?
5. What evidence accompanies each delivery?
6. How do we correct, remove or reverse the result?
7. What happens when the model, tool or environment changes?

These questions allow you to compare systems, not demonstrations.

What technical teams need to make explicit

For engineering, the useful minimum is an operating contract:

• input and output with schema;
• identity and capabilities by task;
• versioned and retrievable context;
• sandboxing and secrets out of reach of untrusted code;
• attempt limits, cost and duration;
• evals and tests linked to the risk profile;
• provenance of the artifact and verifiers;
• human gate for irreversible or external action;
• sufficient telemetry to explain failure and consumption;
• rollback and public withdrawal when applicable.

This doesn't require starting with a huge platform. It requires that the first version does not hide the boundaries that will need to exist when the experiment becomes an operation.

A seven-question test

Before calling a solution an “agent in production”, try responding without resorting to the model name:

1. What is he authorized to do?
2. How do we know we did the right thing?
3. Who can stop or revoke the execution?
4. What survives if the process crashes?
5. What evidence links the result to the executed version?
6. Who responds when the outcome affects someone else?
7. How does the system learn without silently rewriting the past?

If the answers don't exist, perhaps there is an impressive capacity. There is still no reliable system.

Conclusion

Better models matter. They expand the space of what can be delegated and allow you to simplify parts of scaffolding. But the more capable they become, the greater the effect of a mistaken permission, an ambiguous goal, or a circular check can be.

The harness does not compete with the model. It converts capability into governable work: intent, policy, execution, verification, evidence, and feedback.

The model generates possibilities. The system is what makes one of them acceptable, assignable and correctable.

Next reading: Accountable human, executing AI, the operational contract that organizes authority and execution within the squad.