In the Lab/Research

The Missing Piece in Harness Engineering: The Behavioral Harness

The LitmusLab Team

The LitmusLab Team

August 25, 2026·6 min read

Harness engineering has had a remarkable 2026. What started as a handful of practitioner blog posts has become one of the most discussed disciplines in software engineering. Martin Fowler published a full framework for it. Addy Osmani called it the most important shift in AI-assisted development. The community has converged on a powerful mental model: Agent = Model + Harness. If you're not the model, you're the harness.

The work being done here is genuinely valuable. Feedforward guides, feedback sensors, AGENTS.md files, hooks, computational controls, inferential evaluators... These are real tools that make coding agents more reliable, more consistent, and more trustworthy in production. The community has mapped this territory with impressive speed.

But there's a piece missing. And the most credible voices in harness engineering have already named it.


What harness engineering has solved

The maintainability harness is mature. We have access to type checkers, linters, architectural fitness functions, mutation testing, structural analysis, to name some of hte more common tooling. These computational sensors catch the things that can be checked deterministically, and they catch them reliably. Coding agents that would otherwise accumulate complexity, violate module boundaries, or write tests that don't actually test anything can be meaningfully constrained by a well-built maintainability harness.

The architecture fitness harness is developing. Teams are building skills that encode performance requirements, observability standards, and structural conventions. Agents are being steered toward the kind of code that's maintainable, not just code that compiles.

Both of these operate at the code layer. They're concerned with how the code is written and whether it meets structural standards.


What harness engineering hasn't solved

Birgitta Böckeler, whose harness engineering framework on martinfowler.com has become the reference document for the discipline, puts it plainly: the behavioral harness is "the elephant in the room."

How do we guide and sense whether the application functionally behaves the way we need it to?

The current answer in most teams: feed the agent a functional specification, check whether the AI-generated test suite is green, and combine that with manual testing. Böckeler's assessment of this approach: "it puts a lot of faith into the AI-generated tests. That's not good enough yet."

The CAAF paper (Harness as an Asset, 2026) makes this even more concrete. Across 80 trials, unsupervised LLM-based behavioral evaluation, where an LLM evaluates its own output without human-defined standards to evaluate against, achieved 0% paradox detection. The problem isn't LLM evaluation itself. It's LLM evaluation without separation between the agent that produced the output and the standards used to judge it. Matt Wear's principle #8 names this exactly: separate generation from evaluation.

When evaluation is grounded in human-defined quality standards, evaluated by a separate LLM against those standards, the picture changes. But most teams aren't doing that. They're relying on the same agent, in the same context, to evaluate its own work. This is where the research shows the approach breaks down.

The behavioral harness remains the hardest problem in harness engineering, and the current approaches leave it substantially unsolved.


Why the behavioral harness is different

The maintainability and architecture harnesses work because their success criteria are structurally definable. "No function over 50 lines" is a rule you can enforce computationally. "No circular dependencies" can be checked with a graph traversal. These rules live in the codebase, and tools can verify them against the codebase.

Behavioral correctness is different in kind. Take a simple example: a social app whose recommendation feature is supposed to suggest films based on a user's friends' watch history. "Recommend based on friends' history, not popularity" is not a rule you can express as a linter. It requires knowing what the product was supposed to do. Instead of what the code does, that needs to be expressed as what was intended.

This is the insight the RepoRescue paper surfaced so clearly: when agents are allowed to modify both the source code and the tests, a meaningful portion of "passing" comes from rewriting the tests to match whatever the code currently does. The code and tests are consistent with each other. They're not necessarily consistent with the original intent.

No amount of harness engineering at the code layer closes this gap. The gap is between the code and the intent. And the intent doesn't live in the codebase.


What the behavioral harness actually requires

If the problem is that intent doesn't live in the codebase and therefore agents can't be trusted to evaluate it, the answer is to put intent somewhere agents can't touch.

This means quality definitions that live outside the codebase. Intent statements, reference outputs, scenario-based evaluations, owned by the people who understand what the product is supposed to do: product managers, domain experts, QA leads, customer success teams. Written in terms of outcomes, not code structure. Evaluated against actual product output, not against the test suite.

A behavioral harness built this way has four properties the current approaches lack:

It's immutable to agents. Agents can rewrite tests. They can't rewrite a quality standard that lives in a separate system, owned by humans, and evaluated externally.

It's cross-functional. The PM knows what on-brand means. The QA lead knows the failure modes. The customer success team knows what users actually complain about. A behavioral harness gives all of them a seat at the table, not just the engineers who can read a test result.

It's continuous. Software behavior changes continuously. It's through deployments, dependency updates, refactors, and for AI-powered features, model updates and prompt drift. Running the harness only before release is inadequate. A behavioral harness that runs after every change catches regression before customers do.

It's outcome-oriented. The question isn't whether the tests pass. It's whether the product does what it was built to do, for the users it was built to serve.


The harness is almost complete

The harness engineering community has done remarkable work. The maintainability harness is real and working. The architecture fitness harness is developing. The behavioral harness is named, understood as unsolved, and waiting for tooling that addresses it properly.

The good news is that the infrastructure for this already exists in a different form: the evaluation frameworks that AI feature teams have been building to test non-deterministic outputs. Scenario-based testing, LLM-as-judge grounded in human-approved standards and separated from the generating agent, continuous evaluation pipelines. These are exactly the components a behavioral harness needs.

The insight that harness engineering surfaces is that these tools are for any team that cares whether their software does what it was supposed to do, and that can no longer rely on the test suite alone to answer that question.

The behavioral harness is the missing piece. And it's the piece LitmusLab is built to provide.


*Sources: Harness Engineering for Coding Agent Users (martinfowler.com); Agent Harness Engineering (Addy Osmani); Harness as an Asset: CAAF (arXiv:2604.17025); RepoRescue (arXiv:2607.01213); Awesome Harness Engineering (GitHub)