The AI harness: the code around the model is what makes it reliable

Adding a model call to a product takes an afternoon. Making that call dependable enough that people act on its answers takes engineering. Most of that engineering is not in the model at all. It is in the code around it: the harness.

The idea that helps most is a simple reframing: the model is a component; the harness is the product. Models get replaced, upgraded and repriced. The harness is what your users actually experience, and what decides whether a bad answer reaches them.

What the harness is made of

• Prompt templates. Prompts live in version control, with named variables, not as strings scattered through the code. A change to a prompt is reviewed like a change to code, because it is one.
• Structured outputs and schema validation. Ask the model for fields, not paragraphs, and check every response against a schema before anything uses it. A missing field or a value out of range is caught in code, not discovered by a user.
• Tool calling. When the model needs data or needs to act, it requests a defined tool with typed arguments. Your code runs the tool, checks permissions and returns the result. The model asks; the harness decides.
• Retries and timeouts. Model providers have slow moments and outages. Every call gets a timeout, a bounded number of retries with backoff, and a clear outcome when they run out.
• Fallbacks. Decide in advance what happens when the model cannot answer in time: a simpler method, a cached result, or an honest “not available yet”. Never a silent blank, and never a guess dressed as an answer.
• Logging and provenance. Record what went in and what came out: the inputs, the prompt version, the model, the result. When someone asks why the system said something, the answer exists.
• Tests. A set of known inputs with expected outputs, run whenever the prompt, the model or the code changes, so a regression is found before release.

A harness in practice

On the AI sales-coaching product we built, the model scores sales calls, and those scores reach people who make decisions with them. The harness around it follows the list above:

• The model returns structured fields: a score, the reasons for it, and evidence quoted from the call. A quote can be checked against the call itself; a vague impression cannot.
• The output is validated before it is stored or shown.
• A rule-based scorer runs alongside the AI. When the model is slow or unavailable, the rule-based score takes over, and it is labelled as a fallback, so nobody mistakes it for the full AI assessment.
• Every score records its inputs, so any result can be traced back to what produced it.

None of these pieces is exotic. Together they mean an outage at a model provider degrades the product instead of breaking it, and a strange score can be investigated instead of argued about.

Why the harness outlasts the model

A good harness makes model changes routine. With schemas, tests and provenance in place, trying a newer or cheaper model is an experiment with a clear pass or fail, not a leap of faith. Without them, every model change is a risk nobody can measure, so teams either stop improving or ship changes blind.

It also makes the system explainable to the business. Managers do not need to understand the model; they need to know that its answers are checked, that there is a defined behaviour when it fails, and that any result can be traced. That is a property of the harness, and it is the part we put the most care into when we build AI on a company’s own data.