The Harness Is Not Plumbing

Most enterprises are still evaluating AI the way they evaluated cloud a decade ago: pick the best provider, sign the contract, delegate the integration. The model is the decision. Everything downstream is engineering detail.

That framing is already showing its limits, and the companies discovering this first are the ones whose AI spend has quietly outrun their AI value.

The layer between your application and the model (the wrappers, routers, evaluators, context managers, guardrails, and telemetry that surround every call) is starting to get named. Thoughtworks has taken to calling it the harness. Others are calling it the new battleground. Whatever you call it, it is the place where cost, latency, quality, compliance, and vendor sovereignty are actually decided. Not logged. Decided.

And in most organizations, no one at the executive table owns it.

Why this sounds like plumbing (and isn't)

The consensus posture on harness engineering is reasonable on its face. Model capability is moving faster than any scaffolding around it. Context windows are expanding, tool use is going native, reasoning is improving. Under that trajectory, the harness looks like a workaround for deficiencies the labs will eventually erase. Stay model-agile. Keep integrations thin. Let the frontier absorb the complexity.

I understand the argument. I made a version of it myself when the first wave of RAG frameworks appeared and most of them didn't survive the year.

But the argument mistakes what the harness actually is. It is not a set of patches. It is the interface layer between your organization and a probabilistic system whose behavior, cost, and risk vary by call. That interface layer does not go away when the model gets better. It gets more important, because the surface area of what the model can do, and of what it can do wrong, expands with capability.

This is the same category error the industry made with billing. Billing looks like plumbing because invoices go out and payments come in. In fact, billing is where pricing strategy, revenue leakage, and financial truth are decided. File it under finance and treat it as ledger infrastructure, and you cede one of the most consequential architectural decisions in the company to whoever draws the shortest straw.

The harness is the same shape of mistake, in a newer domain.

Where the money actually moves

Executive conversations about AI economics still center on price-per-token and seat pricing. Those are the visible numbers. They are not the only numbers that matter.

The numbers that matter also live in the harness. Model routing decides which call goes to a frontier model versus a cheaper one versus a self-hosted open-weight, and it can produce large cost swings on the same workload depending on how aggressively you tier. Context trimming determines whether you are paying to re-send a long document on every turn or a compact summary. Caching policy decides whether the same question from a hundred users costs you a hundred inferences or a handful. Tool-call budgeting decides whether an agent completes a task in a small number of calls or many multiples of that.

None of this shows up on the vendor invoice as a line item. It shows up as a total that runs meaningfully higher than finance modeled, with no obvious explanation, because the harness silently made a thousand small decisions no one architected.

This is the same inversion that applies to billing. Billing is where revenue is decided, not just recorded. Harnesses are where AI unit economics are decided, not just reported. If you cannot see the decisions at the call level, you cannot manage them. And most companies cannot see them, because observability of model calls was treated as a logging concern rather than a financial control surface.

The connection to the ARR attribution problem is direct. Companies claiming AI-driven revenue growth cannot honestly attribute that revenue without call-level telemetry linking features, customers, cost-to-serve, and outcomes. The harness is where that telemetry either exists or doesn't. No harness discipline, no honest AI unit economics.

Sovereignty is a harness decision

The question of whether your company can switch models to a cheaper vendor, an open-weight alternative, or an on-prem deployment for compliance reasons is largely a harness question.

If your prompts are tuned to a specific vendor's quirks, your tool definitions match one vendor's function-calling schema, your evaluators are calibrated against one vendor's outputs, and your context assembly assumes one vendor's window size, you do not have a model. You have a marriage. Switching costs are measured in quarters, not sprints.

The self-hosting conversation is accelerating for reasons that have very little to do with cost. Export controls are reshaping what models certain customers can be served from. Information security teams are asking harder questions about what leaves the perimeter. Regulated industries are increasingly uncomfortable with vendor training and retention dynamics they cannot fully audit. And there is a growing catalog of incidents where a harness silently shipped sensitive context, including source code, to a vendor endpoint no one had reviewed for that purpose. Those are not model failures. They are governance failures at the interface layer.

Most enterprises have rigorous code review for what ships to production. They have no equivalent for what their harness ships to a vendor endpoint on every call. That is an architectural gap, and it will not be closed by choosing a better model.

Executives who do not own the harness architecture cede sovereignty by default. Not deliberately. Just by not asking the question.

The missing link between pilot and ROI

I have written repeatedly about the gap between AI pilots and AI outcomes. The widely cited MIT figure, 95% of enterprise pilots delivering no measurable ROI, is one data point among many, and the exact number is contestable. The direction is not. Multiple analyses, including BCG's, place the majority of failure on people and process rather than on the model itself. My own argument has been that AI strategy stalls without an organizational home, and that the interface between users and the model matters as much as the model.

The harness is the technical artifact that ties much of that together.

Evaluation lives in the harness. So do guardrails, human-in-the-loop patterns, retry logic, fallback routing, and the observability that tells you whether the thing is actually working. Most of the "people and process" failures eventually have to be enforced somewhere in software. That somewhere is the harness.

Which means the emphasis on people and process, correctly read, does not argue against harness investment. It argues for it. The harness is the layer where discipline gets translated into runtime behavior. Without a named owner and an architectural mandate, the discipline stays on slide decks and the runtime does whatever the SDK defaults happen to do that week.

This is a large part of what closes the demo-to-deployment gap. Not a better model. A harness that enforces evaluation, monitors drift, budgets tool calls, routes intelligently, and produces the telemetry that lets a business decide whether to scale a workflow or kill it.

The objections worth taking seriously

Four counterarguments deserve engagement rather than dismissal.

The model will eat the harness. If frontier models become good enough at long-context attention, self-evaluation, and tool orchestration, much of today's scaffolding becomes vestigial. This is plausible for specific patterns; I would not bet the company on today's RAG stack surviving intact for several years. But the interface layer itself does not disappear when the model improves. It changes shape. You still need routing (because you will have multiple models at different price points), observability (because auditors will require it), guardrails (because regulators will require them), and evaluation (because your business will require it). The specific components are transient. The architectural layer is not.

The ESB analogy. Heavyweight middleware has a bad track record. Service-oriented architecture, enterprise service buses, iPaaS platforms: all promised leverage and often delivered lock-in. Why is the harness different? Two reasons. First, it is code-native and owned by product teams, not a separate integration organization with its own governance body. Second, it works best when kept thin; a harness that ossifies loses to one that iterates. The ESB failed when it was sold as a destination. The harness only works as a discipline.

Hyperscaler absorption. Bedrock, Azure AI Foundry, and Vertex are productizing routing, guardrails, and observability. If the harness becomes a checkbox in a cloud console, is the architectural framing weakened? Partially. But hyperscaler harness features tend toward lowest-common-denominator by design, and they lock you to a control plane in exactly the way the sovereignty argument warns against. Using them for baseline capability is fine. Assuming they substitute for architectural ownership is the same bet as assuming cloud billing tools substitute for FinOps.

Talent scarcity. There is no established discipline of harness engineering, no curriculum, no hiring pipeline. This is real. It is also the argument that was made about site reliability engineering, data engineering, and ML platform engineering at earlier points in each of their lifecycles. Disciplines emerge because the leverage is real and someone has to own it. Executives who staff for this now, from adjacent talent pools like platform engineering, ML infrastructure, and developer experience, will have capability their competitors are still writing job descriptions for.

What this means for the executive agenda

The harness is not a DevOps concern to be delegated three levels down. It is an architectural decision with P&L, sovereignty, and compliance consequences. Handling it as plumbing produces the same outcome that handling billing as plumbing produces: silent margin erosion, opaque unit economics, and strategic optionality that quietly disappears.

Concretely, this means a few things.

Name an owner. Not a task force. Not a working group. A named leader with a mandate that covers routing, evaluation, observability, guardrails, and the telemetry that connects model calls to business outcomes. If your AI strategy has an organizational home, the harness sits inside it. If it doesn't, start there first.

Demand call-level observability before you demand more pilots. You cannot manage what you cannot see, and you cannot attribute AI spend to business value without per-call data on cost, latency, quality, and outcome. Every pilot launched without this telemetry produces stories, not evidence.

Treat model portability as an architectural property, not an aspiration. Ask your teams the switching-cost question directly: if we needed to move a meaningful share of our workload to an open-weight model on-prem within a quarter for compliance reasons, could we? The answer is a harness answer.

Review what your harness ships. Someone in your organization should be able to describe, in one page, what leaves your perimeter on a typical AI call, to which endpoints, under what contractual terms, with what retention. If no one can, that is the first artifact to produce.

Budget for the discipline before the market prices it. Harness engineering is on track to become an established function. The talent is scarce now and will likely be scarcer as demand catches up. Hiring adjacent skill sets and giving them the mandate is cheaper today than hiring the specialists later.

The frontier labs will keep releasing models that make last year's scaffolding look ornate. That is not an argument against owning the interface layer. It is the reason to own it. Model capability is the exciting part. The harness is where that capability becomes value, or doesn't.

File it under DevOps and you have made one of the more expensive architectural decisions in your AI program by not making it.