Writer's AI harness cuts token spend nearly 40% — without sacrificing accuracy
Enterprise AI is facing an ROI paradox. While throwing more compute at the strongest foundation model works well in product experiments, the costs become unbearable when the product is deployed in production.
A new paper from researchers at Writer provides a solution that is accessible to engineering teams. The study takes a systematic look at optimizing the different components of the orchestration layer that wraps around the foundation model, aka the AI harness.
By optimizing the harness, the researchers show dramatic reductions in tokens per task, a drop in cost-per-successful-task by up to 61%, and quality that holds steady, all without changing the underlying foundation model.
Because the harness is fully under the developer's control and requires no model fine-tuning, engineering teams can apply these findings to build highly cost-efficient AI applications.
The ROI crisis of tokenmaxxing
The current state of AI engineering is plagued by "tokenmaxxing," an industry trend where developers rely on massive context windows and brute-force token consumption as a substitute for good system design.
Rather than engineering elegant workflows, developers have imported a reflex from traditional software development: generate, run, fail, stuff the error and more context back into the window, and retry.
"Teams tokenmaxx because it's the cheapest fix in the moment, and because it's literally how most engineers work today," Waseem AlShikh, CTO and co-founder of Writer, told VentureBeat. Because this approach succeeds often enough on coding tasks, it has become the default reflex for every other agentic workload. The danger is that per-token price drops mask the underlying inefficiency.
"Your invoice is tokens-per-task times price-per-token, and most teams only watch the second number," AlShikh said. "In agentic workloads, tokens-per-task compounds — every loop iteration re-transmits the growing context — and it compounds faster than prices fall. The price cut becomes an anesthetic. It masks the fact that the loop itself is bleeding."
Tokenmaxxing leads to several enterprise failure modes. Teams route simple tasks to premium frontier models by default. They use the LLM as a lazy search index, stuffing the context window with raw documents instead of retrieving exact answers. Most destructively, they build unconstrained agentic loops that spiral out of control when the model encounters an error. Because output tokens cost significantly more than input tokens across all major model providers, inefficient task execution acts as a silent budget killer.
The industry has introduced several efficiency techniques to curb these costs, but they largely fall short because they treat the model in isolation:
Prompt compression condenses input text to save space, but ignores how the system sequences those inputs across complex workflows.
Budgeted reasoning caps the computational steps a model can take, which often degrades output quality if the workflow isn't intelligently routed.
Terse coding forces models to output minimal code to save output tokens, but does nothing to solve inefficient tool calling.
Speculative decoding uses a smaller draft model to speed up a larger model's text generation, optimizing inference speed while failing to address bloated agent architectures.
These efforts fail because they optimize the engine while ignoring the transmission. They do not look at the orchestration layer, leaving underlying architectural inefficiencies unresolved.
Unpacking the harness: the levers of efficiency
The harness is the orchestration layer that routes, formats, and turns the underlying LLM into a working system.
The core levers of harness optimization include system prompt caching, interaction history compaction, tool management, retrieval strategies, and error management. These are the most accessible intervention points for engineering teams looking to improve AI performance.
As the Writer researchers note in the study: “If the harness is the layer that composes model calls into work, it is also the layer that sets the price of work.”
Historically, developers have treated the harness as disposable glue code designed simply to connect an API to a user interface. The study signals that the harness must now be treated as a first-class object: a primary software artifact that requires its own testing, versioning, and rigorous design.
For enterprises, this reframes the "own-versus-rent" decision.
"Enterprises spend months on model evaluations and then rent their orchestration off the shelf — which means they're optimizing the smaller lever and outsourcing the bigger one," AlShikh said. "Whoever owns the harness owns your unit economics, and an open framework tuned for demos is not tuned for your invoice."
Inside the experiments
To isolate the impact of the orchestration layer, the researchers ran experiments on six foundation models spanning multiple vendors and weight classes: Claude Sonnet 4.6, Gemini 3.1, Gemini Flash 3.5, Qwen 3.6, GLM 5.1, and Writer’s own model, Palmyra X6.
Their experiments compared a frozen, conventional production agent loop against the finished Writer Agent Harness on the same 22 locked enterprise tasks, spanning capabilities like grounding and retrieval, multi-step workflows, tool use, and content generation. By holding the models and tasks constant, they could isolate the effects of the orchestration layer itself.
The optimized harness drove a significant drop in costs, cutting the blended cost per task by 41%, from 21 cents to 12 cents. This was largely achieved by slashing token consumption, with the number of tokens per task falling 38%, from 14.2k to 8.8k.
The harness is designed to delegate tasks like search to specialized sub-agents. A sub-agent receives only the tool and the specific query it needs, retrieves the exact data, and returns a capped, clean summary to the main agent — keeping the primary context window from filling up with raw search results.
Task success rates held steady even as token use fell — moving from 78% to 81%, a gain the researchers describe as directional rather than statistically significant at their sample size, meaning quality didn't suffer even as costs dropped.
End-to-end task latency also dropped significantly, reducing the median wall-clock time by 44%, from 48 seconds to 27 seconds, due to prompt caching and the elimination of dead-end reasoning loops.
However, the researchers also found limits to multi-agent orchestration. Smaller models like Gemini Flash 3.5 and Qwen 3.6 scored well below a usable reliability threshold on sub-agent delegation tasks (0.45 and 0.42, respectively) — the capability simply isn't dependable yet on lighter-weight models.
Sub-agent orchestration only crossed a usable reliability threshold on the two strongest models tested: Writer's own Palmyra X6 (0.86) and Claude Sonnet 4.6 (0.85).
The developer’s playbook: actionable takeaways and tradeoffs
The findings from the study translate into a playbook for enterprise developers building agentic workflows at scale. The first step is to implement what AlShikh calls the "Two-Zone Prompt" and "Context Offloading."
Structure for system prompt caching (The Two-Zone Prompt): Modern LLM APIs offer prompt caching, but developers must structure their payloads correctly to trigger it. Developers must separate the "stable zone" from the "volatile zone." Place static, unchanging elements (e.g., core rules, large tool schemas, and standard operating procedures) at the top of the prompt. Dynamic elements, such as the specific user query or recent conversational task state, must be appended at the bottom. This ordering allows the harness to reuse the cached prefix across hundreds of calls. "That single separation makes prompt caching actually work and stops you from re-paying for the same instructions on every one of an agent's thirty steps," AlShikh said.
Manage context with Context Offloading: Avoid context stuffing, where every turn of a loop is appended into a monolithic prompt until the window maxes out. Instead, move history and intermediate artifacts out of the window into retrievable storage, and pull back only what the current step needs. If possible, delegate tasks to single-purpose sub-agents to avoid context bloat. As AlShikh points out, "the biggest line item in agent spend isn't reasoning — it's re-sending things the model has already seen."
Build resilient loops and redefine KPIs: Unmanaged agent loops drain API budgets rapidly. Teams must begin tracking Completions Per Million tokens (CPM) to understand their true task costs, but the harness itself must contain physical guardrails. "The core principle is that you never ask the model to police its own spending," AlShikh said. "The fence has to live below the model, in code, on your side of the API." This requires three hard checks:
Hard per-task token budgets: The run terminates when the budget is spent, no exceptions.
Generation fencing: Caps on steps, tool calls, and recursion depth to stop non-converging agents.
Failure-spend governance: Cap what a run can spend after its first failed validation so a failing task doesn't become your most expensive task.
Avoid unnecessary complexity: Optimizing the orchestration layer comes with engineering overhead. If you're in the prototyping and exploration stage, that overhead isn't justified — iterate fast with a strong model and a light harness. Once you're scaling to millions of requests a day, the savings from harness optimization become substantial.
However, teams must be aware of "harness leverage." Adding structural scaffolding requires the model to hold and obey that context. If a model is too small, it will spend its limited capacity parsing the scaffolding instead of doing the task, causing accuracy to drop and tokens to rise. The rule for adding complex orchestration features is strictly mathematical: "If a feature adds more coordination tokens than it removes task tokens for that specific model, cut it," AlShikh said. "Nothing in the harness is free."
The future of the enterprise harness
The era of tokenmaxxing and treating context windows like bottomless buckets is coming to an end. Throwing more compute at poorly designed systems is not a viable strategy for companies that need to demonstrate a return on their AI investments.
As foundation models evolve to absorb planning, tool selection, and multi-step reasoning natively into their weights, the role of the harness will shift from compensating for model weakness to enforcing enterprise policy.
"What never moves into the model is the 'allowed': budgets, permissions, data boundaries, audit trails, deterministic kill-switches," AlShikh said. "Five years from now, the harness will be thinner but more important. There will be less scaffolding and more governance. However capable the model gets, someone external to it still has to define what it may spend, see, and touch. That layer belongs to the enterprise, and it should never be rented."
