DEVOURED
HarnessTax: How Much Does the Harness Matter for Coding Agents?
A study on 'agentic harnesses' reveals that developer tools can differ in cost by up to 5x for the same task.
What: Researchers evaluated 21 combinations of agents (Claude Code, Codex CLI, etc.) on SWE-bench Lite and Terminal-Bench 2.0, finding that provider-native harnesses are not always the most efficient.
Why it matters: This highlights the 'HarnessTax'—the hidden costs of different agent orchestration logic—and underscores that efficient prompt engineering and tool-use strategy can significantly impact operational expenses.
Decoder
- Harness: In the context of AI agents, the software wrapper that manages the agent’s loop, including file system access, terminal execution, and planning logic.
Original article
Coding-agent harnesses such as Claude Code, Codex CLI, and Pi can produce similar task success rates while using dramatically different amounts of tokens and money. Across 21 model–harness combinations on SWE-bench Lite and Terminal-Bench 2.0, harness choice changed cost by as much as 5×, and the model provider's own harness was not consistently the most efficient option.
Explore all 58 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi