Back to Devoured - June 04, 2026
DEVOURED
Intelligence Per Dollar

Intelligence Per Dollar

AI enterpriseperformancecloud Tom Tunguz
Microsoft is introducing "average token usage" to model release cards, benchmarking AI models on both performance and the cost of achieving that intelligence, pushing for efficiency over raw power.
What: Microsoft added "average token usage" to a model release card on June 3, 2026, for its MAI-Code-1-Flash model. This new metric forces model companies to compete on "intelligence per dollar," highlighting how models like GPT 5.5 can be 40% cheaper than Claude Opus 4.8 for similar results on an "Intelligence Index."
Why it matters: This marks a significant shift in the AI industry, moving beyond raw performance benchmarks to focus on economic efficiency and cost-effectiveness. It signals the end of the "subsidy era" for AI and prioritizes tangible outcomes and budget constraints for enterprise adoption.
Takeaway: If evaluating AI models for enterprise use, prioritize benchmarks that include cost-per-token or intelligence-per-dollar metrics alongside traditional performance scores.
Deep dive
  • Microsoft has introduced "average token usage" as a new standard metric on its model release cards, beginning with MAI-Code-1-Flash.
  • This metric measures the efficiency of AI models by benchmarking performance against the cost of achieving that intelligence.
  • The move suggests a shift away from "tokenmaxxing" and an era where AI subsidies are ending, as even large companies like Uber and Salesforce face budget constraints on AI spending.
  • For example, on the SWE-Bench Verified benchmark, a Microsoft model uses about a third of the tokens compared to Claude Haiku 4.5 for similar performance.
  • Artificial Analysis's "Intelligence Index" shows GPT 5.5 and Claude Opus 4.8 performing similarly (around 60 index points) but GPT 5.5 costs $3,357 to run the index, while Opus 4.8 costs $4,685, indicating a 40% cost difference for the same answer.
  • The new benchmark encourages model providers to compete on both performance and cost-efficiency.
  • Application layer companies are now expected to compete on "dollars per outcome," such as the cost of a closed support ticket or a shipped PR, rather than per token.
Decoder
  • Token usage: In large language models, a "token" is a basic unit of text or code (e.g., a word, part of a word, or punctuation mark). Token usage refers to the number of these units processed by the model, which directly correlates with computational cost.
  • Tokenmaxxing: A term describing models that might achieve high benchmark scores by using an excessive number of tokens, which can lead to higher costs in real-world applications.
  • Intelligence Index: An independent benchmark by Artificial Analysis that measures the overall performance of AI models, often used in conjunction with cost to determine "intelligence per dollar."
Original article

Yesterday Microsoft added a new metric to a model release card, one that will likely become a standard.

Average token usage.

In the first row, the Microsoft model hits 71.6 on SWE-Bench Verified using about a third of the tokens Claude Haiku 4.5 burns.

Benchmarks are now measured on two different dimensions, the overall performance & the cost to achieve that intelligence.

This is yet another sign that the era of subsidies, tokenmaxxing, & all-out performance for many use cases is over.

Even the most valuable companies in the world cannot afford state-of-the-art intelligence for every conceivable use case. Uber capped employee AI spending after blowing through its budget in four months. Salesforce is spending $300M on Anthropic tokens & has frozen engineering hires.

This new dual benchmark answers the buyer’s only question : what is my intelligence per dollar?

Artificial Analysis already benchmarks this. GPT 5.5 & Claude Opus 4.8 land within a point of each other on the Intelligence Index, around 60. Running the index costs $3,357 on GPT 5.5 & $4,685 on Opus 4.8. Same answer, 40% more expensive.

Model companies must now compete on both dimensions. The application layer will compete one level up, on dollars per outcome, what a closed ticket, a shipped PR, or a resolved support case actually costs.

Every layer in the stack now has to price the same way the customer thinks : per result, not per token.

  1. Introducing MAI-Code-1-Flash — Microsoft announces a new coding model with average token usage on the release card. 

  2. The Unsustainable Subsidy — The era of AI subsidies is ending. 

  3. Tokenmaxxing — Models that game benchmarks with extra tokens are losing their edge. 

  4. Microsoft cancels Claude Code licenses, shifting developers to GitHub Copilot CLI — Microsoft cancelled Claude Code licenses across its Experiences and Devices division (Windows, Microsoft 365, Outlook, Teams, Surface) after engineering usage outran budgets. 

  5. Uber caps employee AI spending after blowing through budget in 4 months — Uber caps employee AI spending after blowing through budget in four months. 

  6. Salesforce Spends $300M on AI, Freezes Engineering Hires — Salesforce Spends $300M on AI, Freezes Engineering Hires. 

  7. AI Model & API Providers Analysis — Independent analysis of AI model costs. 

Explore all 55 articles in today's digest

Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi