Tech companies move to open AI models
Major tech firms are aggressively reducing AI costs by shifting from frontier models to cheaper, open-weight models and implementing smart model routing.
Deep dive
- Cost Reduction: Uber achieved a 52% reduction in cost per session through prompt compaction and caching.
- Routing: Implementation of model routers allows traffic to be directed to the most efficient model based on task complexity.
- Model Quality: AT&T noted that swapping from closed to open models resulted in only a 2% performance drop for significantly lower costs.
- Post-training: Pinterest post-trains open models on unique proprietary data to achieve performance that outperforms generic frontier models.
- Efficiency Tools: Companies are increasingly using automatic token compaction for prompts exceeding 400K tokens.
Decoder
- Inference Provider: A service that hosts and provides access to AI models, allowing developers to run them via API without managing the underlying hardware.
- Model Routing: The practice of directing an AI request to a specific model based on factors like cost, speed, and expected performance required for the task.
Original article
In May, I covered an emerging trend of companies wanting to cut back their AI spending, starting with engineering departments. Different approaches were being tried:
- Experimentation with running cheaper, open models on inference providers
- More investment in model routing to route simpler requests to cheaper models
- Knowledge-sharing sessions on how to use AI models cost-effectively
- Setting per-developer monthly AI usage limits
A few months later, it seems that several companies have managed to achieve this, according to sources I’ve spoken with.
Uber: AI costs down 50%
Uber managed to blow through its annual AI budget in the first three months of this year, and it wasn’t a surprise to hear, in May, Uber’s COO say that it was getting harder to justify spending on tools like Claude Code without seeing benefits from the leading models. It wasn’t long until engineering teams at the ridesharing giant set to work on how to optimize AI spend, and their efforts weren’t in vain.
Uber cut the cost per AI request by 34%, and the cost per AI session by 52%:
Of course, Uber keeps using more AI tokens and starting more AI sessions, but thanks to optimizations the cost has been flat since March, despite significantly more usage:
How did they pull it off at Uber? It was via a combination of different approaches:
- Open weight models, run using inference: run open weight models on inference services, which are significantly cheaper than frontier ones.
- Optimized model selection: benchmark all available frontier and open models, to build an accurate picture of their present capabilities
- Ongoing benchmarking: run benchmarks every week based on real work, and update them
- Cheaper subagent models: subagents do smaller tasks not requiring the most expensive models
- Reduce model effort: Uber found that defaulting to Medium effort gives the best cost-to-output ratio with advanced models
- Optimize requests: trigger automatic compaction above 400K tokens, even for models with 1M context windows
- Cache prompts: cache prompts to save money when using Uber’s own harness, Minions
- … and more: Uber wrote an engineering blog post detailing the dozens of optimizations taken to reduce token cost without noticeable change in the quality of code generated by agents
From the outside, the single biggest win seems to be Uber’s transition to using open models for certain tasks. Open models cost 2-20x less, compared to frontier ones:
Pinterest makes 90%+ cost savings by dropping frontier models
Interesting details from Pinterest’s earnings call last month reveal how much the social media platform saves by running open models. Here’s what CEO William Ready said:
“Our approach to model deployment includes our own compact fit-for-purpose models built for Pinterest-specific use cases and suitable open source models post-trained in our own environment within our secure cloud infrastructure. When we leverage open source models, such as with Pinterest Assistant, we are seeing superior performance for our use cases when compared to closed third-party models because we are able to post-train open models on our highly unique data.
With open models, we are achieving cost per transaction at less than 8% of the cost of comparable closed proprietary models. This gives us substantial headroom to deepen and extend these capabilities over time in a way that is differentiated, highly effective, and cost efficient.”
Basically, what used to cost Pinterest $100 to run on a closed, frontier model, they now spend $8 on by using open models on owned or rented inference!
AT&T: 56% savings by swapping Claude for open models
With 100,000 employees, AT&T is a big spender on AI. The telco giant cut its AI bill by 56% while measuring a 2% decrease in the quality of AI’s output, after they moved workloads over to open models.
“Austin said he’s found that open source models are “just as good or better” than older models sold by the likes of Anthropic and OpenAI. For instance, AT&T’s software developers still rely on cutting-edge models for complex tasks like generating code, but can use cheaper open source models for less intense tasks like generating summaries of previously submitted code, he said.
After the company began using router provider LiteLLM, the costs of some advanced AI tasks such as coding fell by as much as 56% while the quality of the AI’s performance fell just 2%, Austin said.”
Anthropic overpriced compared to the rest of the market?
Only a few months ago, Anthropic was the preferred model (Claude) and harness (Claude Code) among engineers. But Anthropic’s models are becoming steeply more expensive at a time when open weight models – and also OpenAI – are getting much cheaper. Meanwhile, Opus 5 is 100x more expensive (!!) than models like GPT-5.6 Luna xhigh and DeepSeek. That may be simply too much to ignore for some tech companies:
Seeing this data, I’m not surprised that more tech companies are looking to run open weight providers on inference providers, due to the significant savings available from a model that’s similarly capable as one from Anthropic.
What worked for Stripe, Coinbase, Uber & Ramp
The engineering team at Databricks interviewed engineers at Stripe, Coinbase, Uber, and Ramp, and collected how different approaches helped save costs for them.
To answer the question posed in the header of this report, it’s apparent that using open models is indeed the approach offering the biggest savings, followed by smart model routing. Spending controls and context optimization also bear down on costs, but they don’t come close to the first two techniques in results.
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi