Back to Devoured - September 25, 2026
DEVOURED
Intelligence Density

Intelligence Density

AI Trajectory.ai
Trajectory.ai is introducing 'intelligence density' as a metric for AI efficiency, prioritizing cost-per-task over traditional cost-per-token metrics.
What: Trajectory.ai argues that optimizing for token price encourages 'reward hacking' where models output excessively long, wordy answers. Their 'density-aware' training methods for models like Nemotron 3.5 Nano reduced output tokens by 59% while maintaining performance on legal benchmarks like LAB.
Why it matters: As AI moves into enterprise workflows, the industry must pivot from training models to be cheap at producing tokens to being effective at completing a job without unnecessary verbosity.
Deep dive
  • Standard RL training rewards output length, leading to inefficient model behavior.
  • 'Intelligence density' rewards models for completing tasks with the minimal effective compute.
  • The researchers evaluated density-aware training across legal, financial, and insurance benchmarks.
  • Density-aware training prevents reward hacking and creates more stable, usable model behaviors.
  • Peak accuracy often increases with additional compute, but density-aware models are better at determining when that extra compute is actually necessary.
Decoder
  • Intelligence density: A metric defined as the effectiveness of a model relative to the cost of the entire task, rather than just the price of individual tokens.
  • Reward hacking: A failure mode where an agent exploits a loophole in the reward function (e.g., generating long responses) to maximize its score without actually performing better on the underlying task.
  • Post-training: The stage of refining a foundational model through fine-tuning or RLHF to align it with specific professional tasks or behaviors.
Original article

A cheaper token does not necessarily make a cheaper task. A model can use more tokens, make more tool calls, or take longer to reach the same result. Tibo, who leads Codex at OpenAI, described this with a pizza analogy: each slice can get cheaper while the whole pizza gets more expensive. And we say this often: open source models often cost more than their closed source counterparts over time.

That's because we're measuring the wrong thing: cost per token. What do we actually care about? Something closer to cost per task. Cost per unit of intelligence. We're calling this Intelligence Density.

Intelligence density does not mean forcing every answer to be short. Some tasks require long analysis. The question is whether additional computation improves the result.

Introducing density-aware training

Alongside this, we're sharing our explorations in density-aware training, a way to train models with higher intelligence density. How do we reward a model for doing the work with the right amount of effort? We explore that question on realistic professional tasks.

Models and benchmarks

The experiments use the following open models:

Model Used in Total params Active params
Nemotron 3.5 Nano 30B-A3B Harvey LAB, Rogo's BigFinanceBench 30B 3B
Nemotron 3.5 Lightning 30B-A3B Sierra's Tau3 30B 3B
Nemotron 3 Ultra 550B-A55B Rogo's BigFinanceBench baseline 550B 55B

Benchmarks: Harvey's Legal Agent Benchmark (open-source, MIT), Rogo's BigFinanceBench (public), and Sierra's Tau3, an insurance-workflow suite built on the τ²-bench framework (private).

Why it matters: capability that stays usable

We trained and evaluated Nemotron 3.5 Nano 30B-A3B on Harvey's open-source Legal Agent Benchmark (LAB): long-horizon legal work across 24 practice areas such as drafting memos and analyses from source documents.

The first post-training run makes the model capable, but at a much higher output cost. Density-aware training keeps the same 8.3% pass rate while bringing mean output from 90,000 tokens down to 37,000. That is the product consequence of intelligence density: a model can do the professional task without turning every task into its most expensive version.

Why it matters: training that learns the right behavior

Usability is only half of the story. Density-aware training also changes what a model learns from reward. We trained NVIDIA Nemotron 3.5 Lightning 30B-A3B on Sierra's Tau3 and evaluated 50 held-out tasks at each checkpoint. Held-out score is mean partial-credit reward. Strict accuracy counts only fully completed tasks.

  • Density-aware RL: 55.6% held-out score, 14% strict accuracy, and 2,267 training output tokens per task.
  • Standard RL training: 5.7% held-out score, 2% strict accuracy, and 17,877 training output tokens per task.

Standard RL training recorded the higher training score at the same step, 56.7% versus 46.3%, despite much worse held-out results. Its training reward rises with output length while held-out performance collapses. That is reward hacking. Density-aware training gives the model a reason to learn the task itself, yielding a more stable run and a better held-out result with far less training-time output.

Why it matters: extra compute becomes more useful

Efficiency is not the only goal. Intelligence density should also teach a model when additional test-time compute is worth spending. We measured how NVIDIA Nemotron 3.5 Nano 30B-A3B accuracy changes as maximum output length increases on Rogo's BigFinanceBench. The question is not whether a model can produce a longer answer. It is whether the extra work turns into a better final answer.

On the public-facing evaluation set, density-aware training raises the 30B model's peak final-answer accuracy from 24% at step 0 to 36% at step 30. The trained NVIDIA Nemotron 3 Ultra 550B-A55B reference scores 32% in a separate evaluation. This is the other side of intelligence density: it reduces unnecessary work on routine tasks while preserving the ability to spend more compute where it improves the result.

Where this goes

Human experts and veterans in the field accumulate experience through decades of specialized work. With experience, work that once required intense concentration becomes second nature, allowing experts to act precisely with less effort. Experts also know when to probe for deeper analysis when approaching novel environments.

We think that models should learn this judgement as well. Intelligence density is a first step towards tackling this by formalizing how to complete specialized tasks efficiently; to condense intelligence density is to have models that can do much more per dollar.

The best interface was no interface

We are a research and product company, so we went looking for interfaces to intelligence density. We tried dials, routes, and budgets. The sketches below explore those ideas.

What we found was that the best interface was no interface. Every model on Trajectory is trained with intelligence density on by default. The model learns when more computation will improve the result and when it is time to finish. You just ask for the work.

At Trajectory, we are building models that close the experience gap. We want systems that keep learning from work and use that experience to deliver better results at lower cost per task. If you are interested in this research, join us.

Explore all 50 articles in today's digest

Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi