DEVOURED
Two Kinds of Test-Time Scaling
True 'compute extrapolation' allows models to achieve performance gains at test-time that exceed the limits of their training-time compute budgets.
What: James Tillman distinguishes between standard test-time scaling and compute extrapolation. Extrapolation, likely enabled by context management techniques like 'chunking' or 'compaction,' allows models to scale performance well beyond the max compute-per-task observed during training.
Why it matters: This shift moves the primary locus of AI capability gains from training-time infrastructure to inference-time spending, making dangerous capability evaluations significantly more difficult to conduct.
Deep dive
- Distinguishes between test-time scaling (up to training maximums) and compute extrapolation (beyond training maximums).
- Compaction methods like chunked chain-of-thought enable models to leverage massive inference budgets.
- Compute extrapolation accelerates the frontier of AI capabilities, making upper-bound evaluations prohibitively expensive.
- AI labs are likely tracking extrapolation ceilings as a competitive metric for future R&D.
- Multi-agent systems may offer a more scalable form of extrapolation than single-agent chains-of-thought.
Decoder
- Compute extrapolation: The ability of a model to continue improving performance when provided with more inference-time compute than it ever experienced on individual tasks during training.
- LLM compaction: Techniques, such as summarizing previous reasoning steps, that allow models to process effectively infinite context lengths without exceeding memory limits.
Original article
Full article content is not available for inline reading.
Explore all 45 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi