DEVOURED
Generalization Dynamics of LM Pre-training
UC Berkeley and Stanford researchers found that large language models frequently "mode-hop" between pattern-matching and intelligent generalization during pre-training, defying stable maturation.
What: Researchers Jiaxin Wen, Zhengxuan Wu, Dawn Song, and Lijie Chen observed that LMs like OLMo3 (7B, 32B) and Apertus (8B, 70B) don't smoothly progress to intelligence but rather exhibit sudden shifts between parroting learned patterns and applying adaptive intelligence, even after trillions of tokens.
Why it matters: This discovery challenges the common assumption that LMs stably improve their generalization abilities during pre-training, suggesting that current training methods might struggle with fundamental capacity allocation issues where shallow patterns compete with deeper, generalizable circuits.
Takeaway: If you are pre-training large language models, use the provided `GDsuite` (github.com/Jiaxin-Wen/GDsuite) to monitor for "mode-hopping" behavior. Consider experimenting with checkpoint selection based on generalization metrics rather than just final loss, as shown to yield better reasoning and alignment in post-training. Be aware that current generalization predictors and "simplicity bias" in models may be insufficient for understanding complex LM behavior.
Deep dive
- The team developed a "toy eval suite" to identify behavioral fingerprints distinguishing "parrots" (pattern-matching) from "intelligence" (inferring functions).
- Mode-hopping manifests across various tasks, including flipped answer classification, repetitive/successive pattern recognition, truth vs. truthiness, System 1 vs. System 2 thinking, and multi-hop persona QA.
- The phenomenon is locally stable, not corrected by single gradient steps, and only mitigated, not fixed, by checkpoint averaging.
- It persists even with large models trained beyond Chinchilla-optimal budgets (up to 9x to 90x).
- Mode-hopping is seen as a "capacity allocation problem" where generalizable circuits compete with shallow ones.
- The research demonstrates applications in selecting better intermediate pre-training checkpoints for improved reasoning and robust alignment, and in curating pre-training data to stabilize generalization dynamics.
- Existing generalization predictors based on solution complexity were found to be nuanced, showing both positive and negative correlations depending on the layer, suggesting that generalizable solutions can be simple or complex.
Decoder
- Mode-hopping: A phenomenon observed during large language model pre-training where models unpredictably switch between behaviors, sometimes relying on shallow pattern-matching and other times exhibiting deeper, intelligent generalization.
- Chinchilla-optimal budget: Refers to the optimal scaling law for language models, suggesting a balance between model size and training data quantity for best performance, where larger models need more data. Training beyond this means consuming significantly more data relative to model size.
- System 1 / System 2 thinking: Concepts from psychology, where System 1 is fast, intuitive, and automatic thinking, while System 2 is slower, deliberate, and logical reasoning.
- In-context learning: The ability of a language model to learn new tasks or adapt to new patterns from examples provided directly in the prompt, without explicit fine-tuning.
- GPQA: General Purpose Question Answering, a benchmark for evaluating complex reasoning in language models.
Original article
Full article content is not available for inline reading.
Explore all 43 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi