DEVOURED
What sort of maths are LLMs good at?
Fields Medalist Tim Gowers suggests that while LLMs are elite at finding mathematical counterexamples, they still lack the 'human' intuition to prune deep search trees effectively.
What: Gowers notes that LLMs shine at searching for existence proofs and standard counterexamples due to their vast knowledge and speed, but they struggle with conceptual, original research that requires non-obvious branching.
Why it matters: This highlights the 'LLM style' of research—brute-force exploration—and contrasts it with human mathematical maturity, suggesting that the path to true autonomous discovery requires models to develop a 'nose' for promising directions.
Deep dive
- LLMs excel at problems where the search tree is shallow enough for high-speed brute force.
- Current successes often involve finding counterexamples that experts had previously suspected but failed to formalize.
- Human mathematicians provide value by 'pruning' the search tree—a capacity LLMs lack.
- LLMs currently struggle with 'null' or 'empty set' answers in reasoning tests.
- The current evaluation methods (e.g., k@pass) often favor standard solutions over novel, conceptual breakthroughs.
- Future LLM utility will be measured by whether they can discover proofs that seem 'surprising' and 'beautiful' to human experts.
Decoder
- Sofic group: A type of group in mathematics that can be approximated by finite symmetric groups.
- Skolemization: A technique in formal logic to eliminate existential quantifiers from a formula by replacing them with functions.
- Ramsey number: The minimum number of vertices in a graph needed to guarantee a specific property.
Original article
Full article content is not available for inline reading.
Explore all 54 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi