DEVOURED
The AI Forecasting Agenda
AI models now rival human superforecasters, shifting the research agenda toward complex tasks like conditional, causal, and long-term prediction.
What: AI forecasting tools (e.g., FutureSearch, Mantic) have reached human-parity on traditional prediction benchmarks. Researchers are now testing how these models handle 'unobservable' causal questions, long-term time horizons, and complex policy cruxes, though AI still struggles with reliably forecasting its own rate of progress.
Why it matters: Shifting forecasting to AI allows for quantifying policy debates and evaluating strategic cruxes, provided these models can overcome their tendency to regurgitate the prevailing liberal consensus or exhibit biased views on AI progress.
Deep dive
- AI forecasters now reach performance parity with top human superforecasters on standardized tasks.
- Current challenges include moving from binary 'yes/no' forecasts to complex conditional and causal scenarios.
- Forecasting AI progress remains an outlier where performance is poor, possibly due to 'scaling law' biases.
- Epistemic questions (e.g., 'Was COVID a lab leak?') are being converted into resolvable 2040-dated questions to allow for standardized tracking.
- AI models are being used to identify 'cruxes'—early indicators that most heavily influence the outcome of far-future events.
- Testing for bias requires identifying where models deviate from the 'respectable liberal consensus'.
Decoder
- Superforecaster: A person who consistently demonstrates superior ability in predicting the outcomes of real-world events using rigorous, data-driven methodologies.
- Conditional Forecast: A prediction about an outcome specifically contingent upon a prior event (e.g., 'If X happens, what is the likelihood of Y?').
- Causal Forecast: An estimate determining whether event X actually drives event Y, rather than merely predicting correlation.
Original article
Full article content is not available for inline reading.
Explore all 56 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi