DEVOURED
My AI Opinions
Scott Alexander details his personal AI outlook, predicting a 50% chance of AGI by 2034 and arguing that recursive self-improvement remains the key unknown variable.
What: Scott Alexander outlines his timelines for AGI, superhuman AI, and the 'point of no return.' He assigns a 20% probability that the first AIs capable of eliminating humanity will attempt to do so, emphasizing that alignment efforts and interpretability work could meaningfully reduce this risk.
Why it matters: This highlights the 'Outside View' method of forecasting, where current AI experts often diverge from broader public consensus based on specific assumptions about compute, algorithmic progress, and corporate safety incentives.
Deep dive
- 25% chance of AGI by 2027; 50% by 2034; 75% by 2045.
- Defines AGI as the capability to perform 90% of knowledge work jobs.
- Identifies 'recursive self-improvement' as the most volatile and unmodelable variable in current forecasts.
- Argues the 'diffusion gap'—the time between AGI capability and actual societal integration—may be slower than expected due to regulatory and infrastructure hurdles.
- Suggests a 20% p(doom) is reasonable, contingent on the efficacy of future alignment techniques and oversight.
- Advocates for a balanced approach between active safety research and nuanced policy advocacy to avoid 'us-vs-them' political polarization.
Decoder
- AGI (Artificial General Intelligence): AI systems that possess the ability to understand, learn, and apply knowledge across a wide range of tasks at a level comparable to or exceeding human capability.
- RLAIF (Reinforcement Learning from AI Feedback): A technique where AI models are trained using feedback generated by other AI models, rather than exclusively by human raters.
- Bostromian superintelligence: A hypothetical future AI that can accelerate technological progress at a scale where one year of work yields a century's worth of traditional human innovation.
- Interpretability: The study of how neural networks process information internally to explain their outputs and decision-making patterns.
Original article
Full article content is not available for inline reading.
Explore all 50 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi