Mozilla: Anthropic's Mythos found 271 security vulnerabilities in Firefox 150 (3 minute read)
Anthropic's Mythos AI model found 271 security vulnerabilities in Firefox 150 before release, a dramatic increase from the 22 bugs found by the previous AI model, prompting Mozilla's CTO to claim defenders are finally winning the cybersecurity battle.
Deep dive
- Mythos Preview found 271 vulnerabilities in Firefox 150 by analyzing unreleased source code, a 12x increase over the 22 bugs found by Anthropic's previous Opus 4.6 model on Firefox 148
- Mozilla CTO Bobby Holley says Mythos performs at the same level as elite human security researchers, based on Mozilla's years of experience evaluating top security talent
- The vulnerabilities could have been found through traditional automated fuzzing or expert manual analysis, but Mythos eliminated months of costly human effort per bug
- Holley argues this tilts the cybersecurity balance toward defenders because when vulnerability discovery becomes cheaper, defenders benefit more than attackers since they can proactively fix issues
- Mozilla believes they've "rounded the curve" on Firefox security with this AI-assisted head start, though future models might find bugs current ones miss
- The shift is particularly crucial for open source projects with public codebases and insufficient volunteer security maintenance, which underpin much of the modern internet
- Mozilla CTO Raffi Krikorian argues that the historical balance between the difficulty of writing complex code and finding bugs is breaking down with AI capabilities
- Anthropic initially limited Mythos Preview release to "critical industry partners," sparking debate about whether this represents a revolutionary capability or just incremental AI progress
- Holley predicts every software project will need to engage with AI-aided vulnerability analysis going forward due to the newly discoverable nature of hidden bugs
Decoder
- Mythos Preview: Anthropic's latest AI model specialized in finding security vulnerabilities by analyzing source code, initially released only to select partners
- Fuzzing: Automated testing technique that feeds random or malformed data to programs to discover crashes and security bugs
- Opus 4.6: Anthropic's previous generation AI model, which found significantly fewer vulnerabilities than Mythos
- Open source vulnerability: Security flaws in publicly available code that anyone can inspect, making them both easier to find and more critical to fix since they affect many downstream projects
Original article
Earlier this month, Anthropic said its Mythos Preview model was so good at finding cybersecurity vulnerabilities that the company was limiting its initial release to "a limited group of critical industry partners." Since then, debate has raged over whether the model presages an era of turbocharged AI-aided hacking or if Anthropic is just building hype for what is a relatively normal step up on the ladder of advancing AI capabilities.
Mozilla added some important data to that debate Tuesday, writing in a blog post that early access to Mythos Preview had helped it pre-identify 271 security vulnerabilities in this week's release of Firefox 150. The results were significant enough to get Firefox CTO Bobby Holley to enthuse that, in the never-ending battle between cyberattackers and cyberdefenders, "defenders finally have a chance to win, decisively."
"We've rounded the curve"
Holley didn't go into detail on the severity of the hundreds of vulnerabilities that Mythos reportedly detected simply by analyzing the unreleased source code of Firefox's latest version. But by way of comparison, he noted that Anthropic's Opus 4.6 model found only 22 security-sensitive bugs when analyzing Firefox 148 last month.
The vulnerabilities identified by Mythos could have also been discovered either by automated "fuzzing" techniques or by having an "elite security researcher" reason their way through the browser's complex source code, Holley writes. But using Mythos eliminated the need to "concentrate many months of costly human effort to find a single bug" in many cases, Holley added.
By identifying bugs so efficiently, Holley writes that AI tools like Mythos tilt the cybersecurity balance toward defenders, who benefit when discovering vulnerabilities becomes cheaper for both sides. "Computers were completely incapable of doing this a few months ago, and now they excel at it," Holley writes. "We have many years of experience picking apart the work of the world's best security researchers, and Mythos Preview is every bit as capable."
In an interview with Wired, Holley said that, from now on, this kind of AI-aided vulnerability analysis is something that "every piece of software is going to have to [engage with], because every piece of software has a lot of bugs buried underneath the surface that are now discoverable." And while it's possible that future models more advanced than Mythos may be able to find bugs that current models miss, Holley said he was confident that "at least on the Firefox side, having had a bit of a head start here, that we've rounded the curve."
Running through the AI-aided defense gauntlet could be especially important for the open source projects that underpin much of the modern Internet. That's both because their public codebases are easier for AI systems to explore for vulnerabilities and because many such projects rely on wildly insufficient volunteer maintenance for their security.
In a New York Times essay last week, Mozilla CTO Raffi Krikorian argued that the human difficulty of both finding bugs and writing complex software has created a kind of balance in cyberthreat research that Mythos could break wide open. "The programmer who gave 20 years of his life to maintain [open source] code that runs inside products used by billions of people? He doesn't have access to Mythos yet. He should," Krikorian wrote.