DEVOURED
Project Glasswing: what Mythos showed us
Cloudflare found Anthropic's Mythos Preview security LLM significantly advanced vulnerability discovery by chaining exploits and generating proofs, though it exhibited inconsistent refusals and high false positive rates in memory-unsafe languages.
What: Cloudflare tested Anthropic's Mythos Preview, a security-focused Large Language Model, on over 50 of its repositories as part of Project Glasswing. The model showed significant progress by constructing exploit chains from multiple low-severity bugs and autonomously generating working proof-of-concept code, a capability general-purpose LLMs lacked. However, Mythos Preview demonstrated inconsistent "organic refusals" to legitimate security research requests and a higher signal-to-noise problem with false positives in memory-unsafe languages like C and C++. Cloudflare developed a "harness" system, with stages like Recon, Hunt, Validate, and Trace, to manage the model's execution and improve findings quality.
Why it matters: This evaluation highlights a pivotal shift in AI's role in cybersecurity, demonstrating that specialized LLMs can now actively construct complex exploits rather than just identify individual bugs. It underscores the dual-use challenge of AI—empowering defenders while also accelerating attacker capabilities—and emphasizes the necessity for sophisticated architectural "harnesses" to manage and validate AI-driven security findings for practical application.
Takeaway: If you are using AI for security analysis, consider developing a multi-stage "harness" or pipeline to manage the LLM's scope, validate findings, and reduce noise, as Cloudflare did with Mythos Preview.
Deep dive
- Cloudflare participated in Project Glasswing, testing Anthropic's security-focused LLM, Mythos Preview, on over 50 internal code repositories.
- Mythos Preview represents a significant advancement over general-purpose frontier models, specifically in its ability to construct exploit chains from multiple minor vulnerabilities and generate executable proof-of-concept (PoC) code.
- Unlike previous models that would identify bugs but stop short of proving exploitability, Mythos Preview can compile and run code in a scratch environment to confirm its hypotheses.
- The model displayed "organic refusals" to certain legitimate security research requests, but these guardrails were inconsistent and could be bypassed by rephrasing tasks, indicating the need for additional, robust safeguards in general-release cyber frontier models.
- A "signal-to-noise" problem persisted, with more false positives observed in memory-unsafe languages (C/C++) compared to memory-safe ones (Rust).
- Mythos Preview's output quality was higher than previous models, delivering clearer reproduction steps and fewer hedged findings, especially when producing PoCs.
- Cloudflare found that pointing a generic coding agent at a repository was ineffective for comprehensive vulnerability research due to context window limitations and lack of parallel processing.
- To address these limitations, Cloudflare developed an architectural "harness" with stages: Recon (architecture mapping), Hunt (concurrent bug searching with PoC tools), Validate (independent agent disproving findings), Gapfill (re-queueing uncovered areas), Dedupe, Trace (checking reachability of shared library bugs), and Feedback (looping back reachable traces).
- The findings suggest that while AI can greatly accelerate vulnerability discovery, defenders must also shorten their response timelines and adopt architectural defenses that make exploitation harder, anticipating that attackers will leverage similar AI capabilities.
Decoder
- Large Language Model (LLM): A type of artificial intelligence model trained on vast amounts of text data to understand, generate, and process human language, here specialized for security tasks.
- Exploit chain: A sequence of multiple, often individually minor, software vulnerabilities or attack primitives that are combined to achieve a more significant and damaging attack or system compromise.
- Proof-of-concept (PoC): A demonstration, often in code, that proves a particular vulnerability or exploit strategy is viable and can achieve its intended effect.
- Memory-unsafe languages (e.g., C, C++): Programming languages that provide direct memory access, allowing for common vulnerabilities like buffer overflows or use-after-free bugs if not handled carefully.
- Memory-safe languages (e.g., Rust): Programming languages designed to prevent common memory-related errors at compile time, reducing certain classes of vulnerabilities.
- Harness (in software testing/security): A framework or system that automates the execution, management, and validation of tests or security analysis tasks, often orchestrating multiple tools or models.
Original article
Cloudflare was invited to use Mythos Preview as part of Project Glasswing. The company pointed it at more than 50 of its repositories to see what it would find and to see how it works. This post shares what the company observed, what the model did well and what it didn't, and how it can be improved. Mythos Preview is a real step forward, and its existence raises questions about how security should be handled in the near future.
Explore all 43 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi