Introducing Devin Security Swarm
Cognition's new Devin Security Swarm uses 'Agentic MapReduce' to shard repositories and verify vulnerabilities in isolated sandboxes.
Deep dive
- Uses Agentic MapReduce to split large repositories into shards.
- Employs specialized agents to analyze shards in parallel.
- Aggregates findings into a single report.
- Performs runtime validation of potential vulnerabilities in sandboxed environments.
- Evaluated against 50 real-world GHSA vulnerabilities across 14 programming languages.
- Aims to solve the difficulty of maintaining context in large-scale codebase reasoning.
Decoder
- Agentic MapReduce: A pattern that distributes complex reasoning tasks across multiple autonomous agents working on isolated segments (shards) of a codebase, then aggregates the results.
- GHSA: GitHub Security Advisory, a database of publicly disclosed security vulnerabilities in open-source projects.
Original article
Introducing Devin Security Swarm
A more cost effective and accurate way to find security vulnerabilities in complex codebases, based on a new architecture: Agentic MapReduce.
In testing, Devin Security Swarm found 36 of 50 real-world GHSA vulnerabilities at 30% lower cost per finding than the next most accurate alternative.
We built a new architecture for whole-codebase reasoning that we’re calling Agentic MapReduce.
Security scanning is different from most coding tasks: a report is only trustworthy if the whole codebase is considered. But most agentic systems struggle to scale reasoning across large repos.
Devin maps relevant signals across the repo, fans out focused agents over bounded shards, reduces their findings into one report, then verifies serious vulnerabilities in isolated sandboxes before marking them confirmed.
The result is simultaneously more efficient and more accurate than other tools. We evaluated a variety of security scanning tools on a dataset of 50 GHSA vulnerabilities across 14 languages including Go, Rust, Python, Ruby, Java, C#, JavaScript, C, Swift, Dart, and Elixir. The dataset spans opens source repos of various sizes and of many software categories.
Beyond excelling on our eval, Devin Security Swarm also found critical vulnerabilities that other tools missed, like a PHP sandbox bypass via template injection, an argument injection through metadata value parsing, and an overly broad deserialization surface.
Security Swarm is a new pillar of Devin for Security: a suite of tools to help you find vulnerabilities, validate their exploitability at runtime, and ship remediation PRs.
Learn more and try it today at: devin.ai/security
We’re also publishing extensive documentation and technical materials about Agentic MapReduce, including a deep-dive on our evals.
Read our announcement: cognition.com/blog/introducing-devin-security-swarm
Learn about Agentic MapReduce: devin.ai/blog/agentic-map-reduce
Check out the evals: devin.ai/blog/security-swarm-eval
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi