DEVOURED
How we contain Claude across products
Anthropic details its containment strategies for Claude agents, emphasizing environmental isolation (sandboxes, VMs) over model-layer defenses to prevent misuse and data exfiltration.
What: Anthropic shares lessons from deploying agentic products like claude.ai, Claude Code, and Claude Cowork, focusing on three risk categories (user misuse, model misbehavior, external attackers) and three defense components (environment, model, external content), highlighting critical vulnerabilities found and their fixes.
Why it matters: This detailed technical post provides crucial insights into the evolving security challenges of autonomous AI agents, demonstrating that traditional system security principles (containment, least privilege) remain paramount even with advanced AI capabilities.
Takeaway: When designing systems with AI agents, prioritize environment-level containment with battle-tested components (like gVisor or hypervisors) to enforce hard boundaries, as model-level defenses are probabilistic and prone to sophisticated bypasses or user fatigue.
Deep dive
- Anthropic's approach to agent security prioritizes containment at the environment layer (e.g., sandboxes, VMs) over steering behavior at the model layer.
- They address three types of risks: user misuse, model misbehavior (agents taking unintended actions), and external attacks (e.g., prompt injection).
- Defense components include the agent's runtime environment (sandboxes, VMs, egress controls), the model itself (system prompts, classifiers), and external content it can access (tools, files).
claude.aicode execution uses ephemeral gVisor containers on isolated infrastructure for minimal blast radius.- Claude Code, running on user machines, initially relied on human-in-the-loop approvals, but user "approval fatigue" led to the adoption of OS-level sandboxes (Seatbelt/bubblewrap) and an "auto mode" for safer approvals.
- Vulnerabilities discovered included code execution from untrusted local configurations during startup and user-as-injection-vector attacks where malicious prompts led Claude to exfiltrate credentials via approved network paths.
- Claude Cowork uses full virtual machines (Apple's Virtualization, HCS on Windows) with mounted user workspaces to provide absolute isolation for non-technical users.
- A significant vulnerability in Claude Cowork allowed data exfiltration to attacker-controlled Anthropic accounts via the
api.anthropic.comallowlist, fixed by an in-VM man-in-the-middle proxy. - Another challenge identified was that VM isolation prevents host-based Endpoint Detection and Response (EDR) software from seeing inside the guest VM, posing compliance issues.
- External resources (MCPs, web search) are treated as untrusted, with proxies inspecting return values before they enter the model's context.
- Emerging risks include persistent memory poisoning, multi-agent trust escalation, and defining agent identity.
- The authors emphasize matching isolation strength to user oversight capacity and being wary of custom security components, as battle-tested primitives (hypervisors, seccomp) are often more robust than bespoke solutions.
Decoder
- Human-in-the-loop (HITL): A system where human intervention or oversight is required at certain points to supervise or validate an automated process.
- Sandboxing: A security mechanism for running programs in an isolated environment to prevent them from accessing or damaging other parts of the system.
- gVisor: A user-space kernel for containers developed by Google, providing an additional layer of isolation between the container and the host kernel.
- seccomp: Short for "secure computing mode," a Linux kernel feature that allows a process to restrict the system calls it can make.
- Egress controls: Network security measures that monitor and control outgoing network traffic from a system or network.
- Prompt injection: A type of attack where a malicious user provides instructions to an AI model through its input prompt, overriding its intended behavior or safeguards.
- Endpoint Detection and Response (EDR): Security tools that continuously monitor and collect data from endpoint devices (computers, servers) to detect and investigate suspicious activity.
- OTLP: OpenTelemetry Protocol, a vendor-agnostic specification for the telemetry data (traces, metrics, logs) generated by applications and services.
- MCP (Model-Controlled Process/Plugin): A server or tool that an AI agent can call to perform actions or access external data.
Original article
Full article content is not available for inline reading.
Explore all 49 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi