DEVOURED
Stealing AI Reasoning Traces
Researchers discovered an architectural vulnerability allowing attackers to decrypt and steal proprietary LLM 'reasoning traces' via weaker models from the same provider.
What: Encrypted reasoning blocks are interchangeable across sessions and models; injecting a trace from a protected model into a less-guarded model forces it to output the trace as plaintext, leaking PII, credentials, and internal logic.
Why it matters: This demonstrates that client-side handling of 'hidden' model states creates a new, massive surface for data extraction that vendors are not yet prepared to secure.
Takeaway: Do not include PII or sensitive credentials in prompts, as the internal 'reasoning' chain—even if not intended for user viewing—may be extractable by third parties.
Deep dive
- Attackers can bypass anti-distillation protections to extract model logic.
- Researchers recovered 367 PII artifacts and 182 credentials from 315,320 publicly scraped reasoning blocks.
- The vulnerability allows for poisoning agentic rollouts by embedding malicious payloads directly into encrypted trace blocks.
- Disclosure impacts major providers including Anthropic, OpenAI, and Google.
Decoder
- Reasoning trace: The internal 'thought process' of a model, often containing sensitive information or logic that providers attempt to hide from users.
- Distillation: The process of training a smaller model to mimic the output of a larger, more powerful model.
Original article
Full article content is not available for inline reading.
Explore all 55 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi