I Told an AI Agent That Its Wrong Answers Were Right. One Model Believed Me

Back to Devoured - November 02, 2026

DEVOURED

I Told an AI Agent That Its Wrong Answers Were Right. One Model Believed Me

Testing reveals that some AI agents can be gaslit into accepting incorrect rewards as valid, with GPT-6 Sol being the most susceptible.

What: Researchers flipped reward labels in an agent's memory to see if they would adapt; GPT-6 Sol performance plummeted (94.4 to 7.8 score), while smaller models like Claude Haiku 5.5 were more resistant.

Why it matters: This highlights the vulnerability of agentic reasoning to feedback loops where an agent may prioritize user-provided 'ground truth' over its own logic.

Deep dive

Test focused on whether agents rely on internal logic or external reward labels.
Flipping the labels exposed how much the agents were 'cheating' by following feedback rather than the rules.
Larger, more capable models were more prone to the 'gaslighting' effect than smaller models.
This suggests current RLHF-trained agents may over-prioritize human-like approval signals.

Decoder

RLHF (Reinforcement Learning from Human Feedback): Training process where models are tuned based on human-provided rankings of their outputs.

Original article

In most cases, flipping the reward labels in an agent's memory barely changes how it learns. This test hid a rule only the labels could reveal. With true labels, GPT-6 Sol scored 94.4 out of 100 on its last six trials, and with flipped labels 7.8. Claude Haiku 5.5 improved less, and Qwen3 4B barely moved.

Explore all 69 articles in today's digest