Back to Devoured - September 24, 2026
DEVOURED
RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

AI agents Arxiv
RRSI improves AI agent harnesses by regularizing the search process for self-improvement instead of overfitting to fixed benchmarks.
What: Researchers from Google Cloud AI, UNC, and Stanford introduced Regularized Recursive Self-Improvement (RRSI), a method that prevents agent harnesses from memorizing tasks, resulting in 22.9% better performance on out-of-distribution benchmarks compared to prior approaches.
Why it matters: Current agent self-improvement methods often suffer from 'benchmark hacking,' where the agent exploits task-specific leakage rather than learning generalizable skills.
Deep dive
  • RRSI replaces unconstrained evolution with a regularized search loop.
  • Introduces a 'leakage critic' to filter out benchmark-specific logic before evaluation.
  • Implements an 'annealed edit budget' to prevent over-optimization in later stages.
  • Uses cost-based pruning to remove agent components that do not justify their token usage.
  • Validated on coding suites, Harvey LAB, JobBench, APEX-Agents, and simulated environments.
Decoder
  • Harness: The surrounding code and infrastructure (prompts, tool definitions, memory) that manages how an AI model operates to complete tasks.
Original article

Evolved harnesses overfit the benchmark they are scored on. RRSI transfers.

RRSI improves every out-of-distribution benchmark without overfitting the split it evolves on. Prior methods do the opposite: large evolve-set gains that shrink or vanish once the benchmark changes, two of them ending below the harness they started from.

Regularize the search, not the harness

Every harness component stays editable. RRSI constrains the loop that edits it: how much one proposal may change, and which measured gains are allowed to stick.

Proposal side

Annealed edit budget: Early rounds may bundle a few coordinated edits to find a mechanism; late rounds get one attributable change.

Evidence-aware credit: Every candidate is logged with its hypothesis, diff, score and cost change, so the proposer builds on what worked and stops re-testing what failed.

Structured exploration: When progress stalls inside the noise band, budget is redirected to components the run has never touched.

Selection side

Leakage critic: Task names, entities, answers or benchmark-specific logic are rejected before a candidate is ever scored.

Noise-adjusted floor: A gain must clear the variance measured on the unchanged base harness.

Cost rule: Extra inference tokens have to be paid for by measured gain.

Pruning: Components that stop earning their place are flagged for deletion.

Every held-out benchmark improves

Evolve on one suite per domain, then run the harness unchanged everywhere else. Same tools, judge, trials and window as H0; the policy is Claude Opus 4.8.

Two rules act on cost directly

The cost rule refuses growth that is not paid for when it is proposed; pruning removes growth that stopped paying for itself since. No prior method carries either.

Watch the harness evolve, round by round

Four real runs, every candidate: what it proposed, what the critic said, why the gate kept or dropped it, and the exact diff.

BibTeX

@article{xia2026rrsi,
  title={RRSI: Regularized Recursive Self-Improvement of Agent Harnesses},
  author={Xia, Peng and Han, Rujun and Wang, Zifeng and Chen, Yanfei and Zhuang, Yufan and Lee, Yoonho and Huang, Chengsong and Yu, Han and CuiZhu, Zhongying and Ming, Yifei and Yao, Huaxiu and Gokturk, Burak and Pfister, Tomas and Lee, Chen-Yu},
  journal={arXiv preprint arXiv:2609.24972},
  year={2026}
}
Explore all 53 articles in today's digest

Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi