DEVOURED
Agent Lightning v1.0: Towards Harnessed Agentic RL
Agent Lightning v1.0 enables reinforcement learning for AI agents by letting the harness own the interaction loop.
What: Zhiyuan He and team released Agent Lightning v1.0, a framework that allows models like Qwen3.5-9B to improve via reinforcement learning by interacting with custom agent harnesses. It achieved a 14.6-point gain on SWE-bench Verified using 6,000 training examples.
Why it matters: This shift moves the training focus from static datasets to agent-environment loops, where the harness itself—not the trainer—manages tool use and state, making the harness a critical component of the model's post-training evolution.
Deep dive
- Implemented in 3,500 lines of code.
- Supports arbitrary agent harnesses.
- Decouples the training engine from the environment interaction loop.
- Specifically addresses challenges in retokenization, advantage calculation, and loss normalization in agentic RL.
- Provides a full pipeline for coding-agent RL.
- Demonstrates performance gains on instruction-following, search, and coding tasks.
Decoder
- Agent harness: A system that manages an AI agent's environment, providing tools, maintaining state, and handling control flow.
- Harnessed agentic RL: An approach where the deployment-time environment harness directly influences the reinforcement learning training process.
Original article
Agent Lightning v1.0: Towards Harnessed Agentic RL
Abstract:Modern agents operate inside agent harnesses that manage tools, context, and control flow, making the harness a critical part of the agent system. Our original Agent Lightning introduced a disaggregated architecture that connects arbitrary agents to RL training through an LLM endpoint proxy, an approach later adopted by frameworks such as verl Uni-Agent, AReaL 2.0, slime, and Polar. We refer to this paradigm as harnessed agentic RL, where the deploy-time harness directly participates in model post-training. Harnessed agentic RL differs fundamentally from traditional agentic RL: the harness, rather than the training engine, owns the environment interaction loop, while the trainer observes only sequences of LLM request-response pairs. This introduces challenges in retokenization, sample merging, advantage calculation, loss normalization, and backend scheduling, which can substantially affect training stability and effectiveness. We present Agent Lightning v1.0, a lightweight framework for harnessed agentic RL implemented in approximately 3,500 lines of code. It supports arbitrary agent harnesses and serves as a practical testbed for studying these challenges. We evaluate it on instruction-following, search, and coding agents, and provide a complete reproducible pipeline for coding-agent RL. Using only 6K training examples and modest compute, RL improves Qwen3.5-9B on SWE-bench Verified from 41.8% to 56.4%, a 14.6-point absolute gain. We release the complete workflow and training scripts to facilitate reproducible research on harnessed agentic RL.
Explore all 57 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi