DEVOURED
Salesforce Finds Better Ways to Co-Evolve Agents and Their Harnesses
Salesforce discovered that training smaller agent models on expert trajectories often backfires because it disrupts the carefully tuned alignment between the agent and its scaffolding.
What: Researchers found that smaller models like Qwen3-Coder and Gemma 4 regress when fine-tuned on expert data, so they created an on-policy 'expert-correction' pipeline that only retrains failing segments of an agent's rollout.
Why it matters: This reveals that agent performance is not just about model intelligence; it is about the 'model-harness fit'—the delicate balance between model weights and the tool-use environment.
Takeaway: Instead of full-trajectory imitation learning, implement a targeted correction loop where an expert model only intervenes at the specific step where the smaller model fails.
Deep dive
- Evaluated seven enterprise agent tasks.
- Found that fine-tuning weaker models on expert trajectories caused 4–30 point performance drops.
- Identified 'scaffold-model misalignment' as the primary failure mode.
- Introduced an automated pipeline using a meta-level MLE agent to identify and correct individual failing turns.
- Demonstrated that on-policy correction preserves model-harness compatibility.
Decoder
- Agent Harness: The system prompt, available tools, execution hooks, and context-management scaffolding surrounding an AI model.
- On-policy: An approach where an agent learns from its own current strategy rather than static, pre-recorded data.
- MLE Agent: A Maximum Likelihood Estimation agent, likely used here to automate the identification of optimal corrections in a decision-making sequence.
Original article
Title: Co-Evolving Harnesses and Models: On-Policy Correction Helps Weaker Models Catch Up Where Imitation Fails
Agent harnesses (the system prompt, tool set, execution hooks, and context-management scaffolding around a model) are a critical determinant of agentic task success. Automated harness evolution can enable smaller models to perform well on domain-specific tasks at a fraction of frontier-model cost. Since both the harness and model weights shape behavior, we ask how harness evolution and lightweight fine-tuning should be combined. Across seven enterprise agent tasks, we first evolve a harness with the weaker model, then find that a stronger expert often uses it more effectively, suggesting expert supervision could close the remaining gap. However, training the weaker model on the expert's complete trajectories under the evolved harness backfires: performance regresses on all seven tasks by 4 to 30 points across Qwen3-Coder and Gemma 4, even though the same procedure helps under the unevolved harness. Our analysis shows that imitation transfers knowledge and increases scaffold usage, but disrupts model-harness fit: the weaker model adopts the expert's planning strategy without the competence to execute it and no longer matches the harness evolved around its native planning style. We therefore develop an on-policy expert-correction pipeline, automated by a meta-level MLE agent, that localizes the failing turn in the weaker model's own rollout and asks the expert to rewrite only that turn. This preserves the model's planning style and combines the gains of harness evolution and model adaptation. Our results identify and resolve a source of contention between harness and weight updates, yielding a compatibility-preserving recipe for economical co-evolution on domain-specific enterprise tasks.
Explore all 56 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi