DEVOURED
TaoLive post-trains a smaller model to follow a changing harness
TaoLive's 'Harness-Aware Training' teaches compact 35B models to adapt to changing tool interfaces at runtime rather than memorizing a fixed setup.
What: TaoLive researchers introduced Harness-Aware Training (HAT), a post-training technique that incorporates variable tool schemas and prompt structures into the training distribution. This allows a 35B parameter model to remain functional in e-commerce workflows even when business logic, tool names, or prompt templates change, achieving 94.8% accuracy on live-stream QA benchmarks.
Why it matters: Decoupling model behavior from a fixed interface makes deployment significantly more flexible, as teams no longer need to retrain or fine-tune models every time a business 'harness'—the set of tools and rules agents use—updates.
Takeaway: If you are managing agentic workflows that require frequent updates to tool schemas or prompt structure, consider implementing a harness-agnostic training loop to prevent your model from overfitting to static configuration strings.
Decoder
- Harness: The collection of tools, system prompts, skills, and interaction constraints that an agent uses to interact with an environment.
- Supervised Fine-Tuning (SFT): The process of training a pre-trained model on a labeled dataset to specialize it for a specific task.
- On-policy distillation: A technique where a model learns to imitate a stronger model's behavior while interacting with an environment, helping it maintain performance during adaptation.
Original article
Title: TaoLive Digital Avatar Agent Technical Report: Training Agents to Evolve with Their Harness
Abstract: AI-powered digital-avatar streamers in live e-commerce must answer product questions, engage viewers, and execute changing business strategies in real time. This requires low latency, factual and effective replies, and rapid adaptation to updated campaign, compliance, and style requirements. We develop an evolvable Harness that decouples Skills, Hooks, system prompts, and tools from model weights, allowing runtime behavior to change without retraining. However, Harness evolution creates a moving execution environment: compact models fine-tuned on one configuration may memorize names, schemas, and prompt templates rather than follow the Harness currently provided, while stronger zero-shot models are too slow for real-time use. We address this tension with Harness-Aware Training (HAT), which makes Harness states part of the training distribution. HAT applies task-preserving Harness-State Augmentation (HSA) to Skills, tool schemas, prompt structures, and interaction constraints, and comprises three stages: HSA-based supervised fine-tuning, general on-policy distillation to recover general capabilities, and HSA-based agentic reinforcement learning in a production-informed live-room simulator. Across four evaluation sets with more than 4,500 cases, our compact 35B model scores 94.8 on real-world Live-Stream QA, versus 80.3 for the base model and 93.0 for the strongest evaluated general LLM, while scoring 94.6 on Harness-Variant QA and retaining 83.5 on IFEval. By contrast, fixed-Harness SFT reduces IFEval by 7.7 points. In a controlled complete-agent replay on one NVIDIA H20 GPU with MTP enabled, the system achieves 3.407 s P50 and 8.114 s P95 latency. These results show that HAT produces a latency-feasible compact agent that remains effective under evaluated Harness changes without sacrificing general instruction following.
Explore all 58 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi