DEVOURED
Contrastive Language Models
Contrastive Language Models (CLMs) offer 9x lower latency by using a contrastive learning objective to bridge agent states and actions.
What: The 8B parameter CLM model matches the performance of existing frontier models like Jev on tool-calling and gaming tasks while significantly reducing inference latency. It is trained on 60M Q&A pairs and 1M agentic trajectories.
Why it matters: Contrastive learning approaches are challenging standard LLM architectures by prioritizing state-action alignment, which is critical for real-time agent responsiveness.
Deep dive
- Achieves parity with Jev on benchmarks while operating at lower latency.
- Uses 60M Nemotron-derived Q&A pairs and 30M synthetic hard negatives for pre-training.
- Fine-tuned on 1M agentic trajectories for improved reasoning in tool-use.
- Optimized for System One agentic tasks where rapid reaction times are prioritized.
Decoder
- Contrastive Learning: A training method that learns to represent data by maximizing the similarity between related inputs and minimizing it between unrelated ones.
- Hard Negatives: Difficult training examples that are similar to the ground truth but are incorrect, forcing the model to learn finer distinctions.
Original article
Contrastive Language Models (CLMs) are a new class of System One model trained with a contrastive learning objective that connects states and actions. CLM-8B delivers performance comparable to Jev across computer-use, gaming, and tool-calling tasks while achieving up to 9x lower latency. It also sets a new state-of-the-art on changing agentic coding benchmarks. The model is pre-trained on 60M Nemotron Q&A pairs, mid-trained on 30M synthetic hard negatives, and post-trained on 1M agentic trajectories.
Explore all 50 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi