DEVOURED
NVIDIA's Cosmos Framework
NVIDIA released the Cosmos Framework to support training, simulation, and deployment of physical AI models using GPU-accelerated NIM microservices.
What: The framework enables generation of synchronized video and LiDAR data for robotics and autonomous vehicle testing. It includes modular recipes for reinforcement learning and fine-tuning, and provides NIM containers that use quantization and speculative decoding to achieve 2–3x speedups on H200 GPUs.
Why it matters: NVIDIA is standardizing the 'world model' development lifecycle by providing a unified stack for both training and accelerated deployment, targeting the high-compute demands of physical AI.
Deep dive
- Supports various models including Cosmos 3, Alpamayo, and GR00T.
- Offers online/offline RL pipelines: DPO, GRPO, PPO, and DiffusionNFT.
- Enables synthetic data generation via Cosmos3-Nano simulation recipes.
- Provides NIM microservices for optimized inference using vLLM and TensorRT-LLM.
- Requires significant VRAM for generation tasks (up to 58 GiB in BF16 for Cosmos3-Nano).
Decoder
- NIM: NVIDIA Inference Microservice, a containerized package for deploying optimized AI models.
- Speculative Decoding: A technique that uses a smaller, faster model to generate draft tokens, which a larger model then verifies to increase throughput.
- LiDAR: Light Detection and Ranging, a remote sensing method that uses light in the form of a pulsed laser to measure ranges.
Original article
Full article content is not available for inline reading.
Explore all 41 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi