Robostral Navigate: single-camera AI navigation
Mistral's new 8B Robostral Navigate model enables robots to navigate complex environments using only a single RGB camera, outperforming multi-sensor systems.
Deep dive
- 8B parameter model trained exclusively in simulation.
- Achieved 76.6% success on R2R-CE unseen validation, surpassing previous state-of-the-art.
- Eliminates the need for depth sensors, LiDAR, or multiple cameras.
- Uses a 'pointing' approach to determine movement, increasing robustness to camera hardware variations.
- Employs CISPO (online reinforcement learning) after supervised training to improve exploration and failure recovery.
- Prefix-caching technique allows for training on entire episodes in a single forward pass, accelerating training cycles from months to days.
Decoder
- R2R-CE: Room-to-Room in Continuous Environments, a standard benchmark for evaluating how well AI agents follow natural language instructions to navigate 3D simulated interior spaces.
- Embodied AI: A field of research focused on building AI agents that operate within physical or simulated bodies (like robots) to interact with their environment.
- Prefix-caching: A memory optimization technique that caches the KV (key-value) cache of previous sequence tokens to avoid redundant computation during inference or training.
- CISPO: A specific reinforcement learning algorithm used for policy optimization in continuous control tasks.
Original article
Introducing Robostral Navigate
Robostral Navigate is an 8B model that enables robots to autonomously navigate complex environments using only a single RGB camera, achieving 76.6% success on unseen R2R-CE benchmarks—outperforming multi-sensor approaches while being more efficient. Built entirely in-house with simulated data and token-efficient techniques, it generalizes across robot types and adapts to real-world obstacles unseen during training. The model combines pointing-based navigation with reinforcement learning for continuous improvement, paving the way for unified embodied AI in robotics.
Today we're introducing Robostral Navigate, our first model built for embodied navigation. It's an 8B model that takes RGB images and a plain-language instruction and moves a robot through an environment:
“Leave the lobby, walk through the corridor, enter the supply room, and stop to face the second shelf.”
To perform such tasks, other models often employ depth sensors, LiDAR, or several cameras working together. Robostral Navigate uses only one ordinary RGB camera and no depth sensors, yet still achieves 76.6% on R2R-CE (Room-to-Room in Continuous Environments) validation unseen, the benchmark for following instructions in environments held out of training. Consequently, it beats the best single-camera approach by 9.7 points and the best system using depth or multiple cameras by 4.5 points, despite using neither.
Navigation
Our model is designed for robotic navigation, enabling robots to autonomously navigate complex environments, including offices, residential and commercial buildings, and outdoor settings.
This technology unlocks numerous applications across manufacturing, delivery, logistics, and hospitality, making it one of the most in-demand capabilities for our customers today. Give Robostral Navigate one instruction and it completes the entire task on its own, moving through a live space full of people and obstacles it was never shown, capable of adapting to any setting.
Highlights
- State-of-the-art performance on R2R-CE
- 79.4% Success Rate on validation seen
- 76.6% Success Rate on validation unseen
- Operates from a single RGB camera, with no LiDAR or depth sensors
- 8B model, built in-house and trained entirely in simulation
- Runs on wheeled, legged, and flying robots, and generalizes across robot sizes
- Robust to differences in camera intrinsics
- Token-efficient training via prefix-caching
Navigation via pointing
Given a task and a history of observations, Robostral Navigate predicts where the robot should move next via pointing: it infers the image coordinates of the target location in the robot's current camera view, together with the desired orientation upon arrival. Unlike commands relying on metric displacements, pointing makes the policy naturally robust to changes in camera intrinsics and world scale.
However, this method cannot handle cases where the target location lies outside the current field of view. When pointing does not apply, the model falls back to displacements in the robot's local coordinate frame, such as:
"Move 2 meters forward, 1.5 meters to the left, and turn 25 degrees left."
Built from the ground up
Robostral Navigate is built entirely in-house and does not rely on existing open-source VLMs.
The model is initialized from our vision-language model specialized for grounding tasks such as pointing, counting, and object localization. Navigation emerges as a natural extension of these capabilities: once it understands where things are, it learns how to move.
We built an efficient data generation pipeline entirely in simulation. This enabled rapid iteration on the data, resulting in a dataset of approximately 400,000 trajectories collected across 6,000 scenes.
Efficient supervised training
A key ingredient of Robostral Navigate is an efficient training algorithm based on prefix-caching. Using a tree-based attention-masking strategy, our method compresses an entire episode into a single sequence, enabling training on all time steps in a single forward pass while preventing information leakage between time steps.
Compared to training with one sample per time step, our approach reduces the number of training tokens by 22× while preserving all of the learning signals. In practice, this method transforms training runs that would take months into runs that complete in days.
Online reinforcement learning
We leverage our knowledge of post-training LLMs at scale, using online reinforcement learning, to boost the performance of Robostral Navigate. After the supervised training stage, we further improve the model's performance using CISPO, an online reinforcement learning algorithm. This enables the model to learn from trial and error, recover from failures, and acquire exploratory behaviors, effectively mitigating the distribution shift issue of vanilla behavior cloning. This alone improved the success rate by 3.2%. We are not seeing any plateauing, so we are confident that more training and more experiments will continue to push this number up.
What's Next
Robostral Navigate is only the first step toward a unified embodied agent.
We believe navigation is a foundational capability for general-purpose robotics. By combining large-scale simulation, efficient training, and strong grounding priors, Robostral Navigate demonstrates that state-of-the-art embodied navigation can be achieved with a compact model and a single RGB camera.
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi