Mistral's Robostral Navigate Guides Robots With Just One Camera
Mistral introduced Robostral Navigate, an 8B model that lets robots follow spoken instructions through buildings using a single ordinary camera.
What Mistral built
Mistral says it is introducing Robostral Navigate, its first model built for embodied navigation. It is an 8B model that takes RGB images and a plain-language instruction and moves a robot through an environment, following commands such as leaving a lobby, walking through a corridor, entering a supply room, and stopping to face a shelf. Mistral says the model is designed for robots operating in offices, residential and commercial buildings, and outdoor settings.
Where other systems typically lean on depth sensors, LiDAR, or several cameras working together, Mistral says Robostral Navigate uses only one ordinary camera and no depth sensors. Despite that, the company reports it still achieves 76.6% success on R2R-CE validation unseen, the benchmark for following instructions in environments withheld from training, beating the best single-camera approach by 9.7 points and the best system using depth or multiple cameras by 4.5 points.
How it decides where to move
Mistral says that, given a task and a history of observations, Robostral Navigate predicts where the robot should move next by pointing: it infers the image coordinates of the target location in the robot's current camera view, along with the desired orientation on arrival. The company says this pointing approach makes the policy naturally robust to changes in camera setup and world scale, unlike commands based on metric displacements.
Mistral notes this method breaks down when the target location sits outside the current field of view. In that case, the model falls back to displacement commands in the robot's own coordinate frame, such as an instruction to move 2 meters forward, 1.5 meters to the left, and turn 25 degrees left.
Built without existing open models
Mistral says Robostral Navigate is built entirely in-house and does not rely on existing open-source vision-language models. Instead, it is initialized from Mistral's own vision-language model specialized for grounding tasks such as pointing, counting, and object localization, with navigation emerging, in Mistral's telling, as a natural extension of those capabilities.
Mistral built what it describes as an efficient data-generation pipeline entirely in simulation, producing a dataset of approximately 2.4 million trajectories collected across 350k scenes. The company says a training method based on prefix-caching, using a tree-based attention-masking strategy, compresses an entire episode into a single sequence and reduces the number of training tokens by 22 times compared with training one sample per time step, while preserving the learning signal. Mistral says this turns training runs that would otherwise take months into runs that finish in days.
Reinforcement learning on top
After the supervised training stage, Mistral says it further improves the model using CISPO, an online reinforcement-learning algorithm, which lets the model learn from trial and error, recover from failures, and pick up exploratory behaviors. Mistral reports this step alone improved the success rate by 3.2%, and says it is not seeing any plateau, giving the company confidence that more training and experiments will keep pushing that number up.
Key facts
- Robostral Navigate is an 8B model that navigates using a single RGB camera, with no LiDAR or depth sensors.
- Mistral reports 79.4% success on validation seen and 76.6% success on validation unseen for the R2R-CE benchmark.
- It beats the best single-camera approach by 9.7 points and the best multi-sensor approach by 4.5 points.
- The model trained on roughly 2.4 million trajectories across 350k simulated scenes.
- A prefix-caching training method cut training tokens by 22 times; a later reinforcement-learning stage added 3.2% more success.
Check your understanding
By how many points does Robostral Navigate beat the best single-camera approach on R2R-CE validation unseen, according to Mistral? What training technique does Mistral say reduced the number of training tokens by 22 times?