Recent advances in Vision-and-Language Navigation in Continuous Environments (VLN-CE) have leveraged multimodal large language models (MLLMs) to achieve zero-shot navigation. However, existing methods often rely on panoramic observations and two-stage pipelines involving waypoint predictors, which introduce significant latency and limit real-world applicability.
In this work, we propose Fast-SmartWay, an end-to-end zero-shot VLN-CE framework that eliminates the need for panoramic views and waypoint predictors. Our approach uses only three frontal RGB-D images combined with natural language instructions, enabling MLLMs to directly predict actions. To enhance decision robustness, we introduce an Uncertainty-Aware Reasoning module that integrates (i) a Disambiguation Module for avoiding local optima, and (ii) a Future-Past Bidirectional Reasoning mechanism for globally coherent planning.
Experiments on both simulated and real-robot environments demonstrate that our method significantly reduces per-step latency while achieving competitive or superior performance compared to panoramic-view baselines. These results demonstrate the practicality and effectiveness of Fast-SmartWay for real-world zero-shot embodied navigation.
Overall workflow of our proposed zero-shot VLN-CE framework. The navigation process begins with panoramic RGB-D observations at the initial or disambiguation stage, and uses three frontal RGB-D views during the step-wise stage. The system first constructs structured prompts, then extracts spatial and semantic descriptions, and finally sends both together to the Multimodal Large Language Model (MLLM). Within the Decision Process, the MLLM performs step-wise reasoning and incorporates Future-Past Bidirectional Reasoning (FPBR) to ensure globally consistent planning. It also determines whether the robot is confused based on the instruction and context. If so, the Disambiguation Module is triggered to collect a panoramic observation and replan. The robot used in the figure is a Hello Robot equipped with an Intel RealSense camera mounted at a height of 125 cm.
Fast-SmartWay: Panoramic-Free End-to-End Zero-Shot Vision-and-Language Navigation
@article{shi2025fastsmartway,
title={Fast-SmartWay: Panoramic-Free End-to-End Zero-Shot Vision-and-Language Navigation},
author={Shi, Xiangyu and Li, Zerui and Qiao, Yanyuan and Wu, Qi},
journal={arXiv preprint arXiv:2511.00933},
year={2025}
}
This website is adapted from Nerfies, which is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License .