NavSpace Tests Whether AI Agents Can Actually Follow Spatial Instructions

Ask a delivery robot to “go to the second floor” or a home assistant to “stop between the two brown sofas,” and you are testing something most navigation benchmarks never measure: whether the agent actually understands space. A new benchmark called NavSpace, accepted to the IEEE International Conference on Robotics and Automation (ICRA 2026), found that even the best multimodal AI systems fail that test more often than they pass it.

The paper, “NavSpace: How Navigation Agents Follow Spatial Intelligence Instructions,” was posted to arXiv on October 9, 2025 (arXiv:2510.08173) by Haolin Yang, Yuxing Long, Zhuoyuan Yu, Zihan Yang, Minghan Wang, Jiapeng Xu, Yihan Wang, Ziyan Yu, Wenzhe Cai, Lei Kang, and Hao Dong, with a revised version posted in March 2026 ahead of the ICRA presentation. “Instruction-following navigation is a key step toward embodied intelligence,” the authors write in the paper's abstract, arguing that “prior benchmarks mainly focus on semantic understanding but overlook systematically evaluating navigation agents' spatial perception and reasoning capabilities.”

What NavSpace tests

NavSpace consists of 1,228 trajectory-instruction pairs spanning six task categories, built on the Habitat 3.0 simulator using photorealistic HM3D indoor scenes. The six categories were derived from a questionnaire survey and cover distinct spatial skills: vertical perception (interpreting floor references like “go to the topmost floor”), precise movement (following exact distance and angle commands such as “turn right 90 degrees”), viewpoint shifting (adopting an object's perspective, as in “imagine you are the television, move toward your front-left”), spatial relationship (ordinal and relational cues like “turn left at the third door” or “stop between the two brown sofas”), environment state (conditional instructions such as “if you see the keys, stop, otherwise go to the front door”), and space structure (layout-dependent behaviors like circling an object or finding the farthest sofa). Trajectories were collected by human annotators teleoperating agents through the scenes, with instructions drafted using GPT-5 assistance and then refined and cross-validated by separate human annotators to confirm each instruction was accurate and executable.

The researchers then evaluated 22 navigation agents across five categories: chance-level baselines, open-source multimodal large language models (including LLaVA-Video 7B and Qwen2.5-VL 72B), proprietary MLLMs (including GPT-5 and Gemini 2.5 Pro), lightweight navigation models (such as BEVBert and ETPNav), and dedicated navigation-focused large models (including NaVid and StreamVLN), scoring each on navigation error, oracle success rate, and success rate (SR). The results, in the authors' words, “lift the veil on spatial intelligence in embodied navigation” — and what they reveal is not encouraging. Open-source MLLMs scored below 10% average success rate, close to chance. Proprietary MLLMs did better, with GPT-5 the strongest of that group, but still averaged below 20% success. Lightweight navigation models largely failed outright, while dedicated navigation large models such as NaVid and StreamVLN outperformed both groups, showing what the authors describe as only “preliminary” spatial intelligence.

To address the gap, the team built SNav, a spatially intelligent navigation model initialized from LLaVA-Video 7B and fine-tuned on newly generated spatial training data covering cross-floor navigation, precise movement, environment-state inference, and spatial-relationship reasoning. SNav posted an average success rate of 26.0% on NavSpace, ahead of GPT-5's 14.2% and StreamVLN's 19.2%. In real-world trials on an AgiBot Lingxi D1 quadruped robot across office, campus, and outdoor settings, SNav reached a 32% success rate versus 14% for NaVid and 6% for NaVILA, according to the paper.

Why It Matters

Spatial reasoning has quietly become one of the starkest gaps in embodied AI. Large language and vision-language models have gotten remarkably good at describing what they see and answering questions about static images, but NavSpace's results suggest that translating spatial language into a sequence of physical actions — in an environment the agent has to keep perceiving and re-perceiving as it moves — is a fundamentally different and much harder problem. The paper's authors attribute the weak MLLM performance to “perception-to-action reasoning inconsistencies” and difficulty maintaining spatial context across frames, while lightweight navigation models fail because their semantic-to-action mapping is too shallow to handle instructions that require actual geometric reasoning. For any real-world deployment of navigation agents — warehouse robots, home assistants, delivery platforms — the difference between recognizing “the brown sofa” and correctly identifying the space “between the two brown sofas” is the difference between a system that works and one that quietly walks into furniture. NavSpace is among the first benchmarks to isolate that failure mode rather than let it hide inside broader semantic-navigation scores.

What to Watch

The NavSpace team has open-sourced the benchmark data, evaluation code, and the SNav baseline, which should make it a reference point for navigation research heading into ICRA 2026 in the coming months. Watch for follow-up submissions that attempt to close the gap between SNav's 26% and genuinely reliable performance, and for whether major labs start reporting NavSpace-style spatial scores alongside the semantic navigation metrics that have dominated the field to date. The real test will be whether spatial intelligence gains transfer cleanly from simulated HM3D scenes to messier real-world robots and outdoor environments, where SNav's real-robot results, while ahead of rivals, still topped out at roughly one-in-three successful runs.