Teaching AI to read a map

Look at a map of a shopping mall or a theme park. Within seconds, your brain processes the visual information, identifies your location, and traces the optimal path to your destination. You instinctively understand which lines are walls and which are walkways. This fundamental skill — fine-grained spatial reasoning — is second nature.

For all their incredible advances, multimodal large language models (MLLMs) often struggle with this particular task. While MLLMs can identify a picture of a zoo and list the animals you might find there, they may have a difficult time tracing a valid path from the entrance to the reptile house. They might draw a line straight through an enclosure or a gift shop, failing to respect the basic constraints of the environment. This reveals a critical gap: today’s models are excellent at recognizing what’s in an image, but they falter when they need to understand the geometric and topological relationships between the objects.

To address this challenge, in “MapTrace: Scalable Data Generation for Route Tracing on Maps”, we introduce a new task, dataset, and synthetic data generation pipeline designed to teach MLLMs the fundamental skill of tracing paths on maps. Our work shows that this complex spatial ability, largely absent from pre-trained models, can be explicitly taught through targeted, synthetically-generated data. We also open-source 2M question answer pairs generated with the proposed pipeline utilizing Gemini 2.5 Pro and Imagen-4 Models to encourage the research community to further explore this area.

The challenge: A lack of grounding in the physical world

This data bottleneck has held back progress. Without sufficient training examples, models lack the "spatial grammar" to interpret a map correctly. They see a soup of pixels, not a structured, navigable space.

The solution: A scalable pipeline for synthetic data

1. Generating diverse maps

2. Identifying traversable paths with an AI "Mask Critic"

3. Building a navigable graph