Building smarter maps with GPT-4o vision fine-tuning

Grab⁠(opens in a new window) is a leading food delivery and rideshare company in Southeast Asia, servicing almost 42 million monthly users across eight countries.

With more than 6 million driver-partners registered on the platform and 3.5 billion annual transactions in 2023, Grab’s impact extends beyond ride-hailing and food delivery. Grab turns street-level imagery collected from their drivers into mapping data used to power GrabMaps⁠, a mapping service enabling all of their Southeast Asia operations. 

GPT‑4o vision fine-tuning enables the service to correctly localize traffic signs and count lane dividers to refine the mapping data. Today, GrabMaps not only supports Grab’s services but also powers enterprises with its location intelligence capabilities.

Mapping Southeast Asia for better mobility

Southeast Asia presents a uniquely challenging environment for mapping. Its road networks include narrow, one-way streets optimized for motorbikes and pedestrians, rapidly changing urban landscapes, and limited coverage from conventional mapping providers. 

“To meet the needs of the region, we had to build something hyperlocal and dynamic—mapping Southeast Asia as it evolves.”

Adrian Margin, Head of Data Science for Geo Mapping at Grab

Grab turned to OpenAI’s GPT‑4o with vision fine-tuning to overcome these obstacles. 

By using its network of motorbike drivers and pedestrian partners, each equipped with 360-degree cameras, GrabMaps collected millions of street-level images to train and fine-tune models for detailed mapmaking. 

GPT‑4o’s vision fine-tuning capabilities allowed GrabMaps to localize speed limit signs, turn restrictions, places, and road geometries more accurately.

Using vision fine-tuning to automate mapmaking

Grab’s initial experiments focused on matching speed limit signs to their corresponding roads.

Starting with a baseline accuracy of 67%, the model improved to 80% after two rounds of fine-tuning—a 13-percentage point gain.