
HY
Heesup Yun, Isaac Kazuo Uyehara, Earl Ranario, Lars Lundqvist, Christine H. Diepenbrock, Brian N. Bailey, J. Mason Earles
· 1 min read
ResearcharXiv cs.CV
Using Vision Language Foundation Models to Generate Plant Simulation Configurations via In-Context Learning
arXiv:2603.08930v2 Announce Type: replace
Abstract: This paper introduces a benchmark for evaluating whether vision-language models (VLMs) can generate plant simulation configurations from imagery using in-context learning. We study this benchmark for cowpea plot reconstruction for plant simulations, where the VLM needs to generate structured JSON configurations that include field and plant information. Open-source multimodal models from Gemma 4 and Qwen3.5 families are evaluated on a synthetic cowpea dataset with known JSON ground truth and on a real drone orthophoto dataset with field-collected JSON. Five in-context learning methods are used, from format restriction instruction to few-shot image examples with auxiliary grounding information. The results show that VLMs can generate valid JSON outputs, can generally estimate days after planting (DAP), plant counts, plant locations, sun angles, and leaf chlorophyll content, and can render approximate simulations of cowpea plots. Error metrics fluctuate across model families and often remain worse than dataset baselines, particularly when VLMs' pretrained knowledge dominates over weak visual evidence. These results position image-to-simulation JSON generation as a promising but currently challenging task, and establish a benchmark for studying how multimodal reasoning, prompt design, and the sim-to-real domain gap affect plant phenotyping tasks.
Original source
This story was published by arXiv cs.CV and written by Heesup Yun, Isaac Kazuo Uyehara, Earl Ranario, Lars Lundqvist, Christine H. Diepenbrock, Brian N. Bailey, J. Mason Earles. SyncAI.news shows a preview; the complete article is on the publisher's site.
Read the full story on arxiv.org


