SyncAI.news, a Varaisys broadcasting
Agri-MRAG: Parameter-Enhanced Multimodal Retrieval-Augmented Generation for Agricultural Visual Question Answering
FL

Fuyong Liu, Xiantao Lei, Xining Zhang

· 1 min read

ResearcharXiv cs.CV

Agri-MRAG: Parameter-Enhanced Multimodal Retrieval-Augmented Generation for Agricultural Visual Question Answering

arXiv:2611.18771v1 Announce Type: new Abstract: Multimodal Retrieval-Augmented Generation (MRAG) has shown promise for Agricultural Visual Question Answering (Agri-VQA), but existing methods still face challenges in retrieving relevant knowledge while avoiding redundancy and in integrating retrieved evidence beyond simple prompt concatenation. To address these issues, we propose Agri-MRAG, a multimodal retrieval-augmented framework for agricultural VQA. Specifically, we first introduce a semantic-consistency-guided clustering mechanism that combines multimodal clustering and weighted relevance scoring to select high-confidence knowledge subsets and adaptively determine the retrieval size $k$. We then design a parameterized knowledge injection strategy, where retrieved agricultural knowledge is encoded into LoRA adapters and merged into the feed-forward network (FFN) layers of the MLLM, allowing external knowledge to influence generation through parameter-level adaptation. In addition, a perplexity-guided (PPL-guided) routing mechanism is employed to switch between implicit parametric reasoning and explicit retrieval-assisted generation according to model uncertainty, balancing inference efficiency and answer reliability. For evaluation, we use PlantVillageVQA as a broad agricultural benchmark and construct PlantVQA, a region-specific forestry and fruit crop VQA dataset containing 7,514 images and 15,028 expert-verified question-answer pairs. Experiments show that Agri-MRAG achieves ROUGE-L scores of 41.21 and 46.58 on the two datasets, outperforming mRAG by 11.70 and 4.51 points, and VisRAG by 8.80 and 3.39 points, respectively.

Original source

This story was published by arXiv cs.CV and written by Fuyong Liu, Xiantao Lei, Xining Zhang. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News