SyncAI.news, a Varaisys broadcasting
Turning Speech Language Models into Multilingual Listeners
T\

Tol\'{u}lop\'{e} \`{O}g\'{u}nr\`{e}m\'{i}, Dan Jurafsky, Chris Manning, Ahmet \"Ust\"un, Martijn Bartelds

· 1 min read

ResearcharXiv cs.CL

Turning Speech Language Models into Multilingual Listeners

arXiv:2609.33204v1 Announce Type: new Abstract: Speech Language Models (SLMs) that understand spoken language questions support only a few high-resource languages, limiting access to millions of people worldwide. This gap stems from the scarcity of multilingual speech instruction-tuning datasets. We present MULTISPEECHQA, a large-scale, synthetically generated and human-verified dataset comprising 9200 hours of 10.8 million spoken question-answer pairs in 23 typologically diverse languages. Using MULTISPEECHQA, we also introduce MULTISPEECH-BENCH, a multi-task benchmark for evaluating SLM performance on 23 languages. We compare the performance of a cascading system to open-weight and closed SLMs on MULTISPEECH-BENCH and find that the cascading system outperforms open-weight SLMs but not all closed SLMs. We use MULTISPEECHQA to finetune Qwen 2.5-Omni, which improves its performance on our benchmark. Our findings show that high-quality synthetic datasets offer a cheap solution to improving the multilingual capabilities of SLMs.

Original source

This story was published by arXiv cs.CL and written by Tol\'{u}lop\'{e} \`{O}g\'{u}nr\`{e}m\'{i}, Dan Jurafsky, Chris Manning, Ahmet \"Ust\"un, Martijn Bartelds. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News