SyncAI.news, a Varaisys broadcasting
A Systematic Study of Cross-Token Transfer for Data-Efficient Non-Verbal Text-to-Speech
JP

Jing Pan, Haibin Wu, Minxue Niu, Florian Metze, Yuzong Liu

· 1 min read

ResearcharXiv cs.CL

A Systematic Study of Cross-Token Transfer for Data-Efficient Non-Verbal Text-to-Speech

arXiv:2611.19145v1 Announce Type: new Abstract: Non-verbal vocalizations (NVs), such as laughter and breath, carry affective cues that words alone do not convey. Yet NV data remains scarce because speech annotation pipelines often omit or normalize these events. Existing work has improved NV text-to-speech through model design and data curation, but whether one NV token can help a model learn another has received little systematic study. We investigate this cross-token transferability in a controlled CosyVoice 2 setup. We first trace a laughter-only scaling curve, then hold laughter at 200 or 1,600 examples while adding breath, sigh, cough, lip-smack, or neutral speech. With only 200 laughter examples, adding 3,200 cough examples raises end-to-end laughter F1 from 0.170 to 0.529, surpassing the 0.495 obtained from 1,600 laughter examples alone. At the 1,600-example anchor, cough reaches 0.695, nearly matching 6,400 laughter examples (0.697). Waveform-level Whisper-D scoring and a semantic-token transcriber show consistent trends. Cross-token transfer therefore offers a complementary route to low-resource NV generation.

Original source

This story was published by arXiv cs.CL and written by Jing Pan, Haibin Wu, Minxue Niu, Florian Metze, Yuzong Liu. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News