SyncAI.news, a Varaisys broadcasting
Where Mixture Recognition Fails: An Ingredient-Level Audit of Unseen Combinations in SmellNet
ZL

Zirui Liu, Zheming Zhu

· 1 min read

ResearcharXiv cs.LG

Where Mixture Recognition Fails: An Ingredient-Level Audit of Unseen Combinations in SmellNet

arXiv:2611.19297v1 Announce Type: new Abstract: An electronic nose must name every ingredient of a mixture, including combinations it has never measured. SmellNet, a public benchmark for this task, scores ingredients with a hit rate that is told how many each sample contains. We audit what this protocol hides. We hold out ingredient combinations from the SmellNet training data, evaluate four architectures (MLP, CNN, LSTM and a Transformer encoder) without revealing the ingredient count, and repeat the evaluation once on the official unseen split. Under the count-informed protocol, 90-96% of mixtures receive an ingredient that is absent, on our folds and on the official split alike. On our folds, every model identifies one ingredient reliably but the second rarely, and it completes the pair with a familiar training partner about twice as often as chance. On the official split, even the first-ranked ingredient is correct in only 29-46% of binary mixtures, often because a frequent training ingredient that is absent from every test pair is ranked first, and on its combinations that are truly absent from training no model names both ingredients in more than 1.2% of them. The failures fall on the same ingredients for every architecture, and a frequency correction only moves errors between ingredients. Mixture recognition is therefore limited by ingredients the models cannot separate and by a preference for frequent ingredients, not by the architecture, and it should be evaluated with set-level errors and without oracle cardinality.

Original source

This story was published by arXiv cs.LG and written by Zirui Liu, Zheming Zhu. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News