
AH
Austin H. Cheng, Marcel M\"uller, Andreas Burger, Yeonghun Kang, Tsz Wai Ko, Marta Skreta, Luka Mucko, Ella Miray Rajaonson, Cher Tian Ser, J\'er\^ome F. Gonthier, Alex Zook, Varinia Bernales, Al\'an Aspuru-Guzik
· 1 min read
ResearcharXiv cs.LG
Nahual: A Sequence Model for Language and Atoms
arXiv:2611.19072v1 Announce Type: new
Abstract: Generative models play a rapidly growing role in the design and simulation of molecules. The most widely used applications of generative models are in large language models (LLMs), owing to the natural flexibility of language for specifying and answering queries. However, generalist LLMs do not effectively handle large sets of 3D coordinates, whereas specialist generative models of 3D molecules cannot reach the controllability afforded by language. To bridge these capabilities, we propose Nahual, a decoder-only autoregressive diffusion model for any sequence of text and 3D molecules. To enable expressive autoregression on continuous coordinates, we propose next-token diffusion, whereby a causal transformer denoises each token while attending to the clean tokens before it, without increasing context size. We train Nahual on a curated set of twelve 3D chemistry tasks ranging from microsolvation to molecular conformer search to adsorption on metal surfaces, with associated evaluation metrics and baselines. The result is a single set of end-to-end-trained weights that simultaneously performs all tasks and surpasses state-of-the-art models specialized for crystal structure prediction and property-conditional molecule generation. Our results demonstrate the viability of scaling generalist decoder-only models for native multimodal understanding and generation across language and chemistry. Code is available at https://github.com/the-matter-lab/nahual.
Original source
This story was published by arXiv cs.LG and written by Austin H. Cheng, Marcel M\"uller, Andreas Burger, Yeonghun Kang, Tsz Wai Ko, Marta Skreta, Luka Mucko, Ella Miray Rajaonson, Cher Tian Ser, J\'er\^ome F. Gonthier, Alex Zook, Varinia Bernales, Al\'an Aspuru-Guzik. SyncAI.news shows a preview; the complete article is on the publisher's site.
Read the full story on arxiv.org


