Meta is back with Muse Glimmer: local, agentic, multimodal, and open source

Great news from the OGs of open source LLMs! Muse Glimmer, released today, is Meta’s new multimodal model, especially designed for local agentic use cases. Distilled from Muse to 30B parameters, and released under the Apache 2.0 license, it’s ideal deploying locally for privacy, reducing costs, or just hacking around. It’s intended for privacy-aware applications such as coding, document analysis, personal assistants, Claw- or Hermes-like setups.

To celebrate, we are shipping with Meta day-0 support in transformers, llama.cpp, vLLM, Inference Endpoints, and other libraries. We built a few cool things and explain our findings in this blog.

Check out the demos below for inspiration.

You can find Muse Glimmer on the Hugging Face Hub.

Benchmarks

Benchmark results

Scores are reported as published. Bold indicates the best result among the compared models; ↓ indicates lower is better.

Architecture

Muse Glimmer is a dense 30B parameter model consisting of:

2B ViT-style encoder for vision (Perception Encoder)
28B parameter text decoder

In addition to the main VLM, there’s also a speculative decoding drafter implemented on DFlash. Usage of this module is optional, and it can provide much faster generation in exchange for some memory cost. We found this drafter to be particularly well suited to structured content generation such as coding.

Text Decoder

The language model uses the following architecture components:

Perception Encoder

After transformer, pixel shuffle concatenates 2x2 groups of neighboring spatial tokens which reduces the number of image tokens 4x without discarding their channels. The merged features are then projected to the shared embedding space of the text decoder.

Transformers

Upgrade transformers to the latest version to be able to use Muse Glimmer.

pip install --upgrade transformers accelerate

The same snippet runs unchanged on NVIDIA (CUDA), AMD (ROCm) and Intel (XPU) GPUs, device_map="auto" places the model on whichever accelerator is available.