Exploring Quantization Backends in Diffusers

Large diffusion models like Flux (a flow-based text-to-image generation model) can create stunning images, but their size can be a hurdle, demanding significant memory and compute resources. Quantization offers a powerful solution, shrinking these models to make them more accessible without drastically compromising performance. But the big question always is: can you actually tell the difference in the final image?

Before we dive into the technical details of how various quantization backends in Hugging Face Diffusers work, why not test your own perception?

Spot The Quantized Model

We created a setup where you can provide a prompt, and we generate results using both the original, high-precision model (e.g., Flux-dev in BF16) and several quantized versions (BnB 4-bit, BnB 8-bit). The generated images are then presented to you and your challenge is to identify which ones came from the quantized models.

Try it out here or below!

Often, especially with 8-bit quantization, the differences are subtle and may not be noticeable without close inspection. More aggressive quantization like 4-bit or lower might be more noticeable, but the results can still be good, especially considering the massive memory savings. NF4 often gives the best trade-off though.

Now, let's dive deeper.

Quantization Backends in Diffusers

Building on our previous post, "Memory-efficient Diffusion Transformers with Quanto and Diffusers", this post explores the diverse quantization backends integrated directly into Hugging Face Diffusers. We'll examine how bitsandbytes, GGUF, torchao, Quanto and native FP8 support make large and powerful models more accessible, demonstrating their use with Flux.

Before diving into the quantization backends, let's introduce the FluxPipeline (using the black-forest-labs/FLUX.1-dev checkpoint) and its components, which we'll be quantizing. Loading the full FLUX.1-dev model in BF16 precision requires approximately 31.447 GB of memory. The main components are:

Example (Flux-dev with BnB 4-bit): Example (Flux-dev with torchao INT8 weight-only): Example (Flux-dev with torchao INT4 weight-only): Example (Flux-dev with quanto INT8 weight-only): Example (Flux-dev with quanto FP8 weight-only) Example (Flux-dev with GGUF Q4_1) Example (Flux-dev with BnB 4-bit + enable_model_cpu_offload): Example (Flux-dev with fp8 layerwise casting + group offloading): Example (Flux-dev with torchao 4-bit + torch.compile):