Native-speed vLLM transformers modeling backend

TL;DR: The transformers vLLM backend is now as fast (or faster) than custom vLLM implementations for many LLM architectures. Model authors can automatically leverage their transformers implementations to get ultra fast vLLM inference, for free.

# Upgrade the vllm pip package
uv pip install --upgrade vllm --torch-backend auto

The transformers library has become the reference modeling library for Machine Learning. It supports 450+ architectures through consistent APIs, and is designed with the main goal that model implementations are self contained and easy to understand. Going through transformers code makes it easy for contributors to learn how an architecture works, and then port it to other frameworks such as vLLM, SGLang, MLX, llama.cpp, and many others.

We have fully embraced this role in the ecosystem and are investing a lot of effort to make it easier. A big step in this direction was the integration last year of transformers as a modeling backend in vLLM. This has been allowing model authors to run transformers models (LLMs and VLMs alike) inside vLLM, without having to port anything. Transformers provides the modeling code, and vLLM provides extremely optimized inference techniques such as continuous batching and custom attention kernels.

This integration gets better now 🚀!

Showcase

We put the transformers modeling backend for vLLM head to head with vLLM's hand written native implementations across three very different Qwen3 models:

• 4B dense model on a single GPU
• 32B dense model on tensor parallelism
• 235B-parameter FP8 Mixture-of-Experts on data + expert parallelism on the same 8×H100 node

The result: the transformers modeling backend now meets or beats native throughput on every one of them.

Running any* Hugging Face model through the transformers modeling backend is a single flag — --model-impl transformers. It composes with the usual parallelism options, so nothing about your serving setup changes: