Transformers.js v3: WebGPU Support, New Models & Tasks, and More…

After more than a year of development, we're excited to announce the release of 🤗 Transformers.js v3!

Highlights include:

• WebGPU support (up to 100x faster than WASM!)
• New quantization formats (dtypes)
• A total of 120 supported architectures
• 25 new example projects and templates
• Over 1200 pre-converted models on the Hugging Face Hub
• Node.js (ESM + CJS), Deno, and Bun compatibility
• A new home on GitHub and NPM

Installation

You can get started by installing Transformers.js v3 from NPM using:

npm i @huggingface/transformers

Then, importing the library with

import { pipeline } from "@huggingface/transformers";

or, via a CDN

import { pipeline } from "https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.0.0";

For more information, check out the documentation.

WebGPU support

WebGPU is a new web standard for accelerated graphics and compute. The API enables web developers to use the underlying system's GPU to carry out high-performance computations directly in the browser. WebGPU is the successor to WebGL and provides significantly better performance, because it allows for more direct interaction with modern GPUs. Lastly, it supports general-purpose GPU computations, which makes it just perfect for machine learning!

As of October 2024, global WebGPU support is around 70% (according to caniuse.com), meaning some users may not be able to use the API.

If the following demos do not work in your browser, you may need to enable it using a feature flag:

• Firefox: with the dom.webgpu.enabled flag (see here).
• Safari: with the WebGPU feature flag (see here).
• Older Chromium browsers (on Windows, macOS, Linux): with the enable-unsafe-webgpu flag (see here).

Usage in Transformers.js v3

Thanks to our collaboration with ONNX Runtime Web, enabling WebGPU acceleration is as simple as setting device: 'webgpu' when loading a model. Let's see some examples!

Example: Compute text embeddings on WebGPU (demo)

Example: Perform automatic speech recognition with OpenAI whisper on WebGPU (demo)

See full code example List of new models