Train 400x faster Static Embedding Models with Sentence Transformers

TL;DR

This blog post introduces a method to train static embedding models that run 100x to 400x faster on CPU than state-of-the-art embedding models, while retaining most of the quality. This unlocks a lot of exciting use cases, including on-device and in-browser execution, edge computing, low power and embedded applications.

We apply this recipe to train two extremely efficient embedding models: sentence-transformers/static-retrieval-mrl-en-v1 for English Retrieval, and sentence-transformers/static-similarity-mrl-multilingual-v1 for Multilingual Similarity tasks. These models are 100x to 400x faster on CPU than common counterparts like all-mpnet-base-v2 and multilingual-e5-small, while reaching at least 85% of their performance on various benchmarks.

Today, we are releasing:

• The two models (for English retrieval and for multilingual similarity) mentioned above.
• The detailed training strategy we followed, from ideation to dataset selection to implementation and evaluation.
• Two training scripts, based on the open-source sentence transformers library.
• Two Weights and Biases reports with training and evaluation metrics collected during training.
• The detailed list of datasets we used: 30 for training and 13 for evaluation.

We also discuss potential enhancements, and encourage the community to explore them and build on this work!

Click to see Usage Snippets for the released models

The usage of these models is very straightforward, identical to the normal Sentence Transformers flow:

English Retrieval

Multilingual Similarity

Table of Contents

What are Embeddings?

Embeddings are one of the most versatile tools in natural language processing, enabling practitioners to solve a large variety of tasks. In essence, an embedding is a numerical representation of a more complex object, like text, images, audio, etc.

The embedding model will always produce embeddings of the same fixed size. You can then compute the similarity of complex objects by computing the similarity of the respective embeddings.

Click to expand Click to expand Click to see a table with all values from the next 2 Figures