Train and Fine-Tune Sentence Transformers Models

This guide is outdated. It covers the SentenceTransformer.fit API from Sentence Transformers versions before v3.0, which has since been superseded. The current training API uses the SentenceTransformerTrainer and is covered in these up-to-date guides:

• Embedding models: Training and Finetuning Embedding Models with Sentence Transformers
• Reranker (Cross Encoder) models: Training and Finetuning Reranker Models with Sentence Transformers
• Sparse embedding models: Training and Finetuning Sparse Embedding Models with Sentence Transformers
• Multimodal models: Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers

Please read one of the guides above instead. The content below is kept only for historical reference.

Check out this tutorial with the Notebook Companion:

Training or fine-tuning a Sentence Transformers model highly depends on the available data and the target task. The key is twofold:

1. Understand how to input data into the model and prepare your dataset accordingly.
2. Know the different loss functions and how they relate to the dataset.

In this tutorial, you will:

1. Understand how Sentence Transformers models work by creating one from "scratch" or fine-tuning one from the Hugging Face Hub.
2. Learn the different formats your dataset could have.
3. Review the different loss functions you can choose based on your dataset format.
4. Train or fine-tune your model.
5. Share your model to the Hugging Face Hub.
6. Learn when Sentence Transformers models may not be the best choice.

How Sentence Transformers models work

In a Sentence Transformer model, you map a variable-length text (or image pixels) to a fixed-size embedding representing that input's meaning. To get started with embeddings, check out our previous tutorial. This post focuses on text.

This is how the Sentence Transformers models work:

This figure summarizes the process:

Now for the most critical part: the dataset format.

How to prepare your dataset for training a Sentence Transformers model

Output: