Welcome Gemma - Google’s new open LLM

An update to the Gemma models was released two months after this post, see the latest versions in this collection.

Gemma, a new family of state-of-the-art open LLMs, was released today by Google! It's great to see Google reinforcing its commitment to open-source AI, and we’re excited to fully support the launch with comprehensive integration in Hugging Face.

Gemma comes in two sizes: 7B parameters, for efficient deployment and development on consumer-size GPU and TPU and 2B versions for CPU and on-device applications. Both come in base and instruction-tuned variants.

We’ve collaborated with Google to ensure the best integration into the Hugging Face ecosystem. You can find the 4 open-access models (2 base models & 2 fine-tuned ones) on the Hub. Among the features and integrations being released, we have:

• Models on the Hub, with their model cards and licenses
• 🤗 Transformers integration
• Integration with Google Cloud
• Integration with Inference Endpoints
• An example of fine-tuning Gemma on a single GPU with 🤗 TRL

Table of contents

• What is Gemma?
  • Prompt format
  • Exploring the Unknowns
• Demo
  • Using 🤗 Transformers
  • JAX Weights
• Integration with Google Cloud
• Integration with Inference Endpoints
• Fine-tuning with 🤗 TRL
• Additional Resources
• Acknowledgments

What is Gemma?

Gemma is a family of 4 new LLM models by Google based on Gemini. It comes in two sizes: 2B and 7B parameters, each with base (pretrained) and instruction-tuned versions. All the variants can be run on various types of consumer hardware, even without quantization, and have a context length of 8K tokens:

• gemma-7b: Base 7B model.
• gemma-7b-it: Instruction fine-tuned version of the base 7B model.
• gemma-2b: Base 2B model.
• gemma-2b-it: Instruction fine-tuned version of the base 2B model.

• gemma-1.1-7b-it
• gemma-1.1-2b-it

So, how good are the Gemma models? Here’s an overview of the base models and their performance compared to other open models on the LLM Leaderboard (higher scores are better):