Welcome Gemma 2 - Google’s new open LLM

Google released Gemma 2, the latest addition to its family of state-of-the-art open LLMs, and we are excited to collaborate with Google to ensure the best integration in the Hugging Face ecosystem. You can find the 4 open-weight models (2 base models & 2 fine-tuned ones) on the Hub. Among the features and integrations being released, we have:

• Models on the Hub
• Hugging Face Transformers integration
• Integration with Google Cloud & Inference Endpoints

Table of contents

• What is Gemma 2?
• Technical advances in Gemma 2
  • Sliding window attention
  • Soft-capping and attention implementations
  • Knowledge Distillation
  • Model Merging
• Gemma 2 evaluation
  • Technical Report results
  • Open LLM Leaderboard results
• How to prompt Gemma 2
• Demo
• Using Hugging Face Transformers
• Integration with Google Cloud
• Integration with Inference Endpoints
• Fine-tuning with 🤗 TRL
• Additional Resources
• Acknowledgments

What is Gemma 2?

Gemma 2 is Google's latest iteration of open LLMs. It comes in two sizes, 9 billion and 27 billion parameters with base (pre-trained) and instruction-tuned versions. Gemma is based on Google Deepmind Gemini and has a context length of 8K tokens:

• gemma-2-9b: Base 9B model.
• gemma-2-9b-it: Instruction fine-tuned version of the base 9B model.
• gemma-2-27b: Base 27B model.
• gemma-2-27b-it: Instruction fine-tuned version of the base 27B model.

The Gemma 2 models were trained on ~2x more data than their first iteration, totaling 13 trillion tokens for the 27B version and 8 trillion tokens for the 9B version of web data (primarily English), code, and math. We don’t know the exact details of the training mix, and we can only guess that bigger and more careful data curation was a big factor in the improved performance.

Gemma 2 comes with the same license as the first iteration, which is a permissive license that allows redistribution, fine-tuning, commercial use, and derivative works.