Welcome Llama 3 - Meta's new open LLM

Introduction

Meta’s Llama 3, the next iteration of the open-access Llama family, is now released and available at Hugging Face. It's great to see Meta continuing its commitment to open AI, and we’re excited to fully support the launch with comprehensive integration in the Hugging Face ecosystem.

Llama 3 comes in two sizes: 8B for efficient deployment and development on consumer-size GPU, and 70B for large-scale AI native applications. Both come in base and instruction-tuned variants. In addition to the 4 models, a new version of Llama Guard was fine-tuned on Llama 3 8B and is released as Llama Guard 2 (safety fine-tune).

We’ve collaborated with Meta to ensure the best integration into the Hugging Face ecosystem. You can find all 5 open-access models (2 base models, 2 fine-tuned & Llama Guard) on the Hub. Among the features and integrations being released, we have:

• Models on the Hub, with their model cards and licenses
• 🤗 Transformers integration
• Hugging Chat integration for Meta Llama 3 70b
• Inference Integration into Inference Endpoints, Google Cloud & Amazon SageMaker
• An example of fine-tuning Llama 3 8B on a single GPU with 🤗 TRL

Table of contents

• What’s new with Llama 3?
• Llama 3 evaluation
• How to prompt Llama 3
• Demo
• Using 🤗 Transformers
• Inference Integrations
• Fine-tuning with 🤗 TRL
• Additional Resources
• Acknowledgments

What’s new with Llama 3?

The Llama 3 release introduces 4 new open LLM models by Meta based on the Llama 2 architecture. They come in two sizes: 8B and 70B parameters, each with base (pre-trained) and instruct-tuned versions. All the variants can be run on various types of consumer hardware and have a context length of 8K tokens.

• Meta-Llama-3-8b: Base 8B model
• Meta-Llama-3-8b-instruct: Instruct fine-tuned version of the base 8b model
• Meta-Llama-3-70b: Base 70B model
• Meta-Llama-3-70b-instruct: Instruct fine-tuned version of the base 70b model

Llama 3 evaluation

How to prompt Llama 3

Demo

Using 🤗 Transformers

pip install --upgrade transformers