Llama 2 is here - get it on Hugging Face

Introduction

Llama 2 is a family of state-of-the-art open-access large language models released by Meta today, and we’re excited to fully support the launch with comprehensive integration in Hugging Face. Llama 2 is being released with a very permissive community license and is available for commercial use. The code, pretrained models, and fine-tuned models are all being released today 🔥

We’ve collaborated with Meta to ensure smooth integration into the Hugging Face ecosystem. You can find the 12 open-access models (3 base models & 3 fine-tuned ones with the original Meta checkpoints, plus their corresponding transformers models) on the Hub. Among the features and integrations being released, we have:

• Models on the Hub with their model cards and license.
• Transformers integration
• Examples to fine-tune the small variants of the model with a single GPU
• Integration with Text Generation Inference for fast and efficient production-ready inference
• Integration with Inference Endpoints

Table of Contents

• Why Llama 2?
• Demo
• Inference
  • With Transformers
  • With Inference Endpoints
• Fine-tuning with PEFT
• How to Prompt Llama 2
• Additional Resources
• Conclusion

Why Llama 2?

The Llama 2 release introduces a family of pretrained and fine-tuned LLMs, ranging in scale from 7B to 70B parameters (7B, 13B, 70B). The pretrained models come with significant improvements over the Llama 1 models, including being trained on 40% more tokens, having a much longer context length (4k tokens 🤯), and using grouped-query attention for fast inference of the 70B model🔥!

However, the most exciting part of this release is the fine-tuned models (Llama 2-Chat), which have been optimized for dialogue applications using Reinforcement Learning from Human Feedback (RLHF). Across a wide range of helpfulness and safety benchmarks, the Llama 2-Chat models perform better than most open models and achieve comparable performance to ChatGPT according to human evaluations. You can read the paper here.