Open-Source Text Generation & LLM Ecosystem at Hugging Face

[Updated on July 24, 2023: Added Llama 2.]

Text generation and conversational technologies have been around for ages. Earlier challenges in working with these technologies were controlling both the coherence and diversity of the text through inference parameters and discriminative biases. More coherent outputs were less creative and closer to the original training data and sounded less human. Recent developments overcame these challenges, and user-friendly UIs enabled everyone to try these models out. Services like ChatGPT have recently put the spotlight on powerful models like GPT-4 and caused an explosion of open-source alternatives like Llama to go mainstream. We think these technologies will be around for a long time and become more and more integrated into everyday products.

This post is divided into the following sections:

1. Brief background on text generation
2. Licensing
3. Tools in the Hugging Face Ecosystem for LLM Serving
4. Parameter Efficient Fine Tuning (PEFT)

Brief Background on Text Generation

Text generation models are essentially trained with the objective of completing an incomplete text or generating text from scratch as a response to a given instruction or question. Models that complete incomplete text are called Causal Language Models, and famous examples are GPT-3 by OpenAI and Llama by Meta AI.

One concept you need to know before we move on is fine-tuning. This is the process of taking a very large model and transferring the knowledge contained in this base model to another use case, which we call a downstream task. These tasks can come in the form of instructions. As the model size grows, it can generalize better to instructions that do not exist in the pre-training data, but were learned during fine-tuning.

Models created with love by Hugging Face with BigScience and BigCode 💗

Snippets to use all models mentioned in this blog post are given in either the model repository or the documentation page of that model type in Hugging Face.

Licensing

Further Resources