Inference for PROs

Today, we're introducing Inference for PRO users - a community offering that gives you access to APIs of curated endpoints for some of the most exciting models available, as well as improved rate limits for the usage of free Inference API. Use the following page to subscribe to PRO.

Hugging Face PRO users now have access to exclusive API endpoints for a curated list of powerful models that benefit from ultra-fast inference powered by text-generation-inference. This is a benefit on top of the free inference API, which is available to all Hugging Face users to facilitate testing and prototyping on 200,000+ models. PRO users enjoy higher rate limits on these models, as well as exclusive access to some of the best models available today.

Contents

• Supported Models
• Getting started with Inference for PROs
• Applications
  • Chat with Llama 2 and Code Llama 34B
  • Chat with Code Llama 70B
  • Code infilling with Code Llama
  • Stable Diffusion XL
• Messages API
• Generation Parameters
  • Controlling Text Generation
  • Controlling Image Generation
  • Caching
  • Streaming
• Subscribe to PRO
• FAQ

Supported Models

In addition to thousands of public models available in the Hub, PRO users get free access and higher rate limits to the following state-of-the-art models:

Model	Size	Context Length	Use
Meta Llama 3 Instruct	8B, 70B	8k tokens	One of the best chat models
Mixtral 8x7B Instruct	45B MOE	32k tokens	Performance comparable to top proprietary models
Nous Hermes 2 Mixtral 8x7B DPO	45B MOE	32k tokens	Further trained over Mixtral 8x7B MoE
Zephyr 7B β	7B	4k tokens	One of the best chat models at the 7B weight
Llama 2 Chat	7B, 13B	4k tokens	One of the best conversational models
Mistral 7B Instruct v0.2	7B	4k tokens	One of the best chat models at the 7B weight
Code Llama Base	7B and 13B	4k tokens	Autocomplete and infill code
Code Llama Instruct	34B	16k tokens	Conversational code assistant
Stable Diffusion XL	3B UNet	-	Generate images
Bark	0.9B	-	Text to audio generation

pip install huggingface_hub