Getting Started with Hugging Face Inference Endpoints

Training machine learning models has become quite simple, especially with the rise of pre-trained models and transfer learning. OK, sometimes it's not that simple, but at least, training models will never break critical applications, and make customers unhappy about your quality of service. Deploying models, however... Yes, we've all been there.

Deploying models in production usually requires jumping through a series of hoops. Packaging your model in a container, provisioning the infrastructure, creating your prediction API, securing it, scaling it, monitoring it, and more. Let's face it: building all this plumbing takes valuable time away from doing actual machine learning work. Unfortunately, it can also go awfully wrong.

We strive to fix this problem with the newly launched Hugging Face Inference Endpoints. In the spirit of making machine learning ever simpler without compromising on state-of-the-art quality, we've built a service that lets you deploy machine learning models directly from the Hugging Face hub to managed infrastructure on your favorite cloud in just a few clicks. Simple, secure, and scalable: you can have it all.

Let me show you how this works!

Deploying a model on Inference Endpoints

Looking at the list of tasks that Inference Endpoints support, I decided to deploy a Swin image classification model that I recently fine-tuned with AutoTrain on the food101 dataset. If you're interested in how I built this model, this video will show you the whole process.

Starting from my model page, I click on Deploy and select Inference Endpoints.

This takes me directly to the endpoint creation page.

I decide to deploy the latest revision of my model on a single GPU instance, hosted on AWS in the eu-west-1 region. Optionally, I could set up autoscaling, and I could even deploy the model in a custom container.

Next, I need to decide who can access my endpoint. From least secure to most secure, the three options are: