Deploy MusicGen in no time with Inference Endpoints

MusicGen is a powerful music generation model that takes in text prompt and an optional melody to output music. This blog post will guide you through generating music with MusicGen using Inference Endpoints.

Inference Endpoints allow us to write custom inference functions called custom handlers. These are particularly useful when a model is not supported out-of-the-box by the transformers high-level abstraction pipeline.

transformers pipelines offer powerful abstractions to run inference with transformers-based models. Inference Endpoints leverage the pipeline API to easily deploy models with only a few clicks. However, Inference Endpoints can also be used to deploy models that don't have a pipeline, or even non-transformer models! This is achieved using a custom inference function that we call a custom handler.

Let's demonstrate this process using MusicGen as an example. To implement a custom handler function for MusicGen and deploy it, we will need to:

1. Duplicate the MusicGen repository we want to serve,
2. Write a custom handler in handler.py and any dependencies in requirements.txt and add them to the duplicated repository,
3. Create Inference Endpoint for that repository.

Or simply use the final result and deploy our custom MusicGen model repo, where we just followed the steps above :)

Let's go!

First, we will duplicate the facebook/musicgen-large repository to our own profile using repository duplicator.

Then, we will add handler.py and requirements.txt to the duplicated repository. First, let's take a look at how to run inference with MusicGen.

Let's hear what it sounds like:

Your browser does not support the audio element.

Optionally, you can also condition the output with an audio snippet i.e. generate a complimentary snippet which combines the text generated audio with an input audio.

Let's give it a listen:

Your browser does not support the audio element.

Alright! With the basic usage outlined above, let's deploy MusicGen for fun and profit!

Here's how it sounds like: