Learn the Hugging Face Kernel Hub in 5 Minutes

Boost your model performance with pre-optimized kernels, easily loaded from the Hub.

Today, we'll explore an exciting development from Hugging Face: the Kernel Hub! As ML practitioners, we know that maximizing performance often involves diving deep into optimized code, custom CUDA kernels, or complex build systems. The Kernel Hub simplifies this process dramatically!

Below is a short example of how to use a kernel in your code.

import torch

from kernels import get_kernel

# Download optimized kernels from the Hugging Face hub
activation = get_kernel("kernels-community/activation")

# Random tensor
x = torch.randn((10, 10), dtype=torch.float16, device="cuda")

# Run the kernel
y = torch.empty_like(x)
activation.gelu_fast(y, x)

print(y)

In the next sections we'll cover the following topics:

1. What is the Kernel Hub? - Understanding the core concept.
2. How to use the Kernel Hub - A quick code example.
3. Adding a Kernel to a Simple Model - A practical integration using RMSNorm.
4. Reviewing Performance Impact - Benchmarking the RMSNorm difference.
5. Real world use cases - Examples of how the kernels library is being used in other projects.

We'll introduce these concepts quickly – the core idea can be grasped in about 5 minutes (though experimenting and benchmarking might take a bit longer!).

1. What is the Kernel Hub?

The Kernel Hub (👈 Check it out!) allows Python libraries and applications to load optimized compute kernels directly from the Hugging Face Hub. Think of it like the Model Hub, but for low-level, high-performance code snippets (kernels) that accelerate specific operations, often on GPUs.

Instead of manually managing complex dependencies, wrestling with compilation flags, or building libraries like Triton or CUTLASS from source, you can use the kernels library to instantly fetch and run pre-compiled, optimized kernels.

For example, to enable FlashAttention you need just one line—no builds, no flags:

By contrast, compiling FlashAttention yourself requires: