Few-Shot Text Classification with Scikit-LLM

Introduction

If you have worked with building text classification models, chances are you got familiar with the usual drill: collecting hundreds to thousands of labeled examples, cleaning the raw text, converting it to suitable numerical representations for machine learning models — like embeddings or TF-IDF — and training your model, such as a logistic regression or random forest classifier.

However, when large language models (LLMs) came along, they changed the rules of the game — and that includes traditional machine learning pipelines to some extent. Now it is possible to classify a text by simply asking an LLM to do so. Here's the catch: shifting from classical machine learning workflows to prompt engineering can be challenging. Scikit-LLM is a Python library that bridges the gap between both worlds. It wraps the powerful capabilities of pre-trained LLMs behind the familiar scikit-learn interface — namely fit() and predict().

This article shows how to use Scikit-LLM for one of the simplest yet most practical text classification tasks in the modern AI landscape: few-shot classification, that is, classifying text based on a small number of labeled examples that guide the LLM on what to do and how.

Before getting into our Python example, it's worth clarifying the difference between few-shot classification and another form of text classification that also gained popularity with the rise of LLMs: zero-shot classification.

Few-shot classification is typically better than its zero-shot counterpart for highly domain-specific text classification tasks where texts contain specific jargon, edge cases, or nuanced sentiments. In most cases, providing just 3 to 5 labeled examples may suffice to "teach" a pre-trained LLM the extra rules it needs to perform a good classification — all without retraining the whole model.

The good news: Scikit-LLM makes this process remarkably easy. Here's how it works.