Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps

This guide is a fully public, inexpensive recipe for making a small model substantially better at structured-output compliance. We fine-tune LFM2.5-350M with Group Relative Policy Optimization (GRPO) using the TRL library and evaluate it on the IFStruct benchmark. The full run takes around 500 samples and 100 training steps, small enough for a free-tier Colab or Kaggle GPU, and is available on GitHub. The results show that even a light fine-tuning procedure improves performance from 22.6% to 29.7% on the IFStruct benchmark.

Structured output is one of the most common real-world tasks for LLMs, yet most benchmarks fold it into broader reasoning or extraction scores rather than measuring it on its own. Whether a model reliably returns valid, parseable output in the requested format and shape — schema compliance — is often what decides whether it can be wired into a downstream system at all.

Note that the training pipeline described here is not the one used to train the RL model described in the IFStruct blog. This notebook doesn't aim to recreate the IFStruct benchmark score, but to show how task-specific fine-tuning of smaller models can improve performance and match that of far larger models.

Prerequisites

This guide has two halves that run in different places:

• Fine-tuning runs on a GPU. The accompanying notebook is sized for a free-tier Colab or Kaggle GPU.
• Evaluation can run locally on a MacBook (here, a MacBook Pro with an Apple M5 Max and 36 GB of unified memory) through llama.cpp, which exposes an OpenAI-compatible server that the IFStruct evaluator talks to.

We will need uv for the Python tooling and llama.cpp for serving. Following the Liquid AI llama.cpp deployment docs, install llama.cpp with Homebrew and verify that llama-server is available:

brew install llama.cpp
llama-server --version

IFStruct Evaluation on LFM2.5-350M (Base model)

git clone https://github.com/Liquid4All/ifstruct.git

Then we start the base-model server with the following command: