Granite Switch 4.1: Chat, Decisions, and Speech from One Endpoint

Introduced last spring, the Granite Switch project enables a modular approach to model development. Now Granite Switch 4.1 models, armed with a Decision adapter and the fastest ASR model on the OpenASR leaderboard, can move efficiently between natural-language responses, general and specialized decisions, and speech-to-text tasks—all from one hosted endpoint.

Specifically, Granite-Switch-4.1-8B and -30B models are configured to include:

1. A new Decision aLoRA for general-purpose decision-making (e.g. v1/systemone),
2. A specialized Guardian aLoRA that, on average, outperforms Jev 1.13, Microsoft-Decision-1, GPT-Luna-Decisions, and Clef-Flash on safety-related decision tasks, and
3. Granite-Speech-5.0-Turbo-CTC, the fastest ASR model on the OpenASR Leaderboard, embedded for use in transcription tasks.

All three of these technologies (along with the underlying base model) are composed into one model, taking up approximately the same amount of VRAM on a GPU as the base model would by itself. That means more capabilities with fewer GPUs.

And because the Switch adapters were built as cache-efficient Activated LoRAs, the KV cache created from a base model generation can be reused for any subsequent adapter-based decisions. Combined with our team's recent vLLM contributions to speed up low rank adapter inferencing, as well as a co-located ASR encoder, Granite Switch 4.1 models can switch between a chat request and a decision request and a spoken request, with blazing fast speeds.

Check out our Speech-to-Decisions HF space where granite-switch-4.1-8b transcribes audio, generates summaries, screens for harmful content, and tests 15 other user-defined flags—all running on a single endpoint in real time.

Granite Libraries: The OG Decision Models

Compare granitelib adapters with generalist decision models when evaluated on specialized classification tasks.

Compose and Extend Your Own Granite Switch Model

Resources for Getting Started

Footnotes