Up to 3.2x Faster Inference with LFM2.5-DSpark

Today, we release DSpark draft model checkpoints for three models from our LFM2.5 family: LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B. These add a speculative decoding path that trades a minimal memory increase for a large decoding speedup without changing output quality:

• Faster inference: up to 3.18 throughput improvement on a GPU and up to 2.87x on-device.
• Toward on-device agentic inference: cuts function-calling latency by 57% on average for LFM2.5-2.6B
• Day-one support for llama.cpp and SGLang: LFM-compatible DSpark integration is open-sourced upstream

How does DSpark work

The decode phase in LLM inference is traditionally memory-bound. Most latency comes from streaming weights from DRAM into SRAM, not from intense computation. Speculative decoding addresses this by using a lightweight draft model to produce candidate tokens, then having the target model verify them all in a single forward pass, sharing the cost of loading the weights across all tokens we verify.

Over the years, multiple approaches of speculation have been proposed, with the most prominent being EAGLE-3, DFlash, and, most recently, DSpark, which combines three components:

• DFlash-style parallel backbone conditioned on the target model’s context features, producing hidden states for all draft tokens in a single forward pass.
• A lightweight sequential head, modeled as a Markov chain between neighboring tokens, that adds inter-token dependency, raising the acceptance rate at later positions.
• A confidence-scheduled verifier that predicts each token’s survival probability and prunes low-confidence suffixes when verification would cost more than it saves.

Training and Architecture

The resulting draft models are relatively small, with each around ~300M parameters.

Quality parity

Inference Speed Up on CPU and GPU

All three drafter models deliver noticeable throughput improvements on both the large-scale accelerator (H100) and the edge deployment (M4 Max MacBook).