When To Build Your Own AI Model (And When To Stop Kidding Yourself)

Eilon Reshef, cofounder and CPO of Gong, is a seasoned entrepreneur, executive and investor in the internet and software spaces.

We were running our own models at Gong long before ChatGPT became a household name. This was the era of transformer-based language models, the state of the art for natural language understanding before LLMs. We’d take these prebuilt foundation models, train them further on our domain and data and then deploy them for very targeted jobs.​

We called it fine-tuning. Academics called it transfer learning. Most people just called it a lot of work.​

I tell you this because it’s important context for what comes next: the hype around post-trained small language models (SLMs) and their very real practical potential.​

What We’re Actually Talking About

First, an important distinction: This isn’t about training a foundation model from scratch. For most product companies, that would be economic malpractice. It’s about taking an existing model and making it much better at a specific job through supervised fine-tuning, preference optimization, reinforcement learning, distillation or some combination of them.​

Fine-tuning is one form of post-training. I’ll use the broader term when that distinction matters.​

Most major AI providers offer a lineup of models that sits roughly on a diagonal: cheap and fast at one end, more capable and expensive at the other. The names differ, but the trade-offs don’t.​

The promise of post-training is that it lets you move above this line for a specific task. You can get higher quality than an off-the-shelf model at the same inference cost or comparable task quality at a lower cost.​

That doesn’t mean the smaller model is smarter overall, but that specialization can make it better at the specific job you need it to do. The real question is when that advantage holds and what it costs.​

Killing The ‘Moat’ Story

The Real Decision Framework

The Ongoing Cost: Operations

The overlooked cost of SLMs is keeping them competitive.