AI Weekly Issue #535: AI got too expensive, so companies are moving to cheaper models

The AI bill became a management problem

Harvey's margins went negative, so it switched models

Harvey sells AI tools to law firms. It built them on models from OpenAI and Anthropic, which charge for every piece of text the model reads and writes. These pieces are called tokens.

Bloomberg reported that Harvey's gross margins fell from about 50% to minus 50% by June as its token usage grew 20 times. In August Harvey launched its own model, built on Moonshot's open-weight Kimi K3, and its margins turned positive again. Bloomberg named Abridge and Ramp among the other startups moving the same way.

An open-weight model is one a company can download and run on its own servers. It pays for computing power instead of paying another company for every token.

Open models now carry a large share of AT&T's AI work

The Financial Times reported that open models now handle 40% of AT&T's AI workloads.

Proprietary models from the large labs still take most of the money companies spend on AI. The change is that big buyers now treat the model as something they can swap when the price is wrong.

Anthropic ends discounts when customers hit their limit

Pricing is moving on the seller's side too. The Information reported that Anthropic ends its enterprise discounts, usually around 15%, once a customer uses up its contracted volume. The customer then has to renegotiate or pay list price.

For anyone running AI at work, the practical lesson is to measure cost per completed task. Teams that know that number can negotiate, or move work to a cheaper model when quality allows.

Where AI is now in daily use

McDonald's sets prices with an algorithm

Reuters reported that McDonald's uses machine learning to suggest an "optimal price" for menu items at nearly 14,000 US restaurants. The system analyzes millions of daily transactions.