A free coding agent, paid for by ads
Freebuff is a free AI coding agent. Developers use it in the terminal, in a desktop app or on the web to write and change code in their own projects. Its free plan runs top open-source models with no card or API key, and text ads cover the cost. Paid plans add more hours of use each day and more models.
Many Freebuff users also pay for Claude Code or Codex and switch to Freebuff when they hit those tools’ limits. Most are individual developers, and some companies sign up their employees.
Users pay nothing or a flat monthly price, never per call, so Freebuff pays for every token its agent reads and writes. The less each call costs, the more Freebuff’s margin grows. Victor Cheng, who leads inference at Freebuff, is responsible for that spend.
Freebuff had already cut its costs significantly
Before working with SuperPenguin, Freebuff had already run sprints dedicated to cutting its AI costs and reduced them significantly. Among other changes, it moved most of its traffic to a lower-cost provider for the same model.
After those optimizations, the obvious savings were gone. But Freebuff’s agent uses trillions of tokens a week, so even a few percentage points of savings adds up. Victor wanted to know if there were optimization opportunities left beyond the ones he had already implemented. Ones that would not hurt the user experience, or cut how much users can use Freebuff.
Pricing every call, not a sample
Freebuff added the SuperPenguin SDK to its agent, so SuperPenguin received a record of every call on DeepSeek v4 Flash, adding up to millions per day. SuperPenguin priced each one with Freebuff’s own rates for each provider, so the cost reflected who served the call and when, as well as the customized rate.
For context, coding agents send the whole conversation back into the model at every step. Providers keep recent conversations in a cache and charge much less to read them again, so most of Freebuff’s calls are cheap. A call that misses the cache pays full price to read the whole conversation, many times what a normal call costs. Following each conversation call by call showed when that happens and why. With every call priced, SuperPenguin could detect patterns explaining why some calls cost far more than others.
Where the savings came from
Most of the savings came from calls that missed the cache when they didn’t need to. Some calls moved to a new provider after a brief failure, and the new provider had to read the whole conversation from scratch. Others missed because of how the prompt was ordered: most of the text every user shares came after each user’s own files, so it couldn’t be reused. Changing how the agent retries and reordering its prompt keep these calls cheap. Users see no difference in quality.
Long conversations presented another source of savings. Freebuff’s agent compacts a conversation into a short summary once it exceeds a set length. A replay of millions of real calls showed that compacting sooner costs less overall, even after accounting for the extra overhead each compaction demands.
Together, these changes saved another 10% of Freebuff’s DeepSeek v4 Flash spend, on top of the efficiency gains that Freebuff had already driven themselves. Importantly, none of our optimizations changed the model or how much users could use Freebuff.
Keeping up as Freebuff changes
As an AI startup, Freebuff keeps changing, and so does the AI market. In just over a week of working together, one provider cut its prices and was awarded about half of Freebuff’s traffic, one new provider was onboarded, another was removed, and a new version of the app shortened the average conversation. More developers keep joining, and Freebuff keeps negotiating new prices with its providers.
Each of these changes shift how much it costs to serve each customer and which optimizations matter the most. SuperPenguin updates its recommendations within hours of such changes so that recommendations remain current and effective. Because SuperPenguin prices every call as it arrives, Freebuff gets optimization suggestions in near real time that match its current traffic and prices.
We were one of the harder companies to reduce costs for, because we’d already done so much optimization. But SuperPenguin identified some pretty good stuff.