I Nearly Bought a Capability I Already Owned

By mid-morning I had a shortlist and every intention of spending money on it: a dedicated coding model that ran quicker than mine, and a much larger general-purpose model everyone was rating highly.

I bought neither. The capability I was shopping for was already on the machine in my Dubai office, switched off since breakfast.

The model I run every day scores 73.4% on a standard coding benchmark, at 62.9 tokens per second. The specialist I was about to swap it for scores around 50 to 52% on that same test. I would have paid for a download, a migration and a stretch of re-tuning in order to lose roughly 22 points.

A product category is not a capability claim

The word "coder" in a model's name describes what its trainers were aiming at, not where it stands against what you already own.

Two things were true about it, and I only found them because I pushed back on my own shortlist. It is built for a different job — inline autocomplete, predicting the code that belongs between the text above and below your cursor — rather than the multi-step, tool-using work my daily assistant does. The two styles are scored on entirely different benchmarks.

It is also a full model generation older than the generalist I already ran. That is not a scandal, it is the rhythm of this market: this year's generalist routinely outgrows last year's specialist.

The lesson for businesses: when a vendor sells you something specialised, ask which generation it belongs to before you ask what it specialises in.

Eight times the parameters bought between 0.3 and 5.6 points

The other candidate was a well-regarded external model with roughly eight times the parameters of mine. Its own technical report put it at 73.7%. An independent tracker put it at 79.0%. My incumbent sits at 73.4%.

So the honest range for that upgrade is 0.3 to 5.6 points, depending on whether you believe the vendor's report or somebody else's measurement. Eight times the parameters, and the memory and hardware underneath to match, for a gain I could not separate from the noise between two sources.

A third option, a larger dense model from the same family as mine, is a real gain: 77.2%, up 3.8 points. It also runs two to four times slower. For an assistant you sit and wait for, that is not close.

For a business, the discipline is to write the quality gain and the running cost in the same sentence, from the same source. A gain that changes with who measured it is not yet a gain.

The upgrade I needed was a switch I had flipped that morning

Here is the part that made the morning embarrassing, and then useful.

That 73.4% is a reasoning-enabled score — the model works through the problem before answering. Reasoning had been switched off on both of my servers all day. I turned it off myself, first thing, for an unrelated reason: after moving the server onto a different runtime, raw unformatted reasoning text was leaking into the answers. That was the right call at nine in the morning. The migration causing it was finished before lunch. The setting was never revisited.

So for most of a day I compared candidates against a benchmark number my own machine was not producing. The incumbent was competing with a hand behind its back and losing narrowly — precisely the condition under which switching looks justified.

The fix cost nothing to acquire: no download, no swap, no migration. It cost latency, because reasoning takes longer per answer — a real price, but the kind you evaluate in an afternoon rather than a quarter, and reverse in a second if you are wrong.

The same shape turned up in a system that has nothing to do with code

Twice now. The second one was voice.

A newly announced cloud transcription product reacted noticeably faster than my own setup. The number that mattered was turn-taking — how long a system waits before deciding you have finished speaking. Theirs landed around 0.16 seconds, because it predicts the end of a turn with a purpose-trained model instead of waiting for silence.

Mine put live words on screen in about half a second, then sat through a fixed 0.7 seconds of silence before running the final, authoritative pass — roughly one second per exchange. On a phone call, that gap is the difference between a system that feels like a person and one that feels like a queue.

I started pricing the migration. Then I checked what I already had. The speech model I run locally — Whisper — already emits its own running estimate, several times a second, of whether the speaker has finished. My server received that signal on every chunk of audio and threw it away.

Wiring it up, with a small buffer so a trailing word is not clipped, should bring that 0.7-second wait down to roughly 0.1 to 0.2 seconds, with the transcription model and its accuracy untouched. Should — I have not built it yet. It is tracked work, not a result, and I will report the measured number when there is one.

Audit your off switches before you audit the market

Every disabled setting in your organisation was disabled for a reason, on a specific day, by someone who understood it. Almost none have an expiry date written next to them. The reason goes away — the bug is fixed, the vendor ships the patch, the platform is migrated — and the switch stays down, because switches do not send reminders.

So, an instruction rather than an observation. Before your next evaluation of a new model, vendor or platform, spend the first hour on the system you already pay for. List what is switched off and whether each reason still holds. Then list the signals your stack already computes and nobody consumes.

The bottom line

The cheapest capability you can add is usually one you have already paid for and left switched off. On both of these days the market was not my problem; my own configuration was. I did not look there first because reading your own settings feels like admin, while evaluating a vendor feels like progress.

It is the other way round.

For the engineers

The coding comparison was never like-for-like. Fill-in-the-middle models predict a span given a prefix and a suffix and are scored on infilling suites; agentic coding models are scored on issue-resolution suites with tool calls and multi-turn state. The specialist sat at roughly 50–52% on the agentic benchmark against my incumbent's 73.4% at 62.9 tok/s, and it was a full generation older — which mattered more than the specialisation. That 73.4% is a reasoning-enabled figure. Reasoning had been disabled on both servers because the previous serving runtime emitted unwrapped reasoning tokens into the visible response body; after the migration the wrapping was correct and the flag was never flipped back. The alternatives: the dense sibling scored 77.2% at two to four times the latency, and the 8x-parameter external candidate scored 73.7% by its own technical report against 79.0% on an independent tracker.

The voice pipeline: partial hypotheses arrive at about 0.5 s, then a fixed 700 ms silence timeout gates the final pass — roughly 1.0 s per turn. The speech model already emits a per-chunk end-of-turn probability, several updates per second, which the server discarded. Smoothing it and using it in place of the fixed timeout — a pattern an existing open-source project already demonstrates — projects to 100–200 ms with the transcription model unchanged. Projected, not measured: filed, not shipped.

The arithmetic-first version of this shopping trip — where I priced a "better" model on my own hardware before downloading it — is I Rejected the Better Model With Arithmetic. And for the phone-call side of the voice story, I Built an AI Voice Agent That Makes Real Phone Calls is the system whose turn-taking I was trying to speed up. If you're the one deciding what to buy at all, start with Buying AI: A Field Guide for Non-Technical Leaders — then audit what you own.

If your team is midway through evaluating a new model, a new vendor or a new platform, and nobody has audited what is switched off in the system you already run, that is a consulting day worth booking before the vendor call: I will go through the disabled settings and unused signals in your existing stack first, put a number on what turning them back on is worth, then help you price the thing you were about to buy. Book a consulting day or send me an inquiry telling me what you are evaluating, and I'll ask what you already own.