Three Agents, One Answer

I asked three AI agents to each pick a country at random and report its population. All three came back with Portugal.

I ran it again. Portugal, Portugal, Portugal. So I gave each agent a unique identity in its prompt — a different random ID per agent — and told them explicitly to randomize their choice. Portugal.

Then I stopped talking to the agents and started testing the model underneath them. Temperature is the dial that controls how adventurous a model's word choice is. I ran temperature 0.0, then 1.0, then 1.5, five times each. Identical answer, every single run. I pushed it to 5.0 — a setting so extreme the output should be word salad. It was word salad: the same 15 tokens of word salad, five times in a row. Seven different explicit random seeds — 1, 7, 42, 99, 999 and a few more — produced identical output too.

At that point it was no longer an agent problem. Randomness itself was broken.

You can pay for three opinions and receive one

The whole reason to run agents in parallel is disagreement. You fan a task out to three workers so you get three angles on a document, three candidate shortlists, three drafts — and then a human picks, or a majority wins. It is the same instinct as asking three people on your team rather than one.

If the machine underneath cannot actually vary, that fan-out is theatre. You pay three times, you wait for three runs, and you receive one answer wearing three name tags. Worse, you receive it with a confidence bonus attached, because three matching answers read like consensus. Agreement is the most persuasive-looking output a system can give you, and in this case it was the symptom of the fault.

The lesson for businesses: when the value of a system comes from running it more than once, matching results are not reassurance. They are the thing you have to prove.

The obvious fix would have charged rent on every request

There was a quick workaround available, and it was tempting. Inject a small random token into every prompt — a scrap of noise that differs per agent — and the outputs would be forced apart.

Here is why I turned it down. The machine in my office caches the work it has already done on the front of a prompt. Every agent in a fan-out shares a long identical preamble: the instructions, the tools, the context. Processed once, reused for the rest. That reuse is the difference between a fast answer and a slow one, and I had just finished building it.

Change one character near the front of a prompt and the reuse is gone. The random-token workaround would have bought variety by paying full processing price on every request, forever — a permanent tax, levied to hide a bug that was still there underneath.

The lesson for businesses: a workaround that lives in the hot path is not free, and it is never priced honestly at the moment you adopt it. Price it over a year of traffic, not over the afternoon it saves you.

A broken random generator and a perfect cache look exactly alike

This is the part that made it genuinely hard, and I will admit the fear plainly: I thought fixing randomness might mean giving up the caching. Same input, same output is precisely what a cache is supposed to guarantee. From the outside, a cache doing its job and a random number generator that has stopped generating produce the same evidence — outputs that never change.

They are two independent concerns. Caching is about not repeating work you have already done. Randomness is about the choice made at the very end. They live in different parts of the system, they fail for different reasons, and I had quietly collapsed them into one idea because they wore the same symptom.

The lesson for businesses: when two subsystems can produce an identical symptom, no amount of watching the output will tell you which one broke. You have to go and test the one that is supposed to vary.

Test that your system can disagree with itself

Nobody writes this test. We test that answers are correct, that they arrive quickly, that they come back in the right format. Almost nobody writes the test that says: run this twice and assert the two results are different.

So here is something to do this week. If your business runs anything that fans out — parallel review agents, multiple drafts, a committee of models voting on a decision — send the same input through three times and read the three outputs side by side. If they are word-for-word identical, you do not have three agents. You have one agent and two invoices. Most teams have never looked.

The root cause was one line, and it cost nothing

The fault was in the step where the model picks its next word — a performance optimization had frozen the internal state that makes each pick different from the last. Every draw after the very first one in the server's life reused the same frozen state. That is why temperature and seeds did nothing: they were all feeding a dice roll that had stopped rolling.

Removing one line fixed it. Three sequential requests with an identical prompt returned 317, 487 and 729. Three genuinely simultaneous requests returned 472, 347 and 372 — while still reusing 12 of 13 cached prompt tokens in one measurement and 18 of 19 in another. Variety and caching, both, at no cost to either. I committed the fix and sent it upstream to the project's maintainers.

The workaround would have cost something on every request for as long as the system ran. The root-cause fix cost one afternoon, once.

The bottom line

Randomness and caching are separate concerns that happen to fail identically, and I had assumed they were the same concern until the evidence forced me to separate them. The expensive part was not the repair — that was one line. It was that a healthy-looking system, returning clean answers at full speed, had been giving me one opinion and charging me for three.

For the engineers

The sampling function — the step that draws a token from the output distribution — was wrapped in a compilation decorator for speed. That wrapper is meant to thread the random number generator's state through each call so it advances. In the server's real request path it did not. The state advanced on the first call ever made in the process; every call afterwards silently reused the same frozen state. Same state in, same draw out, regardless of temperature and regardless of seed. It also explains the temperature 5.0 result: the distribution genuinely was flattened into nonsense, but the draw from it was stuck, so the same 15 garbled tokens came back five times.

Finding it meant leaving the agent layer entirely and hitting the model directly: temperature 0.0, 1.0 and 1.5 at five runs each, then 5.0, then seven explicit seeds (1, 7, 42, 99, 999 and others). Identical output across all of them puts the fault below the prompt layer and below the seed plumbing, which leaves the sampler.

The fix was deleting the decorator on that one function. Verification ran in both orders deliberately — sequential requests returned 317, 487, 729; concurrent requests returned 472, 347, 372, with prefix cache reuse intact at 12 of 13 and 18 of 19 tokens. The rejected alternative, a per-prompt nonce, would have restored variety by destroying prefix reuse on every request in the system.

That prefix cache is the same one whose fragility I documented in Two Hundred Seconds, One Shuffled List — one unstable field and the reuse collapses. If your fan-out feeds real decisions, Five Parallel Research Agents is what this looks like when the independence check passes and the parallelism actually pays.

If your company runs AI agents in parallel — for triage, for review, for a shortlist that someone senior signs off on — and nobody has verified that those agents are able to disagree, a consulting day settles it: I'll test whether your fan-out is producing genuinely independent work or one answer billed three times, and leave your team with the checks that catch it the next time it drifts. Book a consulting day or send me an inquiry first if you'd rather describe your fan-out before booking.