Every Model I Tested Failed at the Calendar

I spent a full day running five language models through the same set of tasks on one machine in my Dubai office. Same prompts, same order, two trials each. I was choosing a default assistant, and I had one hard requirement: it has to be right about time.

Two of the tasks were time questions. Convert a meeting slot between two cities. Work out the calendar date of "next Tuesday". Neither is a trick. Both are what an assistant gets asked ten times a day by anyone who books meetings.

Four of the five got at least one of them wrong. Not vaguely wrong — one city landing a full hour off, a weekday resolving to the wrong date, a daylight-saving offset applied the wrong way. The fifth got both right, which sounds like a result until you notice it is a sample of two.

One clean sheet out of five, on the two easiest questions I had

Here is the report card, ranked as I would have ranked them on everything else.

The model I already run in production got the timezone conversion right and "next Tuesday" right. Clean sheet.

A newer dense model — the one with the strongest case for replacing it — got both right with its extended reasoning mode switched on, and got the timezone conversion wrong with that mode switched off.

A small, fast model I keep around for low-stakes triage got the timezone question half right: correct for one city, an hour off for the other.

A larger rival I was evaluating failed both tasks, in both trials. That is not a bad run. That is a defect.

And a model I had retired the day before had, on its way out, put a daylight-saving offset in the wrong place.

One clean sheet out of five, on the two easiest time questions I could think of. These are not weak models.

A wrong date arrives with the same confidence as a right one

When a model invents a restaurant, you find out when you call to book. When it blows a stated budget, the number is sitting in front of you. Those failures announce themselves.

A wrong date does not. The model says Tuesday the 8th in precisely the tone it would use for Tuesday the 15th. No hedge, no flag, no visible drop in confidence. The error is silent, it propagates into a calendar invite, and it stays invisible until somebody joins an empty call or a deadline passes unnoticed. It is the same family of failure as a vision model that confidently captions a photograph it cannot see — The Model That Couldn't Say "I Found Nothing" — fluency with no error term.

The lesson for businesses: rank your AI's failure modes by how long they stay hidden, not by how bad they look in a demo. Date and timezone errors are the worst pairing on that scale: trivially easy to make, expensive to catch.

Correctness that depends on a setting is not correctness

The newer dense model is the most common way this gets missed.

It answered both time questions correctly with its extended reasoning mode on, and got the timezone conversion wrong with that mode off. Same model, same question, same day — the difference was a runtime setting.

Extended reasoning costs latency and money. It is exactly the knob somebody turns off six months into production, when replies feel slow or the bill gets attention. Nobody in that meeting knows they are also switching off the model's ability to add hours correctly, because nobody wrote that dependency down.

If a capability only holds on the expensive setting, you do not have the capability. You have a setting somebody will eventually change for reasons that have nothing to do with dates.

The same models were flawless at using tools

Here is the finding that decided what I did next.

In the same round I ran the tool-use benchmark I had built for an earlier comparison: does each model call the right function, with the right arguments, in the right order, when a task needs an action taken. Every model scored 48 out of 48. No malformed calls, no invented functions, no missing arguments. The design of that test — grading the state of the world rather than the transcript — is in Thirty Tasks, Three Models, One Decision; here the result matters purely as a contrast.

So these models are not sloppy. They are excellent at the part that sounds harder — choosing the right capability and handing it the right inputs — and unreliable at the part that sounds trivial: counting days across a month boundary.

That gap tells you where the fix goes. Not a smarter model, and not a firmer prompt asking it to be careful — both leave the arithmetic exactly where it was already failing. The fix is to move one narrow class of computation out of the model's reasoning and into code that cannot be creative about it.

Put the calendar in a function and make the model call it

So I stopped shopping for a model and wrote a specification instead.

One small date-and-time function, built on the operating system's own timezone database rather than on anything a model remembers. One rule in the system prompt: any question touching a date, a time or a timezone gets answered by calling that function. The model still decides what to ask it. The model does not do the arithmetic.

Then the part I would push hardest on with any client. The two failures from testing went into the brief as required test cases — not war stories in a background section, but acceptance criteria. The city that came out an hour off. The "next Tuesday" that landed on the wrong date. If the tool cannot get those two right, it is not finished.

I wrote that brief and handed it to the team implementing the assistant rather than building it myself — the honest limit here. I have specified the fix and pinned the failures it must survive; I am reporting a diagnosis, not a year of production evidence.

The bottom line

Language models are unreliable at exact arithmetic and calendar logic no matter how good they are at conversation, and the newer, larger, more expensive one does not fix it on its own. If your AI touches scheduling, deadlines or dates, require it to call a calculator it did not write.

I hate this class of error more than any other, and not because it is hard. Because it is boring, silent, and lands in somebody's calendar.

For the engineers

Every failure traced to the same root cause: calendar arithmetic performed inside free-form reasoning tokens. Memorised daylight-saving offsets, weekday counting, month rollover. There is no error term on that path: a wrong date is generated as fluently as a right one, and nothing downstream can tell them apart.

The contrast in the same round is the argument. All models scored 48/48 on tool-use discipline: right function, right arguments, right sequencing, zero malformed or hallucinated calls. Dispatch is solid; computation is not. So the fix is not model capability — it is removing one class of computation from the reasoning path.

The implementation: a single deterministic date-and-time function backed by the operating system's timezone database, not a table of offsets in a prompt. It resolves current time in a named zone, converts between zones, resolves relative expressions ("next Tuesday", "in three weeks"), and does arithmetic across month and daylight-saving boundaries. The model's job collapses to parsing intent into arguments and reading the result back; a system-prompt rule makes the call mandatory for anything date-sensitive.

Regression tests are the two observed failures verbatim: the conversion an hour off for one city, and the weekday that resolved to the wrong date. Worth pinning: one candidate passed only with extended reasoning on, and failed with it off. Treat correctness that depends on a runtime performance flag as a defect, not a configuration.

If your assistant, scheduling bot or intake system touches dates, timezones or deadlines — and nobody has ever checked whether it computes them or looks them up — that is a well-spent consulting day, whether your operation is in Dubai, across the UAE, or entirely remote. I will run the time questions your business actually gets asked, show you where the model is quietly guessing, and leave you with the deterministic tool and the test cases it has to pass. Book a consulting day or send me an inquiry with two date questions your business gets asked, and I'll run them.