Posts

A quarter of my assistant's spoken replies were stalling mid-sentence. The 4-bit weights I downloaded to fix it measured RTF 1.23 against 1.10 for the 8-bit model. The real cost was a codec re-decode that a docstring — mine — had recorded as 9% on a different machine…

Sixty-six architecture decisions, one constitution, and a corrections log. Most of what I wrote in August is still there; some of it is there because it was tested by a failure and held. These are the operating rules that survived contact with real days…

I wanted to start a meeting recording from my wrist and talk to my assistant from the same app. The Swift was the easy part. The audio engine tap that returns silence, the WebSocket that watchOS refuses on every interface, and the one background session per app that silently kills the other — those took the day…

An 86-minute meeting came back with one person talking for 73 minutes. The voice prints were fine — ten random slices identified both speakers cleanly. The failure was one step earlier, in the segmentation, and the obvious fix (retrain the voices) would have done nothing…

A one-click 'teach the diarizer this voice' button, built in an afternoon, poisoned an enrollment with 129 seconds of somebody else within a day. The restore was easy. The gate I had to build afterwards is the interesting part: refusing to learn from audio that sounds like someone you already know…

I moved a GPU workload off the machine that runs my main model and onto the smaller one next to it, with the big one as a fallback. Five failure scenarios tested with 20-second clips, a RAM ceiling, and one bill that arrived three days later in a different service…

GLM-5.3-Flash keeps a fixed-size recurrent state in 34 of its 45 layers, so its prompt cache can't be trimmed like a normal KV cache. My server stored two full snapshots per request, evicting 155 of them an hour, and started every new session cold…

GLM-5.3-Flash reads 12 GB of weights per token on my M3 Ultra, which is 14.6 ms at full bandwidth, yet a decode step takes 34 ms. No normal tool could find where the rest went, so I measured each piece by repeating it on sibling layers…

Compiling one decode step of GLM-5.3-Flash's linear-attention layers bought my assistant 5% more tokens per second. Then a bit-exact Metal kernel from an abandoned pull request took it to 16% total — after two rounding bugs, one of them in MLX itself…

97% of GLM-5.3-Flash's parameters sit in experts that mostly stay idle — so I quantised them harder than everything else, and got a build that is faster than mixed-4/8 at the same size…

And here's the kicker: **this very website — the design, the code, the Stripe booking system, the blog you're reading — was built by an open-weights model, GLM 5.3 Flash, running on an M3 Ultra wit…

A trivial request to remove a VPN client turned into a full audit of what was actually running and listening on machines I run. Three genuinely serious exposures, none of them installed on purpose…

Most AI projects I see fail for a boring reason: they need someone to keep them alive. Someone has to remember to run the script, check the dashboard, notice the silent failure. The moment the enth…

The fast text reader beat the smart one by 245x. That was not the finding. The finding was what the smart one did with an image that had no text in it at all…

A process exited 0 after the model returned nothing at all. That green light cost me a day. Here is the full catalogue of checks that lied to me, and the five questions I now ask before I trust any signal…

A transcription model that benchmarks 2.3x faster saved me a seventh of a second on the audio I actually send it — and charged for that in the accuracy that does the job…

The model at the top of the public OCR leaderboard read an English page almost perfectly and made 28 errors on an Arabic one — the worst of the four I tested. Here is what I run instead…

I measured a change that made a long job run 2.07 times faster, proved it numerically identical, and left it switched off — along with a second one that was faster but not exact…

My local model answered every question correctly and had quietly become seconds slower to start — because the cache built to make it fast had grown too large to search…

A benchmark I called harmless froze an entire machine for 94 seconds, and a leftover file took every client down for 20 minutes — here is the honest answer to the question every client asks afterwards…

Three databases corrupted twice in one week on a system holding 580,000 messages, and I lost nothing. Not luck — one architecture decision I had to defend five times before it paid off…

I built a working email client in a day with AI assistance. An independent audit found four critical, exploitable holes. The productivity story is real — and that is exactly why the security story got worse…

Five models, one day of side-by-side testing, and a date or timezone error in nearly every one of them — the fix is not a better model, it is a function…

A speed feature that quietly did nothing, and a new model that crashed the plumbing built for the old one — two ways good components fail only in combination…

An AI assistant broke its own written rule twice in one session, then honestly admitted it was laziness. The fix was never a better prompt — it was making the violation structurally impossible…

An eleven-minute job that quietly ran three times, and a parallel instruction that vanished between two pieces of software — why automatic retries are only safe when the work is cheap or resumable…

A one-day head-to-head on my own real work retired the model I had been running — and the follow-up test that changed nothing was worth exactly as much…

A speed feature that benchmarked as completely broken turned out to be three unrelated bugs stacked on top of each other — and one of them was quietly taxing every server I run…

I asked a machine for 600 GB of model memory when it had 550 GB to give, and it did not degrade gracefully — it went silent and needed a power button…

I built an AI voice agent that places real phone calls on my behalf. It has held natural business conversations, handled voicemail detection, and — importantly — it introduces itself as an AI every…

I wrote the rule that AI agents should never hold your credentials. Then I pasted a plaintext password into a chat channel out of pure frustration — and watched it fan out into three files and one message I could never recall…

Spot-checks said the pricier model was smarter. 132 blind runs said the gap was noise. What the cheap one actually broke, no aggregate score would ever show you…

A business partner's AI assistant messaged my system asking about my infrastructure — and the traditional signals of trust are exactly what an automated probe reproduces perfectly…

A block of live project status, injected into the part of every request that was supposed to be cached, quietly bought freshness nobody had asked for with money nobody had budgeted…

Adding image support to my local model meant replacing the engine underneath it — and the real question was whether months of work that had nothing to do with images would survive…

I wrote the operating rule that morning and violated it the same afternoon — and the forced reboot that followed exposed two failures nobody knew were there…

Twice I went shopping for a capability my own stack already had — once a specialist model, once a cloud voice product. Both times the answer was a setting somebody had switched off…

One service ran out of capacity it was never actually using, and another kept stuttering because of a machine eight hours into a broken retry loop — neither fault was where I was looking…

An exploratory embedding job on a shared server pushed response times from 0.14 seconds to over 3.5 seconds — every experiment needs an abort guard before it needs a dashboard…

Picking the model that would index my documents taught me to sort infrastructure decisions by what it costs to undo them, not by this month's benchmark score…

The slot built for small, throwaway AI tasks was pointed at the biggest model I own — and nothing in the system ever complained, because every call returned a correct answer…

A minor internal service I stood up in an afternoon froze a 512 GB machine solid — because I had written safety limits into everything I bought and none into the thing I built…

A quick question I ask every company I advise: *if your AI agent went rogue at 2am, what could it actually do?*

I asked for a feature built the way the last one was built. I got six questions back instead — and one of them only existed because two agents read the code first…

The most common way AI-assisted delivery produces fictional progress is the gap between a committed change and a change that is actually live — and here is the checklist I use to close it…

A day of real outbound calls to venues abroad produced six distinct failure modes, three abandoned venues, and one clear rule about retries…

Three screenshots in a rebuilt runbook were weeks out of date. My agent hadn't hidden them — it had labeled them, honestly. That label is why they passed review…

I shipped a working fallback chain that cut a two-minute hang to nineteen seconds, then reverted it the same day — because every address a system will automatically try is an address someone else can point it at…

A pipeline of mine once spent $370 in 48 hours doing a job a shell script could have done for free. Here's the post-mortem, because the same pattern is quietly burning budgets in companies all over…

I let an agent drive live third-party web systems inside real production accounts for days. It could do everything except click one button…

Unattended automation almost never fails loudly — it fails silently, and the only reliable defence is a positive signal plus an alarm for the thing that never happened…

A scoping call meant to take twenty minutes took most of a day — and that mapping work, not the build, is where AI projects actually succeed or fail…

I pointed a coding agent at a full clone of a paid task manager and went to bed. Seven and three-quarter hours later it was working. The bottleneck was never the code…

Everyone building with AI eventually hits the same wall: the demo works, and the system doesn't. I've spent the last year running a full AI-powered personal operations stack — an AI chief of staff …

Five research agents, one afternoon. The most valuable finding wasn't about the vendors or the competition — it was that my own engineering notes were confidently wrong…

Here's the honest anatomy of why pilots die, and the playbook I use to get AI systems into production.

Here's the honest accounting of what an AI project actually costs, from someone who builds and rescues them for a living. Share it with whoever holds the budget.

RAG stands for "Retrieval-Augmented Generation," but the useful translation is: **an open-book exam.**

Traditional build-vs-buy assumed the thing you're buying is stable — a CRM is a CRM. AI products are not stable. Models change under you monthly, pricing swings wildly, and today's "impossible" fea…

The agent said the rebuild was done and fully tested. Three adversarial reviewers found 31 real issues, two of which would have shipped. Then the fixes needed their own audit…

I asked whether a customer-facing money flow was optimized for revenue. The audit came back with 45 findings — and the worst one had nothing to do with revenue…

I handed an agent staging admin, a third-party sandbox and fifteen hours. It found a real duplicate-charge bug — and two critical bugs that turned out to be my own laptop…

I've sat on both sides of this table — building AI systems and reviewing vendors' claims. Here's the field guide I wish every non-technical leader had before signing anything.

A newly released model everyone was praising would have run slower on my hardware — and one line of arithmetic proved it before the download would have finished…

It answered questions about my inbox beautifully and did nothing at all to my incoming mail. Same model, same machine — the difference was the shape of the call…

I added a small model to sort email and it spent 150 to 200 invisible tokens deliberating before every one-word answer — a latency and cost tax hidden behind a default nobody chose…

Three parallel agents, told to work independently, kept returning the same result — and the obvious fix would have quietly taxed every request for the life of the system…

A setting that won every test I ran took the server down under real traffic — and the value I rolled back to turned out to be right for a reason I had never tested…

A defect I could not fix, shipped on purpose — and the test that decides when a known bug is acceptable engineering and when it is negligence with better formatting…

An in-memory cache is a speed boost with a cliff at every restart — what it cost to measure the cliff, build a durable tier, and give the fix back to the tool instead of hoarding it…

A single randomly-ordered list near the top of the payload made every new session pay full price for work the machine had already done — with no error anywhere to point at…

A server rebooted itself overnight and came back to a password prompt nobody could answer remotely — and why a security control and an uptime requirement sometimes have to be traded on purpose…

Three outages in one day that all looked like network faults. None of them were — each was a consumer default nobody had thought to change on a machine quietly promoted to infrastructure…