Shift log
Repair bay

Why your AI automation keeps breaking — and the five fixes

Most "the bot stopped working" tickets have the same handful of causes. Here they are, with the fix for each.

25 Oct 2026 · 2 min read

About one in five small businesses that use AI say they have paid someone to fix or redo AI work. That matches what shows up in the repair queue: the automation worked in the demo, ran for a few weeks, then quietly started producing wrong results or stopped altogether. The causes are boringly consistent.

1. The prompt was tested on three examples

A workflow that was checked against three happy-path inputs will meet its fourth case in production. The fix is not a cleverer prompt; it is a test set. Collect twenty to fifty real inputs — including the ugly ones — with the output you expect, and run them every time anything changes.

2. The model changed under you

Hosted models get updated. A prompt tuned for last quarter's version can drift. Pin the model version where the provider allows it, keep the test set from fix #1, and run it on a schedule so you learn about drift before your customers do.

3. Nothing checks the output

If an LLM step feeds straight into a CRM update or an email send, every mistake ships. Put a validator between them: the output must match a schema, numbers must be in range, required fields must exist. Anything that fails goes to a review queue, not to the customer.

4. The integration, not the AI, broke

Half of "the AI is broken" tickets are an expired API token, a renamed spreadsheet column, or a rate limit. Add a heartbeat that runs the whole workflow with a known input once an hour and alerts when the output differs. It costs cents and catches most outages within the hour.

5. Nobody owns it

Automations built by a departed contractor, with credentials in their personal account, break on the day that account closes. Every automation needs a named owner, shared credentials in a vault, a one-page runbook, and a log of every run that someone can read without the builder present.

The pattern across all five: the fix is almost never a better model. It is a test set, a validator, a heartbeat and an owner.

When to rebuild instead of repair

If the workflow has more than about fifteen steps in a no-code tool, has three or more LLM calls in sequence, or needs retries and branching to behave, it has outgrown no-code. At that point a small amount of real code — with tests — is cheaper to keep alive than a sprawling diagram.

Got one that keeps breaking? Send it to the repair bay — written diagnosis first, then a fixed price for the fix.

SIGNED OFF
Written on shift by Milan Rathod.Clock in →