Four automations that exited zero and did nothing

In a single day I traced four separate outages across automations I had built. Three of them had exited successfully. The scripts ran, returned zero, and did nothing at all. Every dashboard was green.

That is the failure mode nobody demos, nobody sells against, and almost nobody monitors for. It is also the one that costs the most, because it does not cost you an afternoon of firefighting. It costs you three months of a job you believed was being done.

Four ways to fail while reporting success

An access token expired

A social publishing job ran forty six times against an expired credential. Each run started, found nothing it could do, and finished. The platform returned an authentication error, the job caught it, logged it at debug level, and exited zero because from its own point of view it had completed.

Forty six posts that never existed. Nobody noticed for weeks.

A quality gate rejected the work and nothing retried

A writer produced a draft, a fact-checking step rejected it, and the flow discarded the draft and ended. The scheduler marked the task complete, which it was. No article was queued, no alert fired, and the next run started from scratch and did the same thing.

A week of output vanished into a code path that was, strictly speaking, working.

A buffer that could never refill

A content queue generated one item per run and consumed one item per run. On paper that is steady state. In practice any failed generation is permanent loss, because there is no headroom to recover it. The buffer only ever went down.

That is a ratchet, not a buffer, and it looks completely healthy until the day it hits zero.

A partial send that looked like a full one

A newsletter reached about a fifth of its list. The send function returned success, because it had successfully sent the first batch. Nothing compared the number of recipients attempted against the size of the list.

From inside the system a partial send and a complete send are identical. The only way to tell them apart is to count.

Why exit codes are the wrong thing to watch

An exit code answers one question: did the process reach the end without crashing. That is not the question you care about. You care whether the thing you wanted to happen happened.

Those two questions come apart constantly, because well-written code catches its errors. The better your error handling, the more likely a broken run exits cleanly. Defensive programming and silent failure are the same behaviour viewed from two directions.

Monitoring that only asks whether a job ran will tell you your business is fine right up to the moment a customer tells you it is not.

What to check instead

Every one of the four failures above is caught by a check on the outcome rather than the process. None of them are complicated.

• Did the artefact appear? After a publishing run, query the destination and confirm the post exists. Not the log. The destination.
• Is the queue deeper than one? Track depth over time, not presence. A queue that never grows is failing even while it is non-empty.
• How long until this credential expires? Every token has a deadline. Alert before it, refresh ahead of it, and alert if the refresh itself fails.
• Did it reach everyone? Compare recipients attempted against the audience size. Any gap is a failure regardless of what the send function returned.
• When did this last actually succeed? Not last run. Last success, defined as an outcome you can see from outside the system.

One board, not five

The other half of this is where the answers go. Five automations reporting to five places is functionally the same as no monitoring at all, because nobody checks five places every morning.

Everything I build now reports to a single status board with one row per automation, showing the last verified outcome and the age of it. If every row is fresh, nothing needs attention. If a row goes stale, that is the only thing you have to look at.

It is a deliberately boring artefact. Boring is the point. The value is that checking it takes four seconds, which means it actually gets checked.

The uncomfortable part

If you have automation running right now that you have not verified from the outside in the last month, the honest position is that you do not know whether it is working. You know it has not crashed.

Those are very different claims, and only one of them is worth anything.