“Nothing wrong” and “I could not look” are not the same answer
Three safety checks in a product we build looked healthy for weeks while doing nothing. Each one answered “all clear” when the truth was “I could not check.” What we changed, and five questions to ask about the checks you rely on.
A check has three possible answers: everything is fine, something is wrong, or I could not look. Most systems are built to tell the first two apart. The third one usually gets filed under “fine”, and that is where the trouble starts.
In one week of review on a sales management tool we build, we found three controls that had been doing exactly that. None of them had ever raised an error. All three looked healthy on every dashboard.
If you lead the team: what to ask
- For each automatic check we rely on: when did it last run, and where can I see that date?
- What does it report when it cannot reach what it is supposed to check?
- When did we last see it fail on purpose?
The nightly check that had stopped
Every night the product looks over each customer’s data for things that need attention. Its record of having run simply ended, weeks earlier. Nobody noticed.
The reason is the useful part. When the check could not work out which account it was looking at, it returned an empty list. An empty list means “no problems found”. So the job reported success, the monitoring recorded a success, and the page that shows the results displayed a list that just stopped, with nothing beside it to say why.
The release gate that approved everything
Before any release, a gate confirms that the automated tests passed for the exact version being shipped. It asks another service for the results. The credential it used to ask had expired, so every request failed.
The gate had been written to let a release through when it could not get an answer. The reasoning was fair: a gate that blocks releases because of its own connection problems teaches people to bypass it. The result was a gate that could not say no. It approved every release without reading a single test result.
The record that was never written
The product keeps a log of every change to a customer’s data: who did what, to which record, and when. It is the first thing a security review asks to see. A number of actions were not writing to it, among them the moment a customer’s connection to their other systems is stored.
Adding the missing lines would not have been enough. The log identifies the account by looking at who is signed in, and for an overnight job or an incoming notification from another system, nobody is. The write would have failed without a sound, and the code would have read as if the logging were there.
It would also have passed the tests. The test environment has no sign-in step, so the write succeeds there. The failure exists only in production.
What the three had in common
Each one failed toward “all clear”. To anything that only watches for errors, a check that found nothing and a check that never looked are identical. Monitoring cannot see a problem that reports itself as a success.
These were not careless decisions. Returning an empty list when there is nothing to list is normal code. Letting a release through when a status service is down is a reasonable call. Each choice made sense where it was made. Together they produced a system that would have stayed green through almost anything.
What we changed
- A check that cannot see now says so. “Could not look” is its own result, separate from “nothing found”, and it counts as a failure of the job.
- A failed job raises an alert, not a line in a file somebody has to think of opening.
- The release gate refuses when it cannot confirm. Overriding it takes a written reason, and the reason is recorded. There is no setting that can be switched once and forgotten.
- The build fails if someone adds an action that changes data without logging it. The next one cannot be missed the way these were.
- “The tests pass” is no longer accepted alone for anything that runs without a signed-in user. Those paths are run for real and their output is read back.
We also found that our first search for unlogged actions had been too narrow. It looked for one kind of request, and scheduled jobs and browser links are a different kind, so they were invisible to it. Widening the search found more. A check on your checks deserves the same suspicion as the checks.
How to look for this in your own systems
- Ask every check for its last run, with a date. A check without a visible “last ran” is one you cannot tell from a dead one.
- Alert on silence. If a nightly job has not reported by morning, that is the alarm. Do not wait for it to report a failure it may never be alive to report.
- Make it fail on purpose. Put a known problem in front of it and watch it complain. A check nobody has seen fail is a hope. We do the same with warnings, by checking them against a case where we know the right answer.
- Decide what happens when it cannot check. For anything that protects customers or money, the safe default is to stop and ask a person.
- Be suspicious of long calm. Weeks of zero findings from a check on live data is either very good news or no news at all.
Why this matters more with AI in the product
An AI feature adds more places where an empty or plausible answer can stand in for a real one: a model that returns nothing, a source that could not be read, a step that was skipped. In our method, the Trust stage exists for this. Before anyone acts on a number, the system has to be able to say where it came from and whether it actually looked.
It is also part of what a prototype skips. A demo only has to work when everything answers. A product has to tell you when something did not.
This article is drawn from a real engagement. Client details withheld; every figure comes from the client’s own data.
Read the full case study →