ai automation operations review

The problem

AI automation agencies get blamed for outcomes they can't actually verify. A client says something's wrong. The developer says it's fixed. Nobody has proof either way — just a raw log nobody has time to read closely, a claim, and a stakeholder who wants an answer in five minutes. Most incident write-ups end up being whichever explanation sounded most confident, not the one the evidence actually supports.

What it does

• Classifies every piece of evidence before treating it as fact — an observed state, a failed check, a claim, or a stated requirement — so "the verification failed" never quietly becomes "the system failed"
• Cross-checks evidence across sources (same host, version, timestamp, identifier) before calling anything a contradiction, and resolves one only when direct, matching, higher-tier evidence actually settles it — otherwise it stays open, on the record
• Applies a temporal firewall so a later healthy check can't retroactively cover an earlier failure window, and an identity firewall so one host's test result never gets assumed onto another
• Refuses to call a fix "verified" until it sees deployment plus retest evidence, not just a developer's word — tracks Recovery as its own field, separate from severity
• Discloses exactly how much of the supplied evidence it actually reviewed, instead of quietly sampling a large log and reporting as if it read all of it
• Scales from one automation to a full client fleet with a dedicated fleet matrix, without collapsing everything into one severity number

Who it's for

AI automation agencies running more than one automation in a client's production environment — especially anyone who has the "what happened, and is it safe to turn back on" conversation with a client more than once a month.

What you get

• The full review skill (SKILL.md)
• A reference file of 11 worked calibration examples — including when a contradiction should actually be resolved, and when a later check doesn't cover an earlier gap
• A one-page, client-ready verdict up top, with the full evidentiary reasoning underneath

What it's not

Not a monitoring tool, not a log aggregator, not a replacement for your observability stack. It doesn't watch your systems — it makes disciplined sense of the evidence you already have, and says plainly what it couldn't fully review.