AI System Resilience & Evidence Suite

Find pattern-based model/provider references in reviewed source and prepare an explicitly incomplete inventory for deployment review.

Find the LLM integration code that breaks when a model blocks a response or falls back to a different model. Flags calls with no try/except or refusal branch, responses used or parsed with no guard for a blocked or empty answer, and hardcoded model ids with no fallback handling. Built for the Fable 5 era, where a high-risk call is blocked and silently falls back to Opus 4.8.

Generate the model and vendor risk register a security lead asks for the morning after a model gets switched off. It scans your codebase for every model and provider, merges in the owner and data-residency notes you supply, and renders a register with provider, model, where it's used, fallback status, and an availability-risk rating per dependency. Markdown by default, CSV with a flag. Continuity-focused, not a compliance assessment.

Find the LLM integration code that will not survive a provider being pulled or going down. Flags single-provider lock-in with no alternative, calls with no failover branch, missing timeouts, retries with no limit or backoff, no degraded-mode default, and hardcoded endpoints with no alternate. This is about the model going away, not the model declining.

Replay automation traces against explicit rules for required steps, order, retries, approvals, idempotency, external side effects, duration, and terminal success.

Audit real RAG evidence traces for missing expected sources, impossible citations, unsupported claims, stale evidence, unused context, and weak routing.