AI System Resilience & Evidence Suite

Inventory every LLM model and provider your code depends on, the AI bill of materials, and flag the dependency risk. It lists each provider, model, and where it's used, then flags hardcoded model ids, single-provider dependency with no alternative, the same model referenced by different ids, model ids with no config or env indirection, and providers pinned in your manifests. Recognizes OpenAI, Anthropic, Google Gemini, and more from an editable list.

Find the LLM integration code that breaks when a model blocks a response or falls back to a different model. Flags calls with no try/except or refusal branch, responses used or parsed with no guard for a blocked or empty answer, and hardcoded model ids with no fallback handling. Built for the Fable 5 era, where a high-risk call is blocked and silently falls back to Opus 4.8.

Generate the model and vendor risk register a security lead asks for the morning after a model gets switched off. It scans your codebase for every model and provider, merges in the owner and data-residency notes you supply, and renders a register with provider, model, where it's used, fallback status, and an availability-risk rating per dependency. Markdown by default, CSV with a flag. Continuity-focused, not a compliance assessment.

Find the LLM integration code that will not survive a provider being pulled or going down. Flags single-provider lock-in with no alternative, calls with no failover branch, missing timeouts, retries with no limit or backoff, no degraded-mode default, and hardcoded endpoints with no alternate. This is about the model going away, not the model declining.

Replay automation traces against explicit rules for required steps, order, retries, approvals, idempotency, external side effects, duration, and terminal success.

Audit real RAG evidence traces for missing expected sources, impossible citations, unsupported claims, stale evidence, unused context, and weak routing.