Nex Multi Llm Fallback Router

What it does

• Generates fallback_provider.py, a FallbackProvider that wraps an ordered list of providers and tries them in turn, skipping any that times out, errors transiently, or returns invalid output
• Ships base.py with the shared AIProvider interface and error taxonomy (AIProviderTimeout, AIProviderError, AIProviderInvalidOutput), identical across the companion provider skills
• Ships factory.py with build_generation_chain() and build_scoring_chain(), the single place chain order lives, with graceful skipping of providers whose API key is missing
• Default chain is Claude, then Gemini, then Qwen, then Ollama: strong paid, cheaper paid, cheap API, free local model
• The router implements AIProvider itself, so it drops in anywhere a single provider was expected, and routers can nest

Why use this

One rate-limited API should not stall a bulk generation or scoring run; with this router the pipeline degrades to the next model instead of failing, down to a free local Ollama floor. This is the composition layer Nex AI runs in production, and because the router hides behind the same interface as every provider, adding failover to existing code is a one-line swap.

See it in action

Prompt:

If Claude is down or rate-limited, I want it to fall back to Gemini automatically, and to a local model if both are out.

Result:

class FallbackProvider(AIProvider):
    def __init__(self, providers: list[AIProvider], label: str = "chain"):
        self.providers = providers

    def generate(self, prompt: str, **kwargs) -> str:
        last_error = None
        for provider in self.providers:
            try:
                return provider.generate(prompt, **kwargs)
            except (AIProviderTimeout, AIProviderError,
                    AIProviderInvalidOutput) as e:
                log.warning("provider %s failed, trying next", provider.name)
                last_error = e
        raise last_error  # all failed: clean failure, never swallowed

chain = build_generation_chain()  # Claude -> Gemini -> Ollama

Use cases

• Automatic failover when one model API has an outage or rate limit
• A cost-aware scoring chain that starts cheap and escalates only on failure
• A single AIProvider interface in front of multiple model vendors
• Keeping overnight bulk jobs running on a local model when paid APIs are down

Known limitations

• Failover by priority, not load balancing; it does not round-robin for throughput.
• The composition layer only: the individual Claude, Gemini, Qwen, and Ollama providers come from their own integration skills (constructor stubs are commented in factory.py).
• Python only.

FAQ

What exactly is in the download?

SKILL.md, assets/base.py, assets/fallback_provider.py, assets/factory.py, assets/requirements.txt, two reference docs (chain-design.md, shared-interface.md), manifest.json, README, changelog, and license.

Which agents does it work with?

Claude Code, Claude Cowork, Cursor, Codex CLI, and any agent that reads the SKILL.md format.

Do I need anything else?

The router itself has no dependencies beyond the providers it chains. You need API keys for the paid providers you wire in; a missing key skips that provider instead of crashing. Ollama needs your own hardware.

Can I change the chain order?

Yes, the factory is the only place the order is decided. Generation chains usually start at the strongest model; scoring chains start cheap and fall through to local, and the chain-design reference explains why.