model routing

The problem

Running advanced agentic workflows across local hardware often leads to VRAM fragmentation, context window collisions, and unmanaged API costs. Developers struggle to maintain high-performance local coding models while simultaneously running GPU-intensive tasks like ComfyUI renders.

What it does

• Routes tasks through a three-tier system: Local Routine (27B), Agentic Coder (9B), and Remote Premium fallback.
• Enforces render-exclusive windows to prevent OOM errors during scheduled GPU tasks.
• Satisfies Hermes 64K context floors by managing baked Modelfile parameters across different node specs.
• Offloads lightweight internal operations (triage, title gen, approvals) to auxiliary nodes to save primary GPU compute.

Frameworks & tools

Ollama, OpenRouter, Hermes, RTX 3060 Dual-GPU (24GB), Windows WSL, and Python-based agent environments.

Why this beats prompting it yourself

Manual model selection fails during concurrent agent runs and hardware-heavy render cycles. This skill handles the complex logic of VRAM fitting, port-forwarding across network nodes, and specific OpenAI-compatible header requirements for local LLMs.

Use cases

• Executing repo-level bug fixes on a secondary node while the main GPU renders high-resolution video.
• Maintaining a 192K context window for routine health checks without burning through paid API tokens.
• Scaling auxiliary agent functions to low-spec hardware (4GB VRAM) using specific context bakes.
• Falling back to high-reasoning remote models only when local tiers reach saturation.

Known limitations

Ollama /v1 endpoints ignore runtime context parameters, requiring context floors to be baked directly into Modelfiles. SSH access to auxiliary nodes is restricted to HTTP API calls only.