Works with the AI tools you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIGitHub CopilotGitHub CopilotGemini CLIGemini CLI+20 more

    Hardware Model Sizing

    1

    Predict exact LLM fitment for local hardware by calculating weight, KV cache, and VRAM overhead.

    $5

    Secure checkout via Stripe

    30-day refund guarantee

    Converts to your local currency at checkout

    0 installsSecurity scanned
    Hardware Model Sizing

    Hardware Model Sizing

    Example session with this skill installed

    I have a machine with two GPUs, a 3080 10GB and a 4090 24GB. I need to run a 32B model at 32k context for coding. What fits?

    • Read your context and instructions
    • Compiled the hardware model sizing
    • Generated the document

    I have inventoried your hardware and calculated the usable memory pools. Because these are discrete GPUs, they must be treated as separate pools rather than a 34GB sum.

    Hardware Sizing Audit

    Inventory & Usable Budget

    • Pool 1: NVIDIA RTX 4090 (24GB VRAM) -> 22.0 GB usable
    • Pool 2: NVIDIA RTX 3080 (10GB VRAM) -> 8.0 GB usable
    • Target Context: 32,768 tokens (FP16 KV)
    • KV Cache Overhead: ~8.2 GB (for 32B class dense models)

    Model Recommendations (32B Class)

    Model / QuantTotal Runtime (Weights + KV)Status (Pool 1)Status (Pool 2)
    32B Q4_K_M~26.2 GB (18.0 + 8.2)Offload (Needs ~4GB CPU)Don't Bother
    32B Q3_K_L~22.1 GB (13.9 + 8.2)Fits (0.1GB slack)Don't Bother
    32B Q2_K~18.8 GB (10.6 + 8.2)Fits (3.2GB slack)Don't Bother

    Analysis

    Workhorse: At 32k context, a 32B model is too heavy for the 3080. On the 4090, a Q3_K_L is the highest quality that fits entirely in VRAM.

    Quality Ceiling: You can run Q4_K_M on the 4090, but you will see a performance drop as ~4GB of layers spill into system RAM.

    Don't Bother: Avoid running dense 32B models on the 10GB 3080 for this context length; even at 2-bit quants, the KV cache alone consumes nearly the entire pool.

    Next steps

    • Download the Q3_K_L GGUF for the 4090 to maintain full VRAM speed at 32k context.
    • If quality is prioritized over speed, use the Q4_K_M with partial CPU offloading.
    • Consider a 14B model family if you need high-speed inference on the 3080.

    hardware-model-sizing.pdf

    PDF · document

    Generated

    Example file from a real run - the skill writes it into your workspace.

    Connects securely to your tools. The creator never sees your data.

    What you get

    Calculate VRAM requirements including KV cache for specific context lengths.Audit multi-GPU setups to prevent improper tensor-split assumptions.Query Hugging Face API to sum GGUF shards for accurate weight sizing.Identify the maximum quantization quality a specific box can handle.

    About this skill

    The problem

    Developers often struggle to predict if a local LLM will actually run on their hardware without hitting Out-of-Memory (OOM) errors. Marketing nicknames and raw file sizes on Hugging Face are misleading because they ignore KV cache overhead, operating system requirements, and the physics of discrete vs. unified memory pools.

    What it does

    • Inventories local hardware to identify discrete VRAM pools and unified memory availability without relying on marketing stickers.
    • Calculates real-world usable budget by subtracting OS and runtime overhead from raw hardware specs.
    • Queries Hugging Face GGUF file trees to sum multi-file shards and verify actual weight sizes on disk.
    • Factors in KV cache memory requirements based on your specific target context length and data type.
    • Categorizes models into "fits", "offload", or "don't bother" buckets for your specific machine.

    Why this beats prompting it yourself

    General-purpose LLMs frequently hallucinate that you can sum the VRAM of two different GPUs or forget that MoE models require the full weight set in memory. This skill uses a deterministic calculation process and live API lookups to ensure you don't waste hours downloading a 40GB model that will OOM at 8k context.

    Use cases

    • Determining the largest model family and quantization level that fits a specific developer workstation.
    • Validating vendor claims about running massive models on entry-level hardware before purchasing.
    • Sizing a machine's model runway to see if it can handle 128k context tasks versus 8k chat.
    • Auditing a multi-GPU setup to decide between tensor splitting or running separate instances.

    Known limitations

    Does not support cloud API model routing, image/video diffusion models, or sizing for training and LoRA fine-tuning. Assumes GGUF/llama.cpp runtime.

    How to install

    Works the same in every agent - Claude, Cursor, Codex, Copilot and 20+ more.

    ~30 seconds
    1. 1

      Download the ZIP

      Free skills download straight away. Paid skills unlock right after purchase.

    2. 2

      Unzip into your skills folder

      Every agent reads skills from one folder on your machine. Drop the unzipped folder in there.

    3. 3

      Ask your agent to use it

      Restart the agent if it was already running. It picks the skill up automatically - no config needed.

    Skills folder by agent

    Click the path to copy it. Create the folder if it does not exist yet.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill passes our trust checks below.

    Security scanned

    Passed our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    30-day refund

    Not a fit? Get your money back

    Trust & safety

    Security scanned

    Verified clean 1 day ago

    • Passed all security checks, Safe to install

    Listed1 day ago

    Frequently Asked Questions