Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIDeepSeekDeepSeekMuseMuse+22 more

    Local LLM Fit and Benchmark Kit

    1

    Check whether a local LLM fits your GPU and how fast it runs. Three standard library Python scripts estimate memory, benchmark Ollama or vLLM, and write a scored report that ranks your models.

    $5

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    Local LLM Fit and Benchmark Kit

    Local LLM Fit and Benchmark Kit

    Example session with this skill installed

    I have a 24 GB GPU and Ollama is running on this machine. Check whether llama3.1:8b, qwen2.5:14b and qwen2.5:32b fit with an 8k context, benchmark the ones that do, and tell me which is best for a chat assistant.

    • Read your context and instructions
    • Compiled the local llm fit

    Summary

    All three models fit on a 24 GiB GPU at 8,192 tokens of context. The 32B model is tight. [Replace with one sentence on which model scored best and why, from your run.]

    Memory check (GiB, 8,192 token context)

    ModelQuantWeightsKV cacheTotalShare of GPUVerdictLongest context that fits
    llama-3.1-8bq4_k_m4.531.006.5327%comfortableabout 151,000
    qwen2.5-14bq4_k_m8.301.5010.8045%comfortableabout 80,000
    qwen2.5-14bq8_014.551.5017.0571%comfortableabout 46,000
    qwen2.5-32bq4_k_m18.352.0021.3589%tightabout 19,000

    For comparison, llama-3.1-70b at q4_k_m needs 43.36 GiB (181% of this GPU) and does not fit.

    Benchmark (Ollama on [your GPU name], 5 runs, 128 tokens each)

    ModelDecode tokens per secondTime to first token (s)SuccessScoreVerdict
    llama3.1:8b[from your run][from your run][x/5][score][verdict]
    qwen2.5:14b[from your run][from your run][x/5][score][verdict]
    qwen2.5:32b[from your run][from your run][x/5][score][verdict]

    Recommendation

    [Replace with your agent's actual recommendation from the report.]

    Next steps

    • Re-run with --concurrency 4 if more than one person will use the server.
    • A tight verdict leaves little room, so shorten the context or use a smaller quantization before relying on it.
    • Run --list to see the built in presets, or pass your own model's numbers from its model card.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    The problem

    You pick a model and a quantization, then guess whether it fits your GPU and quote speeds nobody measured on your hardware. A chat model asked to guess will sound sure and can be wrong.

    What you get

    • vram_fit.py works out weights, KV cache and overhead, says comfortable, tight or does not fit, and gives the longest context that fits.
    • bench_endpoint.py streams from your Ollama or OpenAI compatible server (such as vLLM) and records time to first token, tokens per second, failed requests and total throughput with several users at once.
    • make_report.py builds a Markdown report that ranks your models with a 0 to 100 readiness score and a plain verdict for each.
    • SKILL.md tells your agent when to run each script and how to explain the results.

    Why this beats prompting it yourself

    A model cannot measure your hardware. These scripts time real requests, count failures, and use the same formula every run, so two models are compared fairly.

    Known limitations

    • You give the GPU memory or paste nvidia-smi text. It does not detect the GPU.
    • Six model presets are built in and they are approximate. Any other model works if you enter its numbers from the model card.
    • Multi GPU splits, CPU offload and speculative decoding are not modelled.
    • It checks fit and speed. It does not test answer quality.
    • Tested with a mock streaming server and hand computed memory cases. Results on your machine depend on your setup.

    Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for LLM Ops.

    No support for multi-GPU, CPU offload, or speculative decoding. Architecture presets are approximate; exact model card parameters preferred for custom models.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before buying a skill.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 3 days ago

    Software
    PythonTerminalShell

    Permissions

    Terminal / Shell
    Read Files
    Write Files
    Network Access
    Listed3 days ago

    What's inside

    Frequently Asked Questions

    This skill automates the complex math of VRAM overhead (including KV cache) and runs live stress tests on your local endpoints to ensure your hardware can handle specific models without crashing or lagging.

    The kit is compatible with any local server providing an OpenAI-compliant API or an Ollama endpoint, such as vLLM, LM Studio, and LocalAI.

    You receive a suite of diagnostic Python scripts (vram_fit.py, bench_endpoint.py, and make_report.md) along with documentation on how to interpret the readiness scores.

    The scripts run in any Python 3 environment; you simply provide your hardware specs (like VRAM capacity) and your local API URL to begin the analysis.

    Yes, the kit specifically evaluates performance degradation during concurrent requests, helping you determine how many simultaneous users or agents your local setup can support.

    The current version focuses on single-GPU setups and does not currently calculate performance for multi-GPU splits or system RAM offloading.

    The memory check is plain math, so you can type your memory in. The benchmark works with any Ollama or OpenAI compatible server on any hardware. Only pasting nvidia-smi text is NVIDIA specific.

    New in LLM Ops