Works with the AI tools you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIGitHub CopilotGitHub CopilotGemini CLIGemini CLI+20 more

    local llm hardware fit

    1

    Fit and optimize LLMs for 100% VRAM residency on lab GPU nodes using Ollama and llama-server.

    Free

    2 installsSecurity scanned
    local llm hardware fit

    local llm hardware fit

    Example session with this skill installed

    I need to fit Qwen3.8-27B on GPU-NODE (2x 3060 24GB total). How do I set the Modelfile and verify zero CPU offload at 132k context?

    • Read your context and instructions
    • Compiled the local llm hardware
    • Generated the document

    I will calculate the layer count, generate the Modelfile, and provide verification commands for your pooled VRAM setup.

    GPU-NODE 27B Fit Configuration

    Qwen3.8-27B has a block_count of 65. To achieve 100% VRAM residency, you must set num_gpu to 66 (layers + predictor).

    1. Create the Modelfile

    Run this on GPU-NODE via SSH to define the optimized model:

    ssh llm-user@your-gpu-node "cat > /tmp/qwen_fit <<'EOF'
FROM qwen3.8:27b
PARAMETER num_ctx 135168
PARAMETER num_gpu 66
EOF
ollama create qwen3.8:27b-132k -f /tmp/qwen_fit"

    2. Verify VRAM Residency

    Warm the model and check the processor allocation. If the PROCESSOR column shows anything less than 100% GPU, the KV cache is spilling to system RAM.

    
# Trigger a non-thinking generation
curl -s -X POST http://YOUR_OLLAMA_HOST:11434/api/generate \
  -H \"Content-Type: application/json\" \
  -d '{\"model\":\"qwen3.8:27b-132k\",\"prompt\":\"test\",\"stream\":false,\"think\":false}' > /dev/null

# Check residency status
ssh llm-user@your-gpu-node \"ollama ps\"

    3. Hardware Audit

    Confirm the pool is spanning both cards without exceeding the 24GB ceiling:

    ssh llm-user@your-gpu-node \"nvidia-smi --query-gpu=memory.used,memory.total --format=csv\"

    Next steps

    • Check df -h / on GPU-NODE to ensure the 17GB blob copy didn't fill the root disk.
    • Verify the Master Schedule to ensure no render windows (06:50–09:10 NZT) are active before running long inference.
    • If tok/s is below 10, consider dropping context to 96k to free up VRAM headroom.

    local-llm-hardware-fit.pdf

    PDF · document

    Generated

    Example file from a real run - the skill writes it into your workspace.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    The problem

    Running local LLMs on consumer hardware often results in frustrating CPU offload, poor token speeds, and out-of-memory errors. Developers waste hours guessing layer counts and context sizes instead of achieving 100% VRAM residency.

    What it does

    • Calculates exact num_gpu layer counts and context limits for 100% VRAM residency.
    • Handles Ollama binary upgrades on Linux via GitHub .tar.zst assets when official downloads fail.
    • Configures llama-server with CUDA pinning and KV cache optimization for dual-GPU pools.
    • Verifies model compatibility across mixed hardware including Nvidia and AMD GPU's.
    • Audits inference processes using nvidia-smi and ollama ps to ensure zero CPU spill.

    Frameworks & tools

    Ollama, llama.cpp (llama-server), CUDA 12.8, ROCm/Vulkan, Ubuntu/Linux, and WSL2.

    Why this beats prompting it yourself

    Generic LLMs don't understand the specific quirks of pooled VRAM, ROCm Polaris limitations, or the text file busy errors during binary swaps. This skill provides a concrete technical playbook based on real lab telemetry and hardware-specific edge cases.

    Use cases

    • Fitting a 27B model into a 24GB pooled VRAM environment with 128K context.
    • Recovering from invalid argument: --load-mode errors after broken Ollama updates.
    • Deploying llama-server with shared library fixes (.so linking) on headless Linux nodes.
    • Benchmarking context ladders for new GGUF quants to find the real VRAM ceiling.

    Known limitations

    Requires SSH access to target nodes. Does not support Flash-Next MoE models on single 24GB cards due to physical VRAM constraints. Linux-centric; avoids Windows shell commands for GPU-NODE operations.

    How to install

    Works the same in every agent - Claude, Cursor, Codex, Copilot and 20+ more.

    ~30 seconds
    1. 1

      Download the ZIP

      Free skills download straight away. Paid skills unlock right after purchase.

    2. 2

      Unzip into your skills folder

      Every agent reads skills from one folder on your machine. Drop the unzipped folder in there.

    3. 3

      Ask your agent to use it

      Restart the agent if it was already running. It picks the skill up automatically - no config needed.

    Skills folder by agent

    Click the path to copy it. Create the folder if it does not exist yet.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill passes our trust checks below.

    Security scanned

    Passed our 8-point scan before listing

    2 installs

    Downloaded by developers to date

    Free forever

    No account required to browse

    Trust & safety

    Security scanned

    Verified clean 11 days ago

    • Free to download with an account

    Listed11 days ago

    What's inside

    Frequently Asked Questions