Works with the AI tools you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIGitHub CopilotGitHub CopilotGemini CLIGemini CLI+20 more

    llama server ops

    1

    Production-grade playbook for serving, fitting, and verifying llama.cpp llama-server on multi-GPU Linux systems.

    $5

    Secure checkout via Stripe

    30-day refund guarantee

    Converts to your local currency at checkout

    0 installsSecurity scanned
    llama server ops

    llama server ops

    Example session with this skill installed

    I need to swap the current model for a Qwen3.6-14B-A3B Q4_K_M GGUF on my dual 3060 setup. Maximize context for a single user. Walk me through the ladder protocol and service update.

    • Read your context and instructions
    • Compiled the llama server ops

    I will now execute the ladder protocol to find the VRAM ceiling for the Qwen3.6-14B-A3B model and update your systemd service.

    Llama-Server Deployment Plan

    StepActionCommand/Metric
    1Stop existing servicessystemctl stop llama-server
    2Ladder Test: 64K Context9.1GB VRAM (Measured)
    3Ladder Test: 128K Context10.0GB VRAM (Measured)
    4Target: 192K Context10.9GB VRAM (Best Fit for 12GB Card)
    5Update Unit Filesed -i 's/old-model.gguf/qwen3.6-14b-a3b-q4_k_m.gguf/'
    6Reload & Restartsystemctl daemon-reload && systemctl restart llama-server
    7Liveness VerificationGET /health -> {"status":"ok"}

    Configuration Details

    
# Optimal Execution Flags
llama-server \
  -m models/qwen3.6-14b-a3b-q4_k_m.gguf \
  -ngl 99 \
  -c 196608 \
  --parallel 1 \
  -fa on \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  --temp 0.4 \
  --top-p 0.92

    Next steps

    1. Run scripts/needle_bench.py to confirm 192K retrieval accuracy.
    2. Monitor journalctl -u llama-server -f for graph_reserve OOM errors during first long prefill.
    3. Verify that reasoning_effort: none is passed in your client requests to maximize generation budget.

    Connects securely to your tools. The creator never sees your data.

    What you get

    Deploy stable high-context LLM endpoints using llama-server.Optimize VRAM allocation for multi-GPU GGUF sharding.Automate safe model-swap routines for systemd services.Benchmark true context limits via the ladder protocol.

    About this skill

    The problem

    Running local LLMs via llama-server often leads to silent failures, Out-Of-Memory (OOM) errors during long-context prefill, and deceptive health checks. Developers struggle to accurately predict VRAM usage for GGUF models across multi-GPU setups, especially when balancing context window size against batch parallelism.

    What it does

    • Manages the llama-server lifecycle, including safe model swaps, systemd unit updates, and daemon-reloads.
    • Calculates real-world VRAM ceilings for GGUF models, accounting for KV cache quantization and activation memory.
    • Executes multi-stage verification protocols including context-soak tests and "needle in a haystack" retrieval benchmarks.
    • Configures optimized sampling defaults and request parameters to suppress reasoning tokens in thinking models.
    • Orchestrates multi-GPU offloading for Mixture-of-Experts (MoE) models and handles PCIe bandwidth optimization.

    Frameworks & tools

    llama.cpp (llama-server), Linux/WSL, systemd, NVIDIA CUDA (nvidia-smi), GGUF model format, and OpenAI-compatible API endpoints.

    Why this beats prompting it yourself

    Generic LLMs often use naive VRAM math that ignores sliding-window compression and compute buffer overhead. This skill uses a proven "ladder protocol" to measure actual hardware limits, preventing silent background launch failures and OOM crashes that occur 200 seconds into a prefill.

    Use cases

    • Deploying a persistent 192K context inference endpoint on consumer GPUs.
    • Upgrading production GGUF weights without dropping the active service unit.
    • Benchmarking MoE model performance and VRAM spill across multi-GPU sharding.
    • Automating liveness probes that distinguish between "Loading model" and "Ready".

    Known limitations

    Platform limited to Linux and WSL. Focuses exclusively on the llama-server runtime; does not support Ollama-specific API endpoints or Intel Arc SYCL backends for non-LLM workloads.

    How to install

    Works the same in every agent - Claude, Cursor, Codex, Copilot and 20+ more.

    ~30 seconds
    1. 1

      Download the ZIP

      Free skills download straight away. Paid skills unlock right after purchase.

    2. 2

      Unzip into your skills folder

      Every agent reads skills from one folder on your machine. Drop the unzipped folder in there.

    3. 3

      Ask your agent to use it

      Restart the agent if it was already running. It picks the skill up automatically - no config needed.

    Skills folder by agent

    Click the path to copy it. Create the folder if it does not exist yet.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill passes our trust checks below.

    Security scanned

    Passed our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    30-day refund

    Not a fit? Get your money back

    Trust & safety

    Security scanned

    Verified clean 6 days ago

    • Passed all security checks, Safe to install

    Listed6 days ago

    What's inside

    Frequently Asked Questions