More screenshots
Works with the AI Agents you already use
llama.cpp: Run LLMs on Hardware You Own
Run LLMs on CPUs, Apple Silicon, and non-NVIDIA GPUs: pick the right GGUF quantization, build for your hardware, serve an API.
$5
llama.cpp: Run LLMs on Hardware You Own
Example session with this skill installed
Run a 7B model on my laptop CPU with no GPU and no Python stack. I need an API my app can call.
- Read your context and instructions
- Compiled the llama cpp
Route chosen: download a pre-quantized GGUF build — no conversion step, no Python stack, one binary and one model file. Quantization picked by the table: Q4_K_M, the default balance at about 4.1 GB for a 7B model — fast enough on a laptop CPU without the visible degradation of Q2_K. Build: CPU-only make — no Metal, no CUDA, no ROCm flags, because this machine has no GPU to accelerate with. Verified the binary runs before serving anything. Server mode: ./llama-server -m model.gguf --host 0.0.0.0 --port 8080 — OpenAI-compatible API on port 8080, host bound to all interfaces so the app on the same network can reach it. Context set with -c 4096, sized deliberately: context memory grows with the square of the length, and it is the usual OOM culprit, not the weights. Measured, not guessed: timed tokens per second on the app's actual prompts — tens of tokens per second on this laptop CPU, matching the rough expectations for a 7B Q4_K_M. Published benchmarks are hardware-specific fiction until reproduced. Extras wired in: a GBNF grammar for the JSON-only endpoint, so the app's structured responses are guaranteed to match the schema instead of hoped-for. Embeddings noted as available from the same binary for a later retrieval step.
Connects securely to your tools. The creator never sees your data.
About this skill
llama.cpp runs modern LLMs on hardware you already own — laptops, desktops, edge boxes, AMD and Intel GPUs — by doing the unglamorous work well: tight C++ kernels, aggressive quantization, and minimal dependencies. This skill covers the whole path: getting a GGUF model by download or conversion, picking a quantization with the size and speed trade made explicit (Q4_K_M as the default balance), building for your hardware target with the right GPU offload flags, and running one-shot, interactive, or server mode behind an OpenAI-compatible API. Performance tuning is measurement, not guessing — time tokens per second on your actual prompts — plus the extras that matter in production: grammar-constrained generation, embeddings, and multimodal builds. The troubleshooting table covers the usual suspects: OOM from context length, garbage output from a bad download, slow Apple Silicon from a Metal-less build.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for LLM Ops.
With NVIDIA datacenter GPUs and maximum throughput needs, a CUDA serving engine will be several times faster — this skill says so rather than overselling CPU inference.
Context length defaults small and context memory grows with the square of the length; raising it without watching RAM is the usual OOM.
Rough token-per-second expectations are hardware-specific until measured on your machine.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 3 days ago
- Software
Git