Works with the AI Agents you already use
vllm optimizer
Optimize vLLM serving performance with automated benchmarking, YAML generation, and performance constraint analysis.
$9
vllm optimizer
Example session with this skill installed
Create a vllm-optimizer YAML for a Llama-3-8B model on a single A100. I need to stay under 50ms TTFT for an interactive chat workload.
- Read your context and instructions
- Compiled the vllm optimizer
I have generated a validated experiment configuration for Llama-3-8B. This setup targets your 50ms TTFT constraint by testing variations in the KV cache allocation and max tokens while keeping the serving engine on loopback.
vLLM Optimization Configuration
project: llama3-chat-optimization
model: /path/to/models/Llama-3-8B
workload:
backend: vllm
traffic:
arrival_rate: 5.0
input_len: 512
output_len: 128
route: /v1/chat/completions
optimization:
maximize: throughput
constraints:
- field: ttft_p95
operator: lt
value: 0.050
search:
fixed:
gpu_memory_utilization: 0.90
enforce_eager: false
tunable:
max_num_seqs: [16, 32, 64]
block_size: [16, 32]
benchmarks:
- name: interactive_chat_smoke
samples: 100
warmup: 10
Next steps
- Update the
modelpath to your local absolute directory. - Run
vllm-opt validate --config experiment.yamlto verify the environment. - Execute the smoke test with
vllm-opt run experiment.yamlto confirm the baseline measurements.
Connects securely to your tools. The creator never sees your data.
About this skill
Running vLLM in production without tuning leads to wasted VRAM and sluggish inference. This skill automates the benchmarking and configuration process using the vllm-optimizer package to find the ideal settings for your specific hardware and workload.
What it does
- Automated Benchmarking orchestrates
vllm-optto measure throughput and latency across different parameter sets. - YAML Generation creates validated experiment files for specific inference use cases like RAG or batch generation.
- Constraint Enforcement maps your SLOs to
optimization.constraintsto filter for configurations that actually meet latency targets. - Result Analysis interprets trial logs and performance reports to identify the best observed configuration.
- Environment Validation checks VRAM availability, port conflicts, and package versions before executing GPU-heavy tasks.
How it works
- Define Workload by specifying your model path, GPU resources, and traffic patterns like arrival rates or input/output lengths.
- Generate Experiment by creating a YAML config that defines fixed baselines and tunable search parameters.
- Validate Config using
vllm-opt validateto ensure the schema matches your installed version before starting runs. - Execute & Monitor a bounded smoke test followed by the full search, recording all trial data and hardware metrics.
- Review Report to see which settings maximized throughput while staying within your defined latency percentiles.
Frameworks & tools
Python, vLLM, vllm-optimizer, and benchmark backends like GuideLLM.
Why this beats prompting it yourself
Generic prompts cannot calculate search space budgets or validate YAML schemas against specific vLLM versions. This skill handles the boilerplate of benchmarking math and interprets raw trial data into actionable hardware settings based on real measurements.
Use cases
- Throughput Maximization for batch processing jobs where total tokens per second is the primary success metric.
- Latency Optimization for interactive chat applications requiring strict p99 response times.
- Hardware Migration to find the best
gpu_memory_utilizationandblock_sizewhen moving to new GPU instances. - RAG Tuning specifically for long-context workloads where KV cache management is critical.
Known limitations
Does not support automatic model acquisition or calibration. Requires existing local model files and manual authorization for GPU execution. Optimized for 0.1.0a21 schema.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for LLM Ops.
Requires local model files. Only supports optimization.maximize objectives. No automatic cloud GPU provisioning. Schema specific to vllm-optimizer 0.1.0a21.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 17 days ago
- Software
Python