Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIDeepSeekDeepSeekMuseMuse+22 more

    vllm optimizer

    1

    Optimize vLLM serving performance with automated benchmarking, YAML generation, and performance constraint analysis.

    $9

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    vllm optimizer

    vllm optimizer

    Example session with this skill installed

    Create a vllm-optimizer YAML for a Llama-3-8B model on a single A100. I need to stay under 50ms TTFT for an interactive chat workload.

    • Read your context and instructions
    • Compiled the vllm optimizer

    I have generated a validated experiment configuration for Llama-3-8B. This setup targets your 50ms TTFT constraint by testing variations in the KV cache allocation and max tokens while keeping the serving engine on loopback.

    vLLM Optimization Configuration

    project: llama3-chat-optimization
    model: /path/to/models/Llama-3-8B
    workload:
      backend: vllm
      traffic:
        arrival_rate: 5.0
        input_len: 512
        output_len: 128
      route: /v1/chat/completions
    
    optimization:
      maximize: throughput
      constraints:
        - field: ttft_p95
          operator: lt
          value: 0.050
      
    search:
      fixed:
        gpu_memory_utilization: 0.90
        enforce_eager: false
      tunable:
        max_num_seqs: [16, 32, 64]
        block_size: [16, 32]
    
    benchmarks:
      - name: interactive_chat_smoke
        samples: 100
        warmup: 10
    

    Next steps

    1. Update the model path to your local absolute directory.
    2. Run vllm-opt validate --config experiment.yaml to verify the environment.
    3. Execute the smoke test with vllm-opt run experiment.yaml to confirm the baseline measurements.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Running vLLM in production without tuning leads to wasted VRAM and sluggish inference. This skill automates the benchmarking and configuration process using the vllm-optimizer package to find the ideal settings for your specific hardware and workload.

    What it does

    • Automated Benchmarking orchestrates vllm-opt to measure throughput and latency across different parameter sets.
    • YAML Generation creates validated experiment files for specific inference use cases like RAG or batch generation.
    • Constraint Enforcement maps your SLOs to optimization.constraints to filter for configurations that actually meet latency targets.
    • Result Analysis interprets trial logs and performance reports to identify the best observed configuration.
    • Environment Validation checks VRAM availability, port conflicts, and package versions before executing GPU-heavy tasks.

    How it works

    1. Define Workload by specifying your model path, GPU resources, and traffic patterns like arrival rates or input/output lengths.
    2. Generate Experiment by creating a YAML config that defines fixed baselines and tunable search parameters.
    3. Validate Config using vllm-opt validate to ensure the schema matches your installed version before starting runs.
    4. Execute & Monitor a bounded smoke test followed by the full search, recording all trial data and hardware metrics.
    5. Review Report to see which settings maximized throughput while staying within your defined latency percentiles.

    Frameworks & tools

    Python, vLLM, vllm-optimizer, and benchmark backends like GuideLLM.

    Why this beats prompting it yourself

    Generic prompts cannot calculate search space budgets or validate YAML schemas against specific vLLM versions. This skill handles the boilerplate of benchmarking math and interprets raw trial data into actionable hardware settings based on real measurements.

    Use cases

    • Throughput Maximization for batch processing jobs where total tokens per second is the primary success metric.
    • Latency Optimization for interactive chat applications requiring strict p99 response times.
    • Hardware Migration to find the best gpu_memory_utilization and block_size when moving to new GPU instances.
    • RAG Tuning specifically for long-context workloads where KV cache management is critical.

    Known limitations

    Does not support automatic model acquisition or calibration. Requires existing local model files and manual authorization for GPU execution. Optimized for 0.1.0a21 schema.

    Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for LLM Ops.

    Requires local model files. Only supports optimization.maximize objectives. No automatic cloud GPU provisioning. Schema specific to vllm-optimizer 0.1.0a21.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before buying a skill.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 17 days ago

    Software
    Python

    Permissions

    Read Files
    Write Files
    Terminal / Shell
    Listed17 days ago

    What's inside

    Frequently Asked Questions

    The skill uses your specific SLOs (like p99 latency) to run automated benchmarks using the `vllm-optimizer` package. It then generates a validated YAML configuration that balances memory utilization and throughput specifically for your hardware.

    This skill is currently optimized for vllm-optimizer schema version 0.1.0a21. It includes environment validation steps to ensure your local package versions and VRAM availability are compatible before executing any GPU-heavy tasks.

    No, this skill focuses on the optimization and benchmarking of existing models. You must have your model files already downloaded and provide the local path or Hugging Face ID for the tool to begin the benchmarking process.

    You will receive the full AI agent skill package, including the logic for workload definition, experiment generation, and automated result analysis. It includes pre-built templates for common use cases like RAG and batch generation.

    Yes, the skill allows you to define specific constraints for latency percentiles. It will filter all benchmarking results to ensure the final recommended configuration strictly adheres to your required response times for interactive applications.

    New in LLM Ops