More screenshots

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIDeepSeekDeepSeekMuseMuse+22 more

    Qdrant Search Quality: Relevance Debugging

    1

    Fix Qdrant relevance the data-first way: a diagnosis ladder (chunking, model, query, config last), a golden set with recall@k, and significance tests so wins are proven.

    $5

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    Qdrant Search Quality: Relevance Debugging

    Qdrant Search Quality: Relevance Debugging

    Example session with this skill installed

    Since we re-indexed our support docs with smaller chunks last month, search feels worse — users say the right article exists but never shows up. Our first instinct was to tune HNSW. Diagnose it properly.

    • Read your context and instructions
    • Compiled the qdrant search quality

    DIAGNOSIS RUN — produced by applying the skill; HNSW explicitly not touched yet.

    Step 1, failing set: 14 real queries pulled from support tickets, each with the article that should have ranked. Failure type on 11 of 14: the right article is in the index but absent from the top 10 ("missing"), not buried at rank 6–10.

    Step 2, exact-match gate: searched a distinctive phrase from a known article ("resetting a stuck provisioning token"). Found instantly. Ingest is intact — the ladder proceeds; this is a ranking/chunking problem, not a missing-data problem.

    Step 3, rung 1 — chunking: read 20 random chunks from the re-indexed collection. Finding: the new splitter cut at 180 tokens with no overlap and no regard for sentence boundaries; 13 of 20 sampled chunks begin mid-sentence, and several procedure articles are split so the question ("how do I reset…") sits in one chunk and the steps in the next. Neither chunk alone matches a how-to query well. This matches the timeline exactly: quality dropped when chunking changed, not when anything about the model or Qdrant did.

    Measurement before fixing: golden set built from the failing set plus 60 curated query→article pairs (versioned v1). Baseline recall@10 with current chunking: 0.57. Single change tested: re-chunk at 350 tokens, sentence-aware splitter, 15% overlap, titles and section headers added as payload. Recall@10 on the same golden set: 0.79 (paired permutation test, 95% CI [0.74, 0.84], p < 0.01). Precision@5 also rose, so the win isn't recall bought with junk.

    Verdict: ship the re-chunking; the embedding model rung never needed testing, and the HNSW tuning the team wanted to start with would have multiplied a broken pipeline. Next candidate after this lands: hybrid fusion for the exact-term queries (error codes) still failing in the set.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Most "vector search is bad" complaints are not vector search problems — and the standard response, a week of HNSW tuning, fixes none of them. This skill walks a diagnosis ladder in the order problems actually occur: chunking first, then the embedding model, then the query side, and Qdrant configuration last, as the multiplier it is rather than the repair it isn't. It starts with the standing rule — reproduce before you tune — and a failing set of 10–20 real queries with the results that should have come back, because if you can't name the expected results you have an undefined problem, not a quality problem. An exact-match sanity check gates the whole ladder: if a known phrase can't be found, ingest is broken and tuning stops. Each rung names its specific checks — mid-sentence chunk splits that can cost 30–40% of retrieval quality, chunk size against question size, metadata poverty, duplicates; domain mismatch, quantization damage, mixed-model indexes after a swap, multilingual gaps; vague queries, filters-first, query embedding care — and only then ef, m, and quantization, tuned by sweeping one parameter against recall and p95 latency on a frozen snapshot. Evaluation gets its own discipline: a human-curated, versioned golden set of 50–200 query-to-document pairs, recall@k as the primary metric with precision/nDCG and MRR for their failure shapes, k chosen from what the product actually consumes, and a paired significance test with a confidence interval before any improvement is declared — a 3-point gain on 50 queries is noise, and the skill says so. Strategy upgrades come last and in effort order: hybrid keyword+dense with RRF fusion, cross-encoder reranking, MMR for diversity, relevance feedback. A symptom-to-rung decision table, a failing-set record format, golden-set rules, and a fix-report template are included, plus a worked example where a model swap dropped recall@10 from 0.74 to 0.58 and the fix that shipped (model B + hybrid, 0.83, p < 0.01) never touched the database config. Edge cases cover tiny corpora, multi-tenancy, drifting collections, and contaminated feedback loops.

    Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Agents & Orchestration.

    • The diagnosis needs expected results: someone who knows the corpus must help build the failing and golden sets. Without that, the skill stops at "undefined problem" — by design.
    • Worked numbers come from document-search workloads; code search and product catalogs have different chunking and metric profiles, though the ladder order holds.
    • Significance testing needs a golden set of reasonable size; with under ~30 pairs the skill will report directional results and say the evidence is thin.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before buying a skill.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 24 days ago

    Permissions

    No special permissions declared or detected

    Listed24 days ago
    Updated21 days ago

    What's inside

    Frequently Asked Questions

    Because config is a multiplier on the pipeline you have. If chunking is splitting procedures in half, higher ef retrieves the wrong chunks more thoroughly. The ladder exists because the observed base rates say the fault is usually in the data, the model, or the query — and the exact-match check plus a failing set tells you which within an hour.

    For most teams, 50 solid pairs is an afternoon for someone who knows the corpus, especially starting from real failing queries. The skill treats it as the highest-leverage step: without it, every subsequent change is guesswork, and "improved" can never be distinguished from "changed."

    That's the worked example's exact shape. Confirm ingest with the exact-match check, then measure old-vs-new recall@10 on the golden set, and check index homogeneity — a half-re-embedded collection is a silent quality killer. Only test replacement models once the drop is measured and attributed.

    New in Agents & Orchestration