Works with the AI Agents you already use
Near-Duplicate Text Detector
This skill measures overlap the same way every time and gives a clear verdict with a receipt.
$5
Near-Duplicate Text Detector
Example session with this skill installed
I asked you to write a fresh help-centre intro, but draft.txt reads a lot like our old one (orig.txt). How close is it? Also compare orig.txt with other.txt, which we wrote from scratch.
- Read your context and instructions
- Compiled the near-duplicate text detector
I ran near-duplicate-check with --k 3 (better for short paragraphs) on both pairs.
orig.txt vs draft.txt
{
"verdict": "NEAR-DUPLICATE",
"similarity": 0.796875,
"receipt": {
"algorithm": "kar-shingle-minhash-v1", "k": 3, "n": 64,
"docA": { "shingleCount": 54,
"normalizedSha256": "250815fb87514efaa1fbc5b61be39a41658d7bf5fe4511e011d4927056214ebe" },
"docB": { "shingleCount": 54,
"normalizedSha256": "b1fd5de07d15b128cfb781de7cb491e5af16a84ba317df2deed3a7350e81937a" },
"similarityMatches": 51, "similarityTotal": 64, "verdict": "NEAR-DUPLICATE"
},
"receiptSha256": "83a5ecdadbb23e1667df8444b5185f5f219974f78f289393c4d361aaa1889b46"
}
orig.txt vs other.txt (default k=5)
{
"verdict": "DISTINCT",
"similarity": 0,
"receipt": { "algorithm": "kar-shingle-minhash-v1", "k": 5, "n": 64,
"similarityMatches": 0, "similarityTotal": 64, "verdict": "DISTINCT" },
"receiptSha256": "56ef6508ddf6e564bf40a437b03fa02b7844117df8dc174a0a93ea0d9657fe92"
}
Summary: draft.txt is a near-duplicate of the old intro (about 80% estimated similarity). Only two words were swapped ("Start" became "Begin", "turn on" became "switch on"). I'll rewrite it properly. other.txt shares no 5-word runs with the original, so it is genuinely new.
Connects securely to your tools. The creator never sees your data.
About this skill
Agent Trust: check how close a draft is to its source
Agents that summarise or rewrite can stay too close to the original, and content teams recycle the same paragraphs. This skill measures overlap the same way every time and gives a clear verdict with a receipt.
What it does
- Normalises both texts and breaks them into overlapping word shingles (5 words by default; adjustable with
--k) - Builds a deterministic MinHash signature (64 hash functions by default;
--n) - Returns a similarity ratio and a verdict:
- IDENTICAL: the normalised texts are the same
- NEAR-DUPLICATE: similarity 0.75 or higher
- OVERLAPPING: 0.35 to 0.75
- DISTINCT: below 0.35
When an agent should use it
- Checking whether a draft or AI output stays too close to a source
- De-duplicating tickets, docs, FAQs or product descriptions
- Comparing two versions of a text to see how much really changed
What it outputs
JSON: verdict, similarity ratio, and a receipt with shingle counts, hashes of each normalised text and signature, match counts, and a receipt SHA-256.
Limitations
- It measures word overlap, not meaning. A full rewrite with the same ideas will score as DISTINCT.
- Shingle size matters. With the default of 5 words, a few changed words in a short paragraph can drop a near-copy to OVERLAPPING; use
--k 3for short texts. - MinHash is an estimate; with 64 hashes the ratio moves in steps of 1/64.
- Not a plagiarism database: it only compares the two texts you give it.
- The exit code is 0 for every verdict, so agents and scripts should read
verdictrather than the exit code.
Open source, packaged skill
Original work by Simon Gant. The core logic is public under the MIT licence (kar-shingle). This listing is the packaged, tested agent skill: SKILL.md workflow, CLI, receipt output and docs. Requires Node.js 18+, zero dependencies, no network, no API keys.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse All Claude Skills.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed today
- Software
Node.js 18+
Permissions
Runs one local Node.js script (`node scripts/near-duplicate-check.js`) through the agent's shell tool. Reads only the two text files you pass it. No network access, no file writes, no environment variables read. Pure compute.