qikly

Developers often struggle with test suites that pass consistently but fail to catch real bugs. This happens when tests are written by reading the implementation, essentially codifying existing behavior rather than verifying a contract. This skill implements a specification-driven testing workflow for Python modules, ensuring tests are honest and capable of failing.

What it does

• Specification splitting divides requirements from acceptance criteria so the coding agent never sees the "answer key."
• Automated scaffolding reads Python function signatures to generate task files and test structures.
• Mutation scoring plants faults in existing code to measure how many a suite actually catches.
• Validation checks identify unreachable criteria, ambiguous requirements, and missing data fixtures before you spend tokens.
• Refinement loops iterate on code patches until the implementation satisfies the withheld criteria.

How it works

1. Scaffold the project using qikly --scaffold to generate a task file from your Python signatures.
2. Define requirements (decisions) and acceptance criteria (consequences) in the task file, providing sample data inputs.
3. Validate the setup for free using --validate to catch logic errors or unreachable code paths.
4. Execute the testing run where the skill coordinates a coding agent and a testing agent in isolation.

Frameworks & tools

This tool is built specifically for Python environments and integrates with pytest. It supports models from Anthropic, OpenAI, and Google Gemini via API keys.

Why this beats prompting it yourself

Writing tests via standard prompting usually results in "consistent" code where the agent justifies its own bugs. This skill enforces a structural wall between the spec and the implementation that simple prompting cannot maintain, ensuring tests actually guard against regressions.

Use cases

• Auditing a legacy test suite to see if it actually catches logical faults.
• Writing test-driven code from a spec without letting the coder see the edge-case answers.
• Resolving ambiguities in financial or mathematical rounding rules before they reach production.
• Generating comprehensive test fixtures for complex boundary conditions.

Known limitations

Supports Python modules only. Requires qikly version 0.5.4 or later. Convergence varies by model and is not a guarantee of correctness: a run can finish without converging, and the report says so when it does.