Works with the AI Agents you already use
Srednoff Promptfoo AI Red-Team CI Review & Design
Deliver a version-bound Promptfoo security test plan, private-by-default CI design and release-decision matrix, without installing dependencies, generating live attacks or changing CI.
$7.99
Srednoff Promptfoo AI Red-Team CI Review & Design
Example session with this skill installed
Review our Promptfoo CI security gate for tool-call leakage and incomplete provider runs.
- Read your context and instructions
- Compiled the srednoff promptfoo ai
Expected report shape, not a completed real audit: Pinned Scope And Compatibility Manifest; Threat To Case To Assertion Matrix; Generation Grading Target And Data-Destination Map
Connects securely to your tools. The creator never sees your data.
About this skill
Srednoff Promptfoo AI Red-Team CI Review & Design
One-Line Outcome
Deliver a version-bound Promptfoo security test plan, private-by-default CI design and release-decision matrix, without installing dependencies, generating live attacks or changing CI.
Short Description
An average eval score can conceal unsafe tool calls, leaked data or a broken provider; a green job may mean the security suite never ran or tested a different revision.
Problem It Solves
An average eval score can conceal unsafe tool calls, leaked data or a broken provider; a green job may mean the security suite never ran or tested a different revision.
Security evaluation engineer: version-scoped Promptfoo red-team case/adapter design, risk-to-assertion mapping, failure taxonomy and CI threshold specification.
What You Get
- Pinned Scope And Compatibility Manifest
- Threat To Case To Assertion Matrix
- Generation Grading Target And Data-Destination Map
- Proposed Mock And Authorized Provider CI Lanes
- Critical Gate And Status Decision Policy
- Baseline Candidate Comparison With Denominators
- Findings With ID Severity Confidence Evidence Impact Recommendation Validation And Rollback/Risk
- Triage Retest Rollback And Artifact-Retention Plan
- Missing Evidence Unknowns And NOT_EXECUTED Steps
How It Works
- Freeze application, prompt, retrieval corpus, tool schemas, Promptfoo and provider-wrapper identities. Define review-only delivery and list any unverified installed-version assumptions.
- Build a threat-to-assertion matrix covering applicable prompt injection, jailbreak resistance, unsafe tool calls, data leakage, RAG poisoning and agent boundaries; explain exclusions rather than adding irrelevant categories.
- Separate deterministic unit tests for parser/auth/state logic from model-dependent security evals. Use inert tool recorders and synthetic confidential markers to inspect requested effects, not merely refusal wording.
- Split approved attack generation from regression evaluation. Review generated fixtures before freezing their hashes; do not run dynamic generation on an ordinary PR or assume generation and target evaluation have the same data recipients.
- Inventory target, generator, grader, telemetry, sharing and artifact destinations. Propose no uploads by default, scoped egress controls and version-checked privacy settings; a remote-generation opt-out is not a network firewall.
- Design a local/mock validation lane for untrusted changes without secrets. Treat config, provider wrappers, executable assertions and pull-request text as executable/untrusted inputs before any privileged CI lane.
- Define CI triggers and explicit check outcomes: relevant changes run the frozen suite, unrelated changes still yield the required status, and skipped/malformed/empty suites cannot produce a security pass.
- Specify pinned tool/action references, least-privilege job permissions, bounded cache keys, isolated runners and a separately authorized provider lane; no production credentials or deployment authority in eval jobs.
- Set deterministic assertions for tool events, schema and synthetic leakage first. Calibrate any LLM judge against supplied source facts and benign controls; keep grading instructions and expected values outside executor input.
- Define release policy by category: any confirmed critical boundary failure blocks; provider errors/timeouts are incomplete evidence, not passes; benign false positives and judge disagreement require triage, not silent threshold relaxation.
- Compare baseline/candidate with matched corpus, versions and budgets, reporting denominators, attempts, failed/unknown cases and confidence limits; changed generation requires an explicit new baseline.
- Deliver a proposed CI/config change plan, security matrix, evidence manifest and triage runbook. Implementation, paid provider calls, scans and branch-protection changes need separate approval; preserve existing gates on rollback.
Best For
AI security and release engineers designing reproducible Promptfoo security regression checks for agents and RAG applications
Example Tasks
- Review our Promptfoo CI security gate for tool-call leakage and incomplete provider runs.
- Design a frozen security corpus and private mock/provider lanes for agent and RAG changes.
- Define release decisions when critical injection cases fail but the average score improves.
- Audit the generation, grading and sharing data paths in this supplied Promptfoo configuration.
Inputs
- Pinned Promptfoo/config/provider-wrapper revisions, target application/model identity and tool schemas; installed-version evidence if available.
- Approved security scope, toy/staging target description, prohibited side effects, data classes and network/cost limits.
- Synthetic or redacted attack/benign corpus and application-specific safety policies with case ownership.
- Existing CI definition, sanitized eval result schema, branch-protection intent, secrets availability rules and artifact-retention policy.
Outputs
- Pinned Scope And Compatibility Manifest
- Threat To Case To Assertion Matrix
- Generation Grading Target And Data-Destination Map
- Proposed Mock And Authorized Provider CI Lanes
- Critical Gate And Status Decision Policy
- Baseline Candidate Comparison With Denominators
- Findings With ID Severity Confidence Evidence Impact Recommendation Validation And Rollback/Risk
- Triage Retest Rollback And Artifact-Retention Plan
- Missing Evidence Unknowns And NOT_EXECUTED Steps
Permissions
Read-only review of owner-supplied artifacts by default. Report file writes, shell/local fixture execution and official documentation lookup are optional, require available tools and their explicit scope. No live credentials are requested.
Compatibility
Standalone workflow, no Srednoff OS dependency. Codex evidence is synthetic entrypoint text only when recorded below. Claude Code, Cursor and actual buyer-runtime task delivery remain UNTESTED. Provider/platform-specific contracts must be refreshed for the installed version.
Limitations
This is an instruction/evidence workflow, not a hosted security service, persistent enforcement engine, production certification or business guarantee. The optional Python helper checks supplied claims, not underlying source artifacts. Agensi delivery is agent-only: do not save/export proprietary skill files to run local helpers; disclose NOT_EXECUTED where execution is unavailable.
Validation Transparency
Offline evidence-precheck status: PASS. Authored synthetic text-response cases: 18/18, status PASS. This is not independent security testing, complete artifact inspection or proof of deployment safety. Exact-text hashes and attempts are retained separately.
License And Publisher
Commercial License 1.0 covers original new expression; historical MIT and third-party rights remain unchanged. Mandatory marketplace conditions prevail. Publisher: Ivan Srednoff / Srednoff.Agency. No proprietary source resale or off-platform paid upsell is included.
FAQ
What is different from a generic security request? Security evaluation engineer: version-scoped Promptfoo red-team case/adapter design, risk-to-assertion mapping, failure taxonomy and CI threshold specification. No generic eval framework duplicate, installed CI, paid model runs, hosted scan or claim that probabilistic red-teaming proves deterministic prevention.
Does it change production automatically? No. Audit/design is the default; changes, secret rotation, IAM, network probes and financial actions need separate explicit authorization.
Are the examples real incidents? No. They are synthetic DEMO DATA.
Is this a certification or guaranteed protection? No. Evidence gaps, residual risk and unexecuted tests remain explicit.
Can I use project-specific reports for clients? Yes under the included license; proprietary workflow source is not a client deliverable.
Version
1.0.0
Release Notes
Initial scoped security workflow with domain decision rules, evidence requirements, synthetic examples, regression cases and explicit authorization/runtime limits. No production/customer outcome is claimed.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Security.
This is an instruction/evidence workflow, not a hosted security service, persistent enforcement engine, production certification or business guarantee. The optional Python helper checks supplied claims, not underlying source artifacts. Agensi delivery is agent-only: do not save/export proprietary skill files to run local helpers; disclose NOT_EXECUTED where execution is unavailable.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 1 day ago
Nothing to install. It runs in your AI agent.
- Optional
Python 3
No external account, API key or Srednoff OS installation is required. Supply authorized sanitized task evidence. Owner-authorized project inspection tools where available.