Shift Left

/shift-left create plans

I have enough to assess and ask the scope question. Here's what I found.

Quality Maturity Assessment

Current Level: 3 -- Workflow-Centric

Evidence:

• Three test tiers in place: tests/unit/, tests/integration/, tests/functional/
• pytest + pytest-django + pytest-xdist; coverage.py uploaded to Codecov
• GitHub Actions matrix: 5 Python versions x 3 Django versions, PostgreSQL service
• Key e-commerce workflows covered: payment (bankcard, models), checkout (calculators, session, surcharges, views), order (creator, event_handler, processing, reports), basket, offer, customer, dashboard, partner, shipping
• Python linting (flake8, isort, pylint, black) and JS linting (eslint) in CI
• Functional tests use django-webtest (HTTP-level, not browser)

Strengths:

• Unusually thorough integration test coverage for a library -- most e-commerce modules have meaningful tests at the right layer
• CI version matrix is a genuine strength; regressions across Django/Python versions are caught before release
• Codecov tracks coverage at patch level

Gaps:

• CodeQL runs on a weekly schedule only -- not gated on PRs; a vulnerability introduced in payment or checkout code can merge before the next scan
• No dependency vulnerability scanning (no pip-audit, Safety, or Bandit)
• No browser-based E2E (functional tests use WebTest -- no JS execution, no real browser rendering)
• No accessibility testing despite a customer-facing storefront surface in the sandbox
• No coverage threshold enforced; CI never blocks on coverage regression Archetype: Django Oscar is a Library/SDK with a customer-facing storefront + admin dashboard delivered via a sandbox. Risk is concentrated in the e-commerce domain -- payment, checkout, offer calculation, order processing, and the customer auth backend.

Likely relevant plans:

Plan Rationale Quality risk strategy Always Unit Existing suite; strengthen offer/voucher calculation and checkout pipeline edge cases Integration Existing suite; strengthen payment seams and order event handling Security High priority -- CodeQL weekly gap, no SAST, payment + auth + admin E2E Sandbox has real customer + admin UI; WebTest covers logic but not JS or real browser rendering Accessibility Lightweight -- customer-facing storefront; no current tooling Not recommended: Performance (no runtime load concern for a library), Contract (no HTTP API surface; Oscar's extension points are covered by integration tests and semantic versioning).

Would you like:

All relevant plans -- quality risk strategy + unit + integration + security + E2E + accessibility (recommended) Only the umbrella quality risk strategy A custom subset

All plans written and dashboard generated. Here's the summary:

Do first (in this order):

CodeQL on PRs -- 3-line YAML edit to .github/workflows/codeql-analysis.yml. Closes the biggest gap immediately; a vulnerability introduced in checkout or auth code currently survives until the next weekly scan. pip-audit in CI -- run it locally first to get a clean baseline, then add a security job to test.yml. ~15 lines. Offer/voucher boundary unit tests -- benefit cap when discount exceeds line total, per-customer voucher reuse, exclusive voucher blocking. Order event handler failure test -- verify test_event_handler.py covers the dispatch-failure path explicitly. Defer for now:

Playwright E2E -- valuable, but start it after security CI is stable Accessibility -- add axe-playwright alongside Playwright once that suite is running Performance and contract -- not applicable at the library layer Deliverables written to django-oscar/quality/:

File 1-quality-risk-strategy.md Risk table, Mermaid diagrams, coverage summary, top 10 test cases 1-quality-risk-strategy-dashboard.html Stakeholder-ready visual 2-unit-test-plan.md Offer/voucher/checkout priority cases 3-integration-test-plan.md Checkout seam, payment, order event handler 4-security-test-plan.md CodeQL gate + pip-audit -- exact YAML included 5-e2e-test-plan.md 5 Playwright journeys against the sandbox 6-accessibility-test-plan.md axe-playwright on 4 key pages, keyboard baseline README.md Entry point with plan table and do-first summary