Playwright E2E Production Pack

The agent loads playwright-e2e-production, then playwright-locators, and runs the linter:

$ node lint-playwright-specs.mjs orders.spec.ts
WARN  page-selector-action orders.spec.ts:5 Selector-based page method. Playwright marks these discouraged; use a locator: page.getByRole(...).click().
WARN  page-selector-action orders.spec.ts:6 Selector-based page method. Playwright marks these discouraged; use a locator: page.getByRole(...).click().
WARN  page-selector-action orders.spec.ts:7 Selector-based page method. Playwright marks these discouraged; use a locator: page.getByRole(...).click().
ERROR wait-for-timeout orders.spec.ts:8 Fixed sleep. Wait for the condition instead: a web-first assertion, page.waitForResponse() or page.waitForURL().
ERROR one-shot-assertion orders.spec.ts:12 Value read once, then asserted. Use the retrying matcher (toBeVisible, toHaveText, toHaveCount, toHaveValue, toHaveAttribute) or expect.poll().
WARN  positional-action orders.spec.ts:16 Acting on first(), last() or nth() usually silences a strict mode violation and clicks whichever match renders first. Filter by text, a child locator or within().
ERROR unawaited-assertion orders.spec.ts:17 Async assertion without await. It never fails the test. Add await.
WARN  structural-css orders.spec.ts:17 Structural CSS (nth-child or deep > chains). Breaks on layout changes; filter by content instead.
WARN  page-selector-action orders.spec.ts:22 Selector-based page method. Playwright marks these discouraged; use a locator: page.getByRole(...).click().
3 error(s), 6 warning(s), 0 note(s)

Then it loads playwright-flake-triage, reproduces CI conditions (retries: 2, trace: 'on-first-retry', an orders API whose latency varies up to 1.5 s like the shared staging backend) with --repeat-each=10, and triages the JSON report:

$ node triage-report.mjs results.json
1 report(s), 3 test(s): 2 passed, 1 flaky, 0 failed, 0 skipped

FLAKY  [chromium] orders.spec.ts:11 lists three orders
       attempts: failed>failed>passed | passed | passed | passed | failed>passed | passed | failed>failed>passed | passed | failed>passed | passed
       cause: value-assertion (Error: expect(received).toBe(expected) // Object.is equality)
       fix: A plain value assertion. If it is flaky, the value was read once while the UI was still changing: use a web-first matcher or expect.poll().
       trace: npx playwright show-trace test-results\orders-lists-three-orders-chromium-retry1\trace.zip

By cause: value-assertion 1

It explains the three problems. The count is read once, one second after login, from a table that renders when the API answers; a slow answer fails the test. The cancel test never asserts anything: the expect is not awaited. Worse, .first() clicks the Cancel button of order A-1001, which is paid, not pending; even with await added the test passes, because it checks the first row too. And the theme test changes a setting on the shared account that other tests read.

It moves the login into a setup project (from playwright-auth-state) and rewrites the spec:

import { test, expect } from '@playwright/test';

test.beforeEach(async ({ page }) => {
  await page.goto('/dashboard');
});

test('lists three orders', async ({ page }) => {
  const orders = page.getByRole('table', { name: 'Orders' });
  await expect(orders.getByRole('row').filter({ has: page.getByRole('cell') })).toHaveCount(3);
});

test('cancels the pending order', async ({ page }) => {
  const row = page.getByRole('row').filter({ has: page.getByRole('cell', { name: 'A-1002', exact: true }) });
  await expect(row.getByRole('cell').nth(1)).toHaveText('pending');
  await row.getByRole('button', { name: 'Cancel' }).click();
  await expect(row.getByRole('cell').nth(1)).toHaveText('cancelled');
});

test('saves the dark theme', { lock: 'user-settings' }, async ({ page }) => {
  await page.getByRole('button', { name: 'Settings' }).click();
  const dialog = page.getByRole('dialog', { name: 'Settings' });
  await dialog.getByLabel('Theme').selectOption('dark');
  await dialog.getByRole('button', { name: 'Save' }).click();
  await expect(page.getByRole('status')).toHaveText('Settings saved');
});

The row is found by its order id, not its status: filtering on 'pending' would stop matching the moment the status changes, and the final assertion would look for a row that no longer exists. The header row has column headers, not cells, so the count filter skips it. The lock (Playwright 1.63+) keeps every test that touches the theme from running at the same time.

$ node lint-playwright-specs.mjs orders.spec.ts
0 error(s), 0 warning(s), 0 note(s)

$ npx playwright test --repeat-each=20
  62 passed (16.3s)

$ node triage-report.mjs results.json --fail-on-flaky
1 report(s), 5 test(s): 5 passed, 0 flaky, 0 failed, 0 skipped

Both specs, the outputs above (trimmed of the echoed source lines and extra trace paths) and the 20x rerun under the same latency are real runs on @playwright/test 1.64.0 with Chromium. Finally it adds the linter and the triage step to the CI workflow from playwright-ci-sharding.