Test-seed harness: atomic manifest+DB seeding, random-by-default, predicate fixtures

Context

The current test-seed harness has three architectural defects that surfaced as a flaky-test class during the 2026-05-12 session.

Failure observed: Playwright specs caps.spec.ts >> authorization tab renders provider row and wic.spec.ts >> nutritional risk tab renders assessment row returned "No assessments on file" / empty panels despite both `caps_authorizations` (33 rows) and `wic_nutritional_risk_assessments` (138 rows) existing in the database.

Root cause: the Playwright TypeScript manifest at `tests/e2e/lib/seed.ts` references specific UUIDs (e.g., `wicDet0.personId = '018cc251-f417-7e0f-a546-5bd3de8aa969'`) that did not exist in the DB because the manifest and the DB had been seeded by separate `canopy-seed` invocations with different `--households` arguments. The manifest was regenerated by `xtask e2e` with `--households 50`; the DB was loaded by `xtask seed` with its default `--households 9`. Even at household-index 0, RNG iteration order causes nested-row UUIDs (persons, assessments) to drift between household-count variants — so the manifest's per-row IDs landed outside what the DB had.

Architectural defects

1. Manifest derivation isn't atomic with DB seeding

Today, `xtask e2e` writes the manifest (`tests/e2e/lib/seed.ts`) AND the SQL files (`test-results/seed/canopy_*.sql`) but does NOT apply the SQL. The user has to run `xtask seed` separately, which calls `canopy-seed` a second time with potentially-different arguments. Two invocations = two opportunities to drift.

2. Seeds are fixed-by-default, hiding fragility

Both `xtask e2e` and `xtask seed` default to `--seed 42` and a hardcoded `--households`. Deterministic-by-default means real flakes from environmental drift (DB state, sibling-test contamination, manifest mismatches) read as deterministic test failures, masking the seed-coupling defect.

3. Playwright fixtures index by UUID, not by predicate

`tests/e2e/specs/wic.spec.ts:14` does `page.goto(`/cases/${wic!.householdId}?program=wic`)` where `wic` is `seed.wicDeterminations.wicDet0`. Coupling the test to a specific named fixture's UUID makes any manifest/DB drift fatal. The test's actual intent is "any approved WIC determination" — that should be expressible as a query helper, not a positional fixture reference.

Proposed architecture

Layer 1: single source-of-truth seed command

``` xtask test-seed [--seed N] [--households N] [--jurisdiction georgia] ├─ canopy-seed generates SQL files + writes manifest with seed embedded ├─ Applies SQL to DB via psql (atomic step with the manifest write) └─ Writes test-results/seed/last.txt with the resolved seed value ```

`xtask e2e` becomes:

``` xtask e2e [--seed N] ├─ Calls xtask test-seed --seed $RESOLVED (one source of truth) └─ Runs Playwright against the freshly-loaded DB ```

The existing `xtask seed` either becomes an alias for `xtask test-seed` or gets renamed to remove the historical confusion. Either way, `--households` and `--seed` defaults are defined exactly once.

Layer 2: random seed by default, captured for replay

The default seed is `std::time::SystemTime::now()` (or `rand::random()`). The resolved seed is:

  1. Embedded as a top-of-file constant in the generated manifest: ```ts /** Generated by canopy-seed with seed=1234567 */ export const SEED = 1234567; ```
  2. Written to `test-results/seed/last.txt` (gitignored).
  3. Echoed by the command on completion: `Seed: 1234567 (replay with --seed 1234567).`

Failure → grep the manifest or `last.txt` → `xtask e2e --seed 1234567` for byte-for-byte reproduction.

Layer 3: predicate-based Playwright fixtures (bigger structural fix)

Replace UUID-positional fixture access (`seed.wicDeterminations.wicDet0.personId`) with predicate queries against the live DB:

```ts const wic = await pickFirstApprovedDetermination('wic'); // queries canopy-wic API const assessment = await pickAssessmentFor(wic.personId); ```

The manifest becomes "what name-keyed scenarios are available" (e.g., `fixtures.has('approved-wic-with-assessment')`) rather than "what specific UUIDs are at index 0." Manifest/DB drift becomes architecturally impossible because the test never references manifest-internal positional structure.

Acceptance criteria

  • `xtask seed` and `xtask e2e` have exactly one place that calls `canopy-seed`; running either + the other in any order produces identical DB state.
  • Default seed is random; resolved seed is captured to `test-results/seed/last.txt` and stamped on `seed.ts`.
  • Documentation in `.claude/docs/testing.md` describes the `--seed` replay flow.
  • (Stretch) Playwright fixtures migrated to predicate-query helpers; UUID-positional access removed from `specs/`.

Rationale for filing

This is the architectural cause of the entire class of flakes I observed during the 2026-05-12 session (#392 (closed) / #448 (closed) push cycle), where reseeding by hand became a recurring workaround. Without the fix, every devstack reset is a hazard.

Cross-refs

  • Symptom-triggers from session: !265 (merged) push retries blocked on these specs.
  • Related: `feedback_glab_auto_merge` memory notes the bypass pattern used to land MRs despite these flakes.