Expand golden QA benchmark scenarios
This commit is contained in:
@@ -2384,3 +2384,26 @@ Limitations:
|
||||
|
||||
Next recommended pass:
|
||||
- Expand golden datasets beyond the current building QA fixtures.
|
||||
|
||||
## Sprint 61 Golden QA scenario expansion (2026-06-18)
|
||||
|
||||
Changed:
|
||||
- Added `fixtures/golden/golden_qa_benchmarks.json` as the explicit scenario manifest for deterministic QA/QC regression coverage.
|
||||
- Added local golden fixture pairs for perfect-match, no-overlap and MultiPolygon building comparisons.
|
||||
- Updated `scripts/run_golden_qa_benchmark.py` to execute all manifest scenarios, assert expected metric drift, persist simulated `QualityCheck`/`Metric` rows for each scenario and report aggregate persistence totals.
|
||||
- Preserved backward-compatible top-level benchmark fields for scripts that still read the original single-scenario output shape.
|
||||
- Updated `scripts/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
||||
|
||||
Tested:
|
||||
- Red step: `cd backend && python -m pytest tests/test_sprint12_golden_qa_benchmark.py -q` failed on missing multi-scenario manifest/output.
|
||||
- `cd backend && python -m pytest tests/test_sprint12_golden_qa_benchmark.py -q` (`4 passed`)
|
||||
- `python scripts/run_golden_qa_benchmark.py --json`
|
||||
|
||||
Open:
|
||||
- Run full readiness and deploy Tower after the expanded golden benchmark passes the release gate.
|
||||
|
||||
Limitations:
|
||||
- These are deterministic local QA/QC fixtures only. They do not introduce new production QA metrics or provider data.
|
||||
|
||||
Next recommended pass:
|
||||
- Continue with frontend visual polish backlog or add a live golden benchmark smoke only if a running PostGIS environment needs that extra release signal.
|
||||
|
||||
+3
-2
@@ -59,7 +59,8 @@ This file now starts with the current implementation status. Older preparation/b
|
||||
- [x] Demo workflow orchestration hook decomposition.
|
||||
- [x] Final `App.tsx` import/encoding cleanup and size audit.
|
||||
- [x] Optional final bootstrap-effect extraction.
|
||||
- [ ] Decide next V1 stabilization focus: golden dataset expansion or frontend visual polish backlog.
|
||||
- [x] Decide next V1 stabilization focus: golden dataset expansion or frontend visual polish backlog.
|
||||
- [x] Expand golden QA/QC benchmark coverage across partial, perfect, no-overlap and MultiPolygon scenarios.
|
||||
|
||||
## Sprint 8 status
|
||||
|
||||
@@ -330,4 +331,4 @@ This file now starts with the current implementation status. Older preparation/b
|
||||
- [x] Add a live dry-run maintenance smoke for demo export cleanup.
|
||||
- [x] Add browser screenshot artifact automation for visual regression handoff.
|
||||
- [x] Add backend error-envelope audit for expected user-error paths.
|
||||
- [ ] Expand golden datasets beyond building QA fixtures.
|
||||
- [x] Expand golden datasets beyond the original single building QA fixture pair.
|
||||
|
||||
Reference in New Issue
Block a user