Jens/geointel
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Expand golden QA benchmark scenarios
GeoIntel CI / docs-smoke (push) Has been cancelled
Details
GeoIntel CI / contract-smoke (push) Has been cancelled
Details

Browse Source
This commit is contained in:
Codex
2026-06-18 21:46:19 +02:00
parent 5128b9ced0
commit c85953f389
14 changed files with 417 additions and 28 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/CODEX_EXECUTION_LOG.md
+23
View File
@@ -2384,3 +2384,26 @@ Limitations:
Next recommended pass:
- Expand golden datasets beyond the current building QA fixtures.
## Sprint 61 Golden QA scenario expansion (2026-06-18)
Changed:
- Added `fixtures/golden/golden_qa_benchmarks.json` as the explicit scenario manifest for deterministic QA/QC regression coverage.
- Added local golden fixture pairs for perfect-match, no-overlap and MultiPolygon building comparisons.
- Updated `scripts/run_golden_qa_benchmark.py` to execute all manifest scenarios, assert expected metric drift, persist simulated `QualityCheck`/`Metric` rows for each scenario and report aggregate persistence totals.
- Preserved backward-compatible top-level benchmark fields for scripts that still read the original single-scenario output shape.
- Updated `scripts/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
Tested:
- Red step: `cd backend && python -m pytest tests/test_sprint12_golden_qa_benchmark.py -q` failed on missing multi-scenario manifest/output.
- `cd backend && python -m pytest tests/test_sprint12_golden_qa_benchmark.py -q` (`4 passed`)
- `python scripts/run_golden_qa_benchmark.py --json`
Open:
- Run full readiness and deploy Tower after the expanded golden benchmark passes the release gate.
Limitations:
- These are deterministic local QA/QC fixtures only. They do not introduce new production QA metrics or provider data.
Next recommended pass:
- Continue with frontend visual polish backlog or add a live golden benchmark smoke only if a running PostGIS environment needs that extra release signal.