feat: add coverage-aware Mol benchmark
This commit is contained in:
@@ -7659,3 +7659,42 @@ Open:
|
||||
from envelope diagnostics. The next safe model step is an evidence-led
|
||||
footprint-label/matching review, followed by a fresh bounded Mol multi-zone
|
||||
benchmark before any activation or retraining decision.
|
||||
|
||||
# Sprint 185 - Coverage-aware Mol operational benchmark
|
||||
|
||||
## Implementation
|
||||
|
||||
- Extended the existing real-data quality matrix summaries with the exact
|
||||
persisted tile-coverage population, raw/evaluated/excluded/clipped candidate
|
||||
and reference counts, CRS/tile provenance and diagnostic-only
|
||||
box-to-footprint match gap.
|
||||
- Preserved manifest-backed Mol municipality, operational-zone, validation
|
||||
split, WGS84 bounds and source reference counts in the multi-sample summary.
|
||||
- Added `build_mol_operational_benchmark_report.py`. It groups evidence by the
|
||||
exact model asset, tile size, overlap and confidence threshold and refuses to
|
||||
blend different candidate configurations.
|
||||
- Added explicit gates for four independent positive Mol holdouts, one
|
||||
pure-empty background control, complete coverage provenance, minimum 95%
|
||||
reference coverage, mean F1 `0.25`, minimum per-zone F1 `0.10` and zero
|
||||
background detections. Envelope matches remain diagnostic and cannot satisfy
|
||||
the canonical F1 gates.
|
||||
- Wired the report into the persistent Mol operator runner, readiness gate and
|
||||
all-in-one image. The report does not activate, replace, download or train a
|
||||
model and does not change APIs, migrations or inference behavior.
|
||||
|
||||
## Local validation
|
||||
|
||||
- Focused Sprint 126/127/178/184/185 regression set passed: `14` tests.
|
||||
- Full readiness passed with `512` backend tests, one Alembic head
|
||||
`202606120900`, 81 audited API operations, frontend typecheck and Vite 7.3.6
|
||||
production build.
|
||||
- `npm audit --audit-level=high` reported zero vulnerabilities.
|
||||
- Shell syntax checks passed for the single-, multi-sample and Mol operational
|
||||
runners. Fixture tests prove both an accepted four-zone result and explicit
|
||||
rejection for missing coverage, one-zone collapse and background pressure.
|
||||
|
||||
## Next pass
|
||||
|
||||
- Deploy the operator tooling to Tower and run the current active local model
|
||||
over Achterbos, Gompel, Donk, Postel and the Postel-bos pure-empty control.
|
||||
Record the coverage-aware operational decision before changing model state.
|
||||
|
||||
@@ -27,6 +27,8 @@ This file now starts with the current implementation status. Older preparation/b
|
||||
- [x] Persist combined Mol operator evidence under the Unraid storage mount so reports survive all-in-one container replacement.
|
||||
- [x] Visually review Mol Postel and Donk false-positive/false-negative evidence, classify the dominant error modes and only then decide whether another model-training pass is justified.
|
||||
- [x] Clip detection QA populations to persisted raster/tile coverage and add box-to-footprint matching diagnostics before reconsidering model training.
|
||||
- [x] Add a coverage-aware Mol multi-zone benchmark report with explicit positive-zone, per-zone collapse, reference-coverage and pure-empty background gates.
|
||||
- [ ] Execute the refreshed coverage-aware Mol operational benchmark against the active local model and record the resulting retain/review decision.
|
||||
- [x] Backend FastAPI foundation, health endpoint and service structure.
|
||||
- [x] React/TypeScript frontend foundation and MapLibre workbench.
|
||||
- [x] Map layer visibility, opacity and feature property inspection.
|
||||
|
||||
Reference in New Issue
Block a user