feat: close measured model review evidence
This commit is contained in:
@@ -8192,3 +8192,40 @@ Next:
|
||||
- Deploy the migration and UI, verify one live review queue and map QA result,
|
||||
then complete representative manual decisions before constructing any new
|
||||
model-training corpus.
|
||||
|
||||
## Sprint 198 - Evidence-closed model review (2026-07-15)
|
||||
|
||||
Implemented:
|
||||
- Generated the previously missing 48-card false-negative contact-sheet bundle
|
||||
from the exact persisted inference manifests for Geel, Herentals and
|
||||
Turnhout; 731 references outside tile coverage remained explicitly excluded.
|
||||
- Completed all 96 FP/FN decisions through orthophoto inspection plus
|
||||
persisted geometry-overlap diagnostics. The result is 5 confirmed model FP,
|
||||
10 confirmed model FN, 59 QA-alignment cases, 10 reference-gap/change cases
|
||||
and 12 uncertain/obscured cases.
|
||||
- Added `validate_detection_false_negative_review_decisions.py`, mirroring the
|
||||
existing FP safety contract and exporting only explicit confirmed misses.
|
||||
- Audited confirmed evidence against the active training split. Geel and
|
||||
Herentals are existing training sources; Turnhout is an excluded operational
|
||||
holdout. The review therefore provides zero novel leakage-free labels and a
|
||||
new fine-tuning run was deliberately rejected.
|
||||
- Added strict match count plus the existing diagnostic reference-envelope
|
||||
result to the map analysis panel. Canonical footprint IoU metrics remain
|
||||
unchanged.
|
||||
|
||||
Validation:
|
||||
- Both 48-row decision CSVs completed with zero `unreviewed` records and passed
|
||||
their validators with `--require-complete`.
|
||||
- Focused validator/readiness/map-contract tests passed.
|
||||
- Full readiness passed 596 backend tests, backend compilation, 88 documented
|
||||
API routes, one Alembic head, frontend typecheck/build and shell syntax gates.
|
||||
|
||||
Known limitation:
|
||||
- The active local model remains useful but imperfect. A new candidate requires
|
||||
independently collected training-only AOIs and complete tile labels; holdout
|
||||
review evidence must not be recycled into training.
|
||||
|
||||
Next:
|
||||
- Collect a new training-only small-building/background evidence pack outside
|
||||
all operational holdouts, then train an inactive candidate only if the pack
|
||||
passes label, leakage and sample-volume audits.
|
||||
|
||||
Reference in New Issue
Block a user