docs: record fail-closed accuracy challenger

2026-07-15 05:42:17 +02:00
parent f82a7f456a
commit ecc9f77732
11 changed files with 218 additions and 22 deletions
@@ -8247,3 +8247,49 @@ Next:
 - Collect a new training-only small-building/background evidence pack outside
  all operational holdouts, then train an inactive candidate only if the pack
  passes label, leakage and sample-volume audits.
+
+## Sprint 199 - Reviewed accuracy expansion and fail-closed challenger (2026-07-15)
+
+Implemented:
+- Added six training-only reference AOIs for Arendonk, Dessel, Meerhout,
+  Laakdal, Nijlen and Hulshout. Tests enforce explicit training role, unique
+  municipality/center pairs and at least 2 km separation from every protected
+  Mol, Turnhout, Retie, Westerlo, Vosselaar and Grobbendonk holdout.
+- Prepared 1024 px orthophoto/GRB pairs on Tower. The paged GRB exports contain
+  9,964 features in total, use EPSG:31370 rasters and have no truncated,
+  invalid or empty reference geometry.
+- Exported `yolo-building-aoi1024-reviewedexp6-minpx3vis035`: 252 tiles, 234
+  positive, 18 negative, 234 train, 18 validation and 79,192 labels. The
+  dataset audit returned `ok`; the 64-tile visual review covered 28 retained
+  sources with no invalid, missing or low-variance selections.
+- Fine-tuned the active local YOLOv8s asset for 20 CPU epochs at image size
+  `640`. The best checkpoint came from epoch 16 and was copied as inactive
+  `geointel-building-yolov8s-reviewedexp6-minpx3-img640-ft20-pt`, SHA256
+  `038f1f97a6afd534f29e1f392a730a58207b928ca01e31ab8d8fed6106705820`.
+- Re-ran both active and challenger assets through the exact current API,
+  persistence and coverage-aware QA/QC path at tile `512`, overlap `64`,
+  confidence `0.15` and match IoU `0.25`. The evaluated reference populations
+  are equal per zone; older unequal-coverage runs were excluded.
+- The active model measured mean precision `0.6141`, recall `0.6062` and F1
+  `0.6069`; the challenger measured `0.6251`, `0.6293` and `0.6248`. Challenger
+  F1 improved in all seven zones and reduced false negatives from 3,416 to
+  3,208.
+- The challenger produced two detections in explicitly empty Postel-bos while
+  the active model remained at zero across Postel-bos, Lommel-heide and
+  Arendonk-heide. The formal promotion report recommended the existing active
+  key. No `.env`, active model path or runtime threshold was changed.
+- Updated the approved Detection Lab profile to the current coverage-aligned
+  active evidence and retained the explicit Postel limitation.
+
+Validation evidence:
+- Full readiness after the documentation/profile update passed 598 backend
+  tests, backend compilation, one Alembic head, frontend typecheck/build and
+  the complete shell syntax gate.
+- Persistent promotion evidence lives under
+  `/app/storage/operator-data/model-review/reviewed-accuracy-expansion/` and
+  `/app/storage/operator-evidence/mol-operational-validation/`.
+
+Next:
+- Convert the two confirmed Postel-bos challenger errors into a complete,
+  leakage-free hard-negative training sample, add independent empty controls,
+  then train a new inactive candidate through the same fail-closed gate.