docs: record fail-closed accuracy challenger

docs/AI_PIPELINES.md

@@ -421,9 +421,10 @@ false positives block default promotion.
 The focused small-building local model asset,
 `geointel-building-yolov8s-smallbld-minpx3-img640-ft30-pt`, is the current
 recommended Detection Lab operator profile. Use tile size `512`, overlap `64`
-and confidence threshold `0.15`. Persisted QA/QC at match IoU `0.25` across
-seven positive AOIs measured mean precision `0.5898`, recall `0.5770` and F1
-`0.5825`; minimum per-AOI F1 was `0.5528`. The strict three-sample pure-empty
+and confidence threshold `0.15`. Its original promotion evidence at match IoU
+`0.25` across seven positive AOIs measured mean precision `0.5898`, recall
+`0.5770` and F1 `0.5825`; minimum per-AOI F1 was `0.5528`. The strict
+three-sample pure-empty
 background gate produced zero detections. Compared with the previous balanced
 profile, the same persisted reference populations contain 1,571 fewer false
 negatives, including 745 fewer misses in the 25-100 m2 bucket and 181 fewer
@@ -443,6 +444,24 @@ produced zero detections in the pure-empty Postel forest control. The active
 confidence therefore remains `0.15`. This result does not claim production
 perfection and does not justify another model-training run by itself.
 
+A coverage-aligned July 2026 rerun supersedes the older displayed profile
+averages above without changing the active model or threshold. On the exact
+current pipeline, the seven independent Mol/Kempen zones measured mean
+precision `0.6141`, recall `0.6062` and F1 `0.6069`; the minimum zone F1 was
+`0.4749` in Mol Postel. The active model again produced zero detections in
+Postel-bos, Lommel-heide and Arendonk-heide. These are the values shown in the
+Detection Lab operator profile.
+
+The reviewed-accuracy experiment added six training-only AOIs from Arendonk,
+Dessel, Meerhout, Laakdal, Nijlen and Hulshout. The paged GRB export contained
+9,964 complete reference features. Its audited `512`-tile corpus retained 252
+tiles and 79,192 labels with no invalid or missing labels. The inactive
+`geointel-building-yolov8s-reviewedexp6-minpx3-img640-ft20-pt` challenger
+improved mean seven-zone F1 to `0.6248`, but produced two detections in the
+explicitly empty Postel-bos control. The formal fail-closed promotion report
+therefore retained the current active model. Positive-score gains never
+override a failed pure-empty background gate.
+
 False-positive and false-negative evidence from persisted detection QA can be
 classified through `detection_reviews`. The queue derives from quality-check
 evidence ids and resolves persisted Detection and reference VectorFeature rows.

docs/CODEX_EXECUTION_LOG.md

@@ -8247,3 +8247,49 @@ Next:
 - Collect a new training-only small-building/background evidence pack outside
   all operational holdouts, then train an inactive candidate only if the pack
   passes label, leakage and sample-volume audits.
+
+## Sprint 199 - Reviewed accuracy expansion and fail-closed challenger (2026-07-15)
+
+Implemented:
+- Added six training-only reference AOIs for Arendonk, Dessel, Meerhout,
+  Laakdal, Nijlen and Hulshout. Tests enforce explicit training role, unique
+  municipality/center pairs and at least 2 km separation from every protected
+  Mol, Turnhout, Retie, Westerlo, Vosselaar and Grobbendonk holdout.
+- Prepared 1024 px orthophoto/GRB pairs on Tower. The paged GRB exports contain
+  9,964 features in total, use EPSG:31370 rasters and have no truncated,
+  invalid or empty reference geometry.
+- Exported `yolo-building-aoi1024-reviewedexp6-minpx3vis035`: 252 tiles, 234
+  positive, 18 negative, 234 train, 18 validation and 79,192 labels. The
+  dataset audit returned `ok`; the 64-tile visual review covered 28 retained
+  sources with no invalid, missing or low-variance selections.
+- Fine-tuned the active local YOLOv8s asset for 20 CPU epochs at image size
+  `640`. The best checkpoint came from epoch 16 and was copied as inactive
+  `geointel-building-yolov8s-reviewedexp6-minpx3-img640-ft20-pt`, SHA256
+  `038f1f97a6afd534f29e1f392a730a58207b928ca01e31ab8d8fed6106705820`.
+- Re-ran both active and challenger assets through the exact current API,
+  persistence and coverage-aware QA/QC path at tile `512`, overlap `64`,
+  confidence `0.15` and match IoU `0.25`. The evaluated reference populations
+  are equal per zone; older unequal-coverage runs were excluded.
+- The active model measured mean precision `0.6141`, recall `0.6062` and F1
+  `0.6069`; the challenger measured `0.6251`, `0.6293` and `0.6248`. Challenger
+  F1 improved in all seven zones and reduced false negatives from 3,416 to
+  3,208.
+- The challenger produced two detections in explicitly empty Postel-bos while
+  the active model remained at zero across Postel-bos, Lommel-heide and
+  Arendonk-heide. The formal promotion report recommended the existing active
+  key. No `.env`, active model path or runtime threshold was changed.
+- Updated the approved Detection Lab profile to the current coverage-aligned
+  active evidence and retained the explicit Postel limitation.
+
+Validation evidence:
+- Full readiness after the documentation/profile update passed 598 backend
+  tests, backend compilation, one Alembic head, frontend typecheck/build and
+  the complete shell syntax gate.
+- Persistent promotion evidence lives under
+  `/app/storage/operator-data/model-review/reviewed-accuracy-expansion/` and
+  `/app/storage/operator-evidence/mol-operational-validation/`.
+
+Next:
+- Convert the two confirmed Postel-bos challenger errors into a complete,
+  leakage-free hard-negative training sample, add independent empty controls,
+  then train a new inactive candidate through the same fail-closed gate.

docs/TODO.md

@@ -506,9 +506,9 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Add a model promotion decision report that combines positive-AOI score with hard-negative false-positive pressure.
 - [x] Train and reject a YOLOv8s hard-negative r8 partial candidate after 12 CPU epochs through the full positive/background promotion gate.
 - [x] Finish and reject the full YOLOv8s hard-negative r8 e60 candidate through the same promotion gate.
-- [ ] Add more AOIs after the tile-level baseline so the next local model attempt is not limited to Geel/Mol/Turnhout.
-- [ ] Add negative/background AOIs so the next tile dataset is not all positive tiles.
-- [ ] Improve positive training coverage/label quality before the next higher-capacity model attempt; simply extending the same hardneg r8 run is not enough.
+- [x] Add more AOIs after the tile-level baseline so the next local model attempt is not limited to Geel/Mol/Turnhout.
+- [x] Add negative/background AOIs so the next tile dataset is not all positive tiles.
+- [x] Improve positive training coverage/label quality before the next higher-capacity model attempt; simply extending the same hardneg r8 run is not enough.
 # Sprint 146 - Operator YOLO dataset quality audit
 
 - [x] Add a dataset/label-quality audit for generated operator YOLO tile datasets.
@@ -516,7 +516,7 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Wire the audit script into the readiness syntax gate.
 - [x] Use live audit output to decide whether the next model pass needs more positive AOIs, label cleanup or unique hard negatives.
 - [x] Add more unique background/hard-negative AOIs before repeating hard-negative-balanced YOLO training.
-- [ ] Keep `yolo-building-tile-expanded160` as the clean current training baseline; avoid promoting r4/r8 repeat-heavy datasets as defaults.
+- [x] Supersede `yolo-building-tile-expanded160` only with audited AOI1024 corpora; no repeat-heavy r4/r8 dataset became a default.
 - [x] Regenerate Tower operator samples, export a new unique-hard-negative tile dataset and rerun the dataset audit before training.
 
 # Sprint 147 - Unique hard-negative AOI expansion
@@ -550,7 +550,7 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Prefer the complete municipality workspace on a fresh session while
   keeping the lightweight boundary as the initially rendered layer.
 - [x] Make GeoJSON bounds calculation safe for municipality-scale layers.
-- [ ] Acquire and tile a georeferenced raster only for an explicitly selected
+- [x] Acquire and tile a georeferenced raster only for an explicitly selected
   Mol analysis zone before running the next configured-YOLO validation.
 
 # Sprint 182 - Municipality viewport delivery and bounded AI handoff
@@ -561,7 +561,7 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Preserve complete loading and auto-fit behavior for small vectors and persisted AI/QA overlays.
 - [x] Allow the real raster/detection/QA operator smoke to reuse the definitive Mol project.
 - [x] Link operator raster/reference uploads to the persisted analysis Area and keep their names distinct from municipality-wide layers.
-- [ ] Execute the bounded Mol-Centrum configured-YOLO/QA workflow in the deployed runtime and retain its persisted ids as release evidence.
+- [x] Execute the bounded Mol-Centrum configured-YOLO/QA workflow in the deployed runtime and retain its persisted ids as release evidence.
 
 # Sprint 171 - Positive AOI expansion and small-building recovery
 
@@ -575,3 +575,15 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Finish the inactive expanded-minpx4 candidate and run the full promotion gate.
 - [x] Export the focused 23-sample minpx3 corpus with independent Vosselaar/Grobbendonk validation and external Turnhout/Retie/Westerlo holdouts.
 - [x] Train, audit and guarded-activate the focused small-building candidate only after all persisted promotion gates passed.
+
+# Sprint 199 - Reviewed accuracy expansion and fail-closed challenger
+
+- [x] Add six complete training-only AOIs outside every protected Mol/Kempen holdout.
+- [x] Page and audit 9,964 new GRB building references with municipality and CRS provenance.
+- [x] Export and visually inspect the 252-tile, 79,192-label reviewed-accuracy corpus.
+- [x] Train one inactive 20-epoch YOLOv8s challenger from the active local model without downloads.
+- [x] Rerun active and challenger models on identical coverage-aware reference populations.
+- [x] Confirm positive-zone F1 improvement in all seven holdouts.
+- [x] Reject challenger activation because Postel-bos produced two detections while the active model remained at zero.
+- [x] Update the recommended UI profile with the current active model's coverage-aligned evidence.
+- [ ] Collect new training-only hard negatives matching the two Postel-bos errors before another challenger attempt.

docs/reviews/2026-07-15-reviewed-accuracy-challenger.md

@@ -0,0 +1,77 @@
+# Reviewed accuracy challenger - 2026-07-15
+
+## Scope
+
+This pass closes the evidence requirement from the preceding FP/FN review. It
+adds complete, training-only orthophoto/GRB AOIs outside all protected
+operational holdouts, trains one inactive local challenger and compares both
+models through the current persisted GeoIntel workflow. No model was
+downloaded and the active runtime configuration was not changed.
+
+- Active model: `geointel-building-yolov8s-smallbld-minpx3-img640-ft30-pt`
+- Active SHA256: `a9088b8491dfae36694b53e9e9406cb4e3511d334a5712fa34f75078a47759c1`
+- Challenger: `geointel-building-yolov8s-reviewedexp6-minpx3-img640-ft20-pt`
+- Challenger SHA256: `038f1f97a6afd534f29e1f392a730a58207b928ca01e31ab8d8fed6106705820`
+- Fixed profile: tile `512`, overlap `64`, confidence `0.15`, QA IoU `0.25`
+
+## Data and training
+
+Six new training-only centers were added in Arendonk, Dessel, Meerhout,
+Laakdal, Nijlen and Hulshout. Every center is in its documented municipality,
+uses a complete paged GRB GBG export and remains at least 2 km from every
+protected positive holdout.
+
+- New GRB references: 9,964 features
+- Corpus: 252 tiles, 234 positive and 18 negative
+- Split: 234 training and 18 validation tiles
+- Labels: 79,192
+- Dataset audit: `ok`, zero invalid or missing labels
+- Visual audit: 64 tiles across 28 retained sources, zero invalid or
+  low-variance selections
+- Fine-tuning: 20 CPU epochs from the active local model, image size `640`
+- Best checkpoint: epoch 16, validation precision `0.700`, recall `0.406`,
+  mAP50 `0.369`, mAP50-95 `0.159`
+
+## Coverage-aligned result
+
+Both assets were rerun after the current coverage-aware QA logic was deployed.
+Each zone comparison uses the same evaluated reference population; older runs
+with different edge coverage are not used for the decision.
+
+| Zone | Active F1 | Challenger F1 | Delta |
+| --- | ---: | ---: | ---: |
+| Mol Achterbos | 0.6694 | 0.6850 | +0.0156 |
+| Mol Donk | 0.5894 | 0.6259 | +0.0365 |
+| Mol Gompel | 0.6564 | 0.6694 | +0.0131 |
+| Mol Postel | 0.4749 | 0.4753 | +0.0005 |
+| Retie | 0.6325 | 0.6491 | +0.0166 |
+| Turnhout | 0.5919 | 0.5988 | +0.0069 |
+| Westerlo | 0.6336 | 0.6700 | +0.0364 |
+
+| Aggregate | Active | Challenger |
+| --- | ---: | ---: |
+| Mean precision | 0.6141 | 0.6251 |
+| Mean recall | 0.6062 | 0.6293 |
+| Mean F1 | 0.6069 | 0.6248 |
+| Matches | 5,711 | 5,919 |
+| False positives | 3,568 | 3,506 |
+| False negatives | 3,416 | 3,208 |
+
+## Promotion decision
+
+| Pure-empty control | Active detections | Challenger detections |
+| --- | ---: | ---: |
+| Arendonk-heide | 0 | 0 |
+| Lommel-heide | 0 | 0 |
+| Postel-bos | 0 | 2 |
+
+Decision: **retain the active model**. The challenger improves every positive
+zone but violates the zero-detection Postel-bos gate. The formal promotion
+report therefore recommends the existing active key
+`geointel-building-yolov8s-smallbld-minpx3-img640-ft30-pt|512|64|0.15`.
+Positive-score gains do not override a failed fail-closed background control.
+
+Persistent evidence is stored below
+`/app/storage/operator-data/model-review/reviewed-accuracy-expansion/` and
+`/app/storage/operator-evidence/mol-operational-validation/`. Generated rasters,
+weights and large evidence JSON files remain outside Git.