docs: record fail-closed accuracy challenger

CHANGELOG.md

@@ -7,6 +7,25 @@
 
 # Changelog
 
+## Sprint 199 Reviewed accuracy expansion (2026-07-15)
+
+- Added six leakage-free training AOIs in Arendonk, Dessel, Meerhout, Laakdal,
+  Nijlen and Hulshout, backed by 9,964 paged GRB building references.
+- Exported and audited a 252-tile, 79,192-label corpus; the configured audit and
+  balanced 64-tile visual review found no invalid, missing or low-variance
+  selections.
+- Fine-tuned one inactive local YOLOv8s challenger for 20 CPU epochs without
+  downloads. Its best checkpoint SHA256 is
+  `038f1f97a6afd534f29e1f392a730a58207b928ca01e31ab8d8fed6106705820`.
+- Re-ran active and challenger models through the same current persisted QA/QC
+  pipeline on four Mol and three regional holdouts. The challenger improved
+  mean F1 from `0.6069` to `0.6248` and improved every zone.
+- Retained the active model because the challenger produced two detections in
+  empty Postel-bos; the active profile remained zero across all three empty
+  controls. No runtime model or `.env` setting was changed.
+- Replaced stale Detection Lab profile averages with coverage-aligned active
+  evidence: precision `0.6141`, recall `0.6062`, F1 `0.6069`.
+
 ## Sprint 198 Evidence-closed model review (2026-07-15)
 
 - Completed visual and geometric review of 48 persisted false-positive and 48

backend/README.md

@@ -519,6 +519,13 @@ false-negative counts, but has lower precision than the previous balanced
 model; operators must review and persist QA/QC rather than treating detections
 as ground truth.
 
+The latest coverage-aligned rerun of this exact profile measured mean precision
+`0.6141`, recall `0.6062` and F1 `0.6069` over Mol Achterbos, Donk, Gompel and
+Postel plus Retie, Turnhout and Westerlo. The three pure-empty controls remained
+at zero detections. A reviewed six-AOI fine-tuning challenger reached mean F1
+`0.6248` but remained inactive because it produced two false detections in the
+Postel-bos empty control.
+
 For model-quality calibration, run the confidence sweep wrapper:
 
 ```bash

backend/tests/test_sprint155_detection_operator_profiles.py

@@ -20,12 +20,14 @@ def test_detection_operator_profiles_define_explicit_yolo_candidates_and_promote
     assert "defaultApproved: true" in source
     assert "promotionRecommendation: 'promote_candidate'" in source
     assert "positiveSampleCount: 7" in source
-    assert "f1: 0.5824578631584316" in source
+    assert "precision: 0.6140895327792112" in source
+    assert "recall: 0.6062221049337548" in source
+    assert "f1: 0.6068607646002744" in source
     assert "f1: 0.5432865390636915" in source
     assert "maxBackgroundDetections: 0" in source
     assert "lege-achtergrondtest is geslaagd" in source
-    assert "1.571 minder gemiste gebouwen" in source
-    assert "onterecht gevonden objecten" in source
+    assert "Postel blijft met 47,5% F1" in source
+    assert "controlekandidaat en niet als grondwaarheid" in source
 
 
 def test_detection_lab_surfaces_profiles_as_deliberate_operator_actions() -> None:

backend/tests/test_sprint193_end_user_workbench.py

@@ -67,7 +67,8 @@ def test_visible_ai_and_quality_labels_are_end_user_facing() -> None:
     providers = read("frontend/src/components/providers/ProviderPanel.tsx")
 
     assert "Aanbevolen controleprofiel kleine gebouwen" in profiles
-    assert "Controleer wel extra op onterecht gevonden objecten" in profiles
+    assert "Postel blijft met 47,5% F1" in profiles
+    assert "controlekandidaat en niet als grondwaarheid" in profiles
     assert "qualityStatusLabel" in quality
     assert "nog niet uitgevoerd" in quality
     assert "Nog geen bestand gekozen." in export_preview

docs/AI_PIPELINES.md

@@ -421,9 +421,10 @@ false positives block default promotion.
 The focused small-building local model asset,
 `geointel-building-yolov8s-smallbld-minpx3-img640-ft30-pt`, is the current
 recommended Detection Lab operator profile. Use tile size `512`, overlap `64`
-and confidence threshold `0.15`. Persisted QA/QC at match IoU `0.25` across
-seven positive AOIs measured mean precision `0.5898`, recall `0.5770` and F1
-`0.5825`; minimum per-AOI F1 was `0.5528`. The strict three-sample pure-empty
+and confidence threshold `0.15`. Its original promotion evidence at match IoU
+`0.25` across seven positive AOIs measured mean precision `0.5898`, recall
+`0.5770` and F1 `0.5825`; minimum per-AOI F1 was `0.5528`. The strict
+three-sample pure-empty
 background gate produced zero detections. Compared with the previous balanced
 profile, the same persisted reference populations contain 1,571 fewer false
 negatives, including 745 fewer misses in the 25-100 m2 bucket and 181 fewer
@@ -443,6 +444,24 @@ produced zero detections in the pure-empty Postel forest control. The active
 confidence therefore remains `0.15`. This result does not claim production
 perfection and does not justify another model-training run by itself.
 
+A coverage-aligned July 2026 rerun supersedes the older displayed profile
+averages above without changing the active model or threshold. On the exact
+current pipeline, the seven independent Mol/Kempen zones measured mean
+precision `0.6141`, recall `0.6062` and F1 `0.6069`; the minimum zone F1 was
+`0.4749` in Mol Postel. The active model again produced zero detections in
+Postel-bos, Lommel-heide and Arendonk-heide. These are the values shown in the
+Detection Lab operator profile.
+
+The reviewed-accuracy experiment added six training-only AOIs from Arendonk,
+Dessel, Meerhout, Laakdal, Nijlen and Hulshout. The paged GRB export contained
+9,964 complete reference features. Its audited `512`-tile corpus retained 252
+tiles and 79,192 labels with no invalid or missing labels. The inactive
+`geointel-building-yolov8s-reviewedexp6-minpx3-img640-ft20-pt` challenger
+improved mean seven-zone F1 to `0.6248`, but produced two detections in the
+explicitly empty Postel-bos control. The formal fail-closed promotion report
+therefore retained the current active model. Positive-score gains never
+override a failed pure-empty background gate.
+
 False-positive and false-negative evidence from persisted detection QA can be
 classified through `detection_reviews`. The queue derives from quality-check
 evidence ids and resolves persisted Detection and reference VectorFeature rows.

docs/CODEX_EXECUTION_LOG.md

@@ -8247,3 +8247,49 @@ Next:
 - Collect a new training-only small-building/background evidence pack outside
   all operational holdouts, then train an inactive candidate only if the pack
   passes label, leakage and sample-volume audits.
+
+## Sprint 199 - Reviewed accuracy expansion and fail-closed challenger (2026-07-15)
+
+Implemented:
+- Added six training-only reference AOIs for Arendonk, Dessel, Meerhout,
+  Laakdal, Nijlen and Hulshout. Tests enforce explicit training role, unique
+  municipality/center pairs and at least 2 km separation from every protected
+  Mol, Turnhout, Retie, Westerlo, Vosselaar and Grobbendonk holdout.
+- Prepared 1024 px orthophoto/GRB pairs on Tower. The paged GRB exports contain
+  9,964 features in total, use EPSG:31370 rasters and have no truncated,
+  invalid or empty reference geometry.
+- Exported `yolo-building-aoi1024-reviewedexp6-minpx3vis035`: 252 tiles, 234
+  positive, 18 negative, 234 train, 18 validation and 79,192 labels. The
+  dataset audit returned `ok`; the 64-tile visual review covered 28 retained
+  sources with no invalid, missing or low-variance selections.
+- Fine-tuned the active local YOLOv8s asset for 20 CPU epochs at image size
+  `640`. The best checkpoint came from epoch 16 and was copied as inactive
+  `geointel-building-yolov8s-reviewedexp6-minpx3-img640-ft20-pt`, SHA256
+  `038f1f97a6afd534f29e1f392a730a58207b928ca01e31ab8d8fed6106705820`.
+- Re-ran both active and challenger assets through the exact current API,
+  persistence and coverage-aware QA/QC path at tile `512`, overlap `64`,
+  confidence `0.15` and match IoU `0.25`. The evaluated reference populations
+  are equal per zone; older unequal-coverage runs were excluded.
+- The active model measured mean precision `0.6141`, recall `0.6062` and F1
+  `0.6069`; the challenger measured `0.6251`, `0.6293` and `0.6248`. Challenger
+  F1 improved in all seven zones and reduced false negatives from 3,416 to
+  3,208.
+- The challenger produced two detections in explicitly empty Postel-bos while
+  the active model remained at zero across Postel-bos, Lommel-heide and
+  Arendonk-heide. The formal promotion report recommended the existing active
+  key. No `.env`, active model path or runtime threshold was changed.
+- Updated the approved Detection Lab profile to the current coverage-aligned
+  active evidence and retained the explicit Postel limitation.
+
+Validation evidence:
+- Full readiness after the documentation/profile update passed 598 backend
+  tests, backend compilation, one Alembic head, frontend typecheck/build and
+  the complete shell syntax gate.
+- Persistent promotion evidence lives under
+  `/app/storage/operator-data/model-review/reviewed-accuracy-expansion/` and
+  `/app/storage/operator-evidence/mol-operational-validation/`.
+
+Next:
+- Convert the two confirmed Postel-bos challenger errors into a complete,
+  leakage-free hard-negative training sample, add independent empty controls,
+  then train a new inactive candidate through the same fail-closed gate.

docs/TODO.md

@@ -506,9 +506,9 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Add a model promotion decision report that combines positive-AOI score with hard-negative false-positive pressure.
 - [x] Train and reject a YOLOv8s hard-negative r8 partial candidate after 12 CPU epochs through the full positive/background promotion gate.
 - [x] Finish and reject the full YOLOv8s hard-negative r8 e60 candidate through the same promotion gate.
-- [ ] Add more AOIs after the tile-level baseline so the next local model attempt is not limited to Geel/Mol/Turnhout.
-- [ ] Add negative/background AOIs so the next tile dataset is not all positive tiles.
-- [ ] Improve positive training coverage/label quality before the next higher-capacity model attempt; simply extending the same hardneg r8 run is not enough.
+- [x] Add more AOIs after the tile-level baseline so the next local model attempt is not limited to Geel/Mol/Turnhout.
+- [x] Add negative/background AOIs so the next tile dataset is not all positive tiles.
+- [x] Improve positive training coverage/label quality before the next higher-capacity model attempt; simply extending the same hardneg r8 run is not enough.
 # Sprint 146 - Operator YOLO dataset quality audit
 
 - [x] Add a dataset/label-quality audit for generated operator YOLO tile datasets.
@@ -516,7 +516,7 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Wire the audit script into the readiness syntax gate.
 - [x] Use live audit output to decide whether the next model pass needs more positive AOIs, label cleanup or unique hard negatives.
 - [x] Add more unique background/hard-negative AOIs before repeating hard-negative-balanced YOLO training.
-- [ ] Keep `yolo-building-tile-expanded160` as the clean current training baseline; avoid promoting r4/r8 repeat-heavy datasets as defaults.
+- [x] Supersede `yolo-building-tile-expanded160` only with audited AOI1024 corpora; no repeat-heavy r4/r8 dataset became a default.
 - [x] Regenerate Tower operator samples, export a new unique-hard-negative tile dataset and rerun the dataset audit before training.
 
 # Sprint 147 - Unique hard-negative AOI expansion
@@ -550,7 +550,7 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Prefer the complete municipality workspace on a fresh session while
   keeping the lightweight boundary as the initially rendered layer.
 - [x] Make GeoJSON bounds calculation safe for municipality-scale layers.
-- [ ] Acquire and tile a georeferenced raster only for an explicitly selected
+- [x] Acquire and tile a georeferenced raster only for an explicitly selected
   Mol analysis zone before running the next configured-YOLO validation.
 
 # Sprint 182 - Municipality viewport delivery and bounded AI handoff
@@ -561,7 +561,7 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Preserve complete loading and auto-fit behavior for small vectors and persisted AI/QA overlays.
 - [x] Allow the real raster/detection/QA operator smoke to reuse the definitive Mol project.
 - [x] Link operator raster/reference uploads to the persisted analysis Area and keep their names distinct from municipality-wide layers.
-- [ ] Execute the bounded Mol-Centrum configured-YOLO/QA workflow in the deployed runtime and retain its persisted ids as release evidence.
+- [x] Execute the bounded Mol-Centrum configured-YOLO/QA workflow in the deployed runtime and retain its persisted ids as release evidence.
 
 # Sprint 171 - Positive AOI expansion and small-building recovery
 
@@ -575,3 +575,15 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Finish the inactive expanded-minpx4 candidate and run the full promotion gate.
 - [x] Export the focused 23-sample minpx3 corpus with independent Vosselaar/Grobbendonk validation and external Turnhout/Retie/Westerlo holdouts.
 - [x] Train, audit and guarded-activate the focused small-building candidate only after all persisted promotion gates passed.
+
+# Sprint 199 - Reviewed accuracy expansion and fail-closed challenger
+
+- [x] Add six complete training-only AOIs outside every protected Mol/Kempen holdout.
+- [x] Page and audit 9,964 new GRB building references with municipality and CRS provenance.
+- [x] Export and visually inspect the 252-tile, 79,192-label reviewed-accuracy corpus.
+- [x] Train one inactive 20-epoch YOLOv8s challenger from the active local model without downloads.
+- [x] Rerun active and challenger models on identical coverage-aware reference populations.
+- [x] Confirm positive-zone F1 improvement in all seven holdouts.
+- [x] Reject challenger activation because Postel-bos produced two detections while the active model remained at zero.
+- [x] Update the recommended UI profile with the current active model's coverage-aligned evidence.
+- [ ] Collect new training-only hard negatives matching the two Postel-bos errors before another challenger attempt.

docs/reviews/2026-07-15-reviewed-accuracy-challenger.md

@@ -0,0 +1,77 @@
+# Reviewed accuracy challenger - 2026-07-15
+
+## Scope
+
+This pass closes the evidence requirement from the preceding FP/FN review. It
+adds complete, training-only orthophoto/GRB AOIs outside all protected
+operational holdouts, trains one inactive local challenger and compares both
+models through the current persisted GeoIntel workflow. No model was
+downloaded and the active runtime configuration was not changed.
+
+- Active model: `geointel-building-yolov8s-smallbld-minpx3-img640-ft30-pt`
+- Active SHA256: `a9088b8491dfae36694b53e9e9406cb4e3511d334a5712fa34f75078a47759c1`
+- Challenger: `geointel-building-yolov8s-reviewedexp6-minpx3-img640-ft20-pt`
+- Challenger SHA256: `038f1f97a6afd534f29e1f392a730a58207b928ca01e31ab8d8fed6106705820`
+- Fixed profile: tile `512`, overlap `64`, confidence `0.15`, QA IoU `0.25`
+
+## Data and training
+
+Six new training-only centers were added in Arendonk, Dessel, Meerhout,
+Laakdal, Nijlen and Hulshout. Every center is in its documented municipality,
+uses a complete paged GRB GBG export and remains at least 2 km from every
+protected positive holdout.
+
+- New GRB references: 9,964 features
+- Corpus: 252 tiles, 234 positive and 18 negative
+- Split: 234 training and 18 validation tiles
+- Labels: 79,192
+- Dataset audit: `ok`, zero invalid or missing labels
+- Visual audit: 64 tiles across 28 retained sources, zero invalid or
+  low-variance selections
+- Fine-tuning: 20 CPU epochs from the active local model, image size `640`
+- Best checkpoint: epoch 16, validation precision `0.700`, recall `0.406`,
+  mAP50 `0.369`, mAP50-95 `0.159`
+
+## Coverage-aligned result
+
+Both assets were rerun after the current coverage-aware QA logic was deployed.
+Each zone comparison uses the same evaluated reference population; older runs
+with different edge coverage are not used for the decision.
+
+| Zone | Active F1 | Challenger F1 | Delta |
+| --- | ---: | ---: | ---: |
+| Mol Achterbos | 0.6694 | 0.6850 | +0.0156 |
+| Mol Donk | 0.5894 | 0.6259 | +0.0365 |
+| Mol Gompel | 0.6564 | 0.6694 | +0.0131 |
+| Mol Postel | 0.4749 | 0.4753 | +0.0005 |
+| Retie | 0.6325 | 0.6491 | +0.0166 |
+| Turnhout | 0.5919 | 0.5988 | +0.0069 |
+| Westerlo | 0.6336 | 0.6700 | +0.0364 |
+
+| Aggregate | Active | Challenger |
+| --- | ---: | ---: |
+| Mean precision | 0.6141 | 0.6251 |
+| Mean recall | 0.6062 | 0.6293 |
+| Mean F1 | 0.6069 | 0.6248 |
+| Matches | 5,711 | 5,919 |
+| False positives | 3,568 | 3,506 |
+| False negatives | 3,416 | 3,208 |
+
+## Promotion decision
+
+| Pure-empty control | Active detections | Challenger detections |
+| --- | ---: | ---: |
+| Arendonk-heide | 0 | 0 |
+| Lommel-heide | 0 | 0 |
+| Postel-bos | 0 | 2 |
+
+Decision: **retain the active model**. The challenger improves every positive
+zone but violates the zero-detection Postel-bos gate. The formal promotion
+report therefore recommends the existing active key
+`geointel-building-yolov8s-smallbld-minpx3-img640-ft30-pt|512|64|0.15`.
+Positive-score gains do not override a failed fail-closed background control.
+
+Persistent evidence is stored below
+`/app/storage/operator-data/model-review/reviewed-accuracy-expansion/` and
+`/app/storage/operator-evidence/mol-operational-validation/`. Generated rasters,
+weights and large evidence JSON files remain outside Git.

frontend/README.md

@@ -183,6 +183,7 @@ Before creating tiles, the guided action inspects raster dimensions and estimate
 - When `yolo-configured` is selected, users can provide an existing raster tile manifest path.
 - Detection Lab lists local model assets from `GET /api/v1/detection/model-assets` so operators can choose an existing mounted model file instead of editing only one hidden `YOLO_MODEL_PATH` slot.
 - Detection Lab exposes explicit operator profiles for mounted local building detectors. The focused small-building model is the recommended recall-balanced `0.15` profile; the previous expanded-AOI `0.15` model remains available for higher precision, and the background-aware `0.35` model remains the conservative review choice. Applying a profile never downloads weights, changes runtime environment or starts inference automatically.
+- The recommended profile metrics are coverage-aligned with the current QA pipeline: mean precision `0.6141`, recall `0.6062` and F1 `0.6069` over seven independent Mol/Kempen zones, with zero detections in all three pure-empty controls. A higher-positive-F1 challenger remains hidden from approved profiles because it failed the Postel-bos empty-control gate.
 - Applying a profile deliberately selects the local model asset and threshold for the browser-run request; runtime default activation remains a separate guarded `.env` operation through `scripts/activate_promoted_yolo_candidate.py`.
 - Detection Lab includes a read-only YOLO runtime preflight panel with backend status, dependency visibility, local model configuration, `torch`/`ultralytics` versions, CUDA state and `YOLO_CONFIG_DIR`.
 - The UI still does not download models or create fake detections; backend status and error codes remain the source of truth.

frontend/src/components/detection/detectionProfiles.ts

@@ -22,15 +22,15 @@ export const DETECTION_OPERATOR_PROFILES: DetectionOperatorProfile[] = [
     confidenceThreshold: 0.15,
     defaultApproved: true,
     promotionRecommendation: 'promote_candidate',
-    precision: 0.5898197517793451,
-    recall: 0.576992100419565,
-    f1: 0.5824578631584316,
+    precision: 0.6140895327792112,
+    recall: 0.6062221049337548,
+    f1: 0.6068607646002744,
     positiveSampleCount: 7,
     maxBackgroundDetections: 0,
     description:
-      'Aanbevolen profiel met een evenwicht tussen gevonden en gemiste kleine gebouwen, gemeten over zeven testgebieden in de Kempen.',
+      'Aanbevolen profiel met een evenwicht tussen gevonden en gemiste kleine gebouwen, opnieuw gemeten over zeven onafhankelijke testgebieden in Mol en de Kempen.',
     limitationMessage:
-      'De lege-achtergrondtest is geslaagd en de kwaliteitsmeting vond 1.571 minder gemiste gebouwen dan het vorige profiel. Controleer wel extra op onterecht gevonden objecten.',
+      'De drie lege-achtergrondtests zijn geslaagd. Postel blijft met 47,5% F1 het moeilijkste testgebied; behandel elke detectie als een controlekandidaat en niet als grondwaarheid.',
   },
   {
     id: 'expanded-balanced-review',

scripts/README.md

@@ -624,10 +624,22 @@ Current Tower audit status:
   review contained no missing, invalid or low-variance selections. The trained
   `geointel-building-yolov8s-smallbld-minpx3-img640-ft30-pt` candidate passed
   seven positive-AOI and three pure-empty background gates at tile `512`,
-  overlap `64`, threshold `0.15` and QA match IoU `0.25`. Mean F1 is `0.5825`
-  and all pure-empty samples remain at zero detections. Persisted comparison
-  found 1,571 fewer false negatives than the previous balanced model, with a
-  lower mean precision and therefore a higher operator review load.
+  overlap `64`, threshold `0.15` and QA match IoU `0.25`. Original promotion
+  evidence measured mean F1 `0.5825`; the later coverage-aligned rerun measured
+  `0.6069`. All pure-empty samples remain at zero detections. The original
+  persisted comparison found 1,571 fewer false negatives than the previous
+  balanced model, with a lower mean precision and therefore a higher operator
+  review load.
+- `yolo-building-aoi1024-reviewedexp6-minpx3vis035`: leakage-free accuracy
+  expansion of the focused corpus with new training-only AOIs in Arendonk,
+  Dessel, Meerhout, Laakdal, Nijlen and Hulshout. The paged GRB preparation
+  added 9,964 reference features. The export retained 252 tiles (234 positive,
+  18 negative) and 79,192 labels; its configured audit and 64-tile visual
+  review found no invalid, missing or low-variance selections. The inactive
+  `geointel-building-yolov8s-reviewedexp6-minpx3-img640-ft20-pt` model improved
+  coverage-aligned seven-zone mean F1 from `0.6069` to `0.6248`, but its two
+  detections in empty Postel-bos failed the strict background gate. The active
+  focused model remains unchanged.
 
 Use `--samples` or `OPERATOR_YOLO_SAMPLES` to make an experimental corpus
 membership explicit. Dataset summaries preserve the complete manifest count,