geointel/docs/reviews/2026-07-15-reviewed-accuracy-challenger.md

Reviewed accuracy challenger - 2026-07-15

Scope

This pass closes the evidence requirement from the preceding FP/FN review. It adds complete, training-only orthophoto/GRB AOIs outside all protected operational holdouts, trains one inactive local challenger and compares both models through the current persisted GeoIntel workflow. No model was downloaded and the active runtime configuration was not changed.

• Active model: geointel-building-yolov8s-smallbld-minpx3-img640-ft30-pt
• Active SHA256: a9088b8491dfae36694b53e9e9406cb4e3511d334a5712fa34f75078a47759c1
• Challenger: geointel-building-yolov8s-reviewedexp6-minpx3-img640-ft20-pt
• Challenger SHA256: 038f1f97a6afd534f29e1f392a730a58207b928ca01e31ab8d8fed6106705820
• Fixed profile: tile 512, overlap 64, confidence 0.15, QA IoU 0.25

Data and training

Six new training-only centers were added in Arendonk, Dessel, Meerhout, Laakdal, Nijlen and Hulshout. Every center is in its documented municipality, uses a complete paged GRB GBG export and remains at least 2 km from every protected positive holdout.

• New GRB references: 9,964 features
• Corpus: 252 tiles, 234 positive and 18 negative
• Split: 234 training and 18 validation tiles
• Labels: 79,192
• Dataset audit: ok, zero invalid or missing labels
• Visual audit: 64 tiles across 28 retained sources, zero invalid or low-variance selections
• Fine-tuning: 20 CPU epochs from the active local model, image size 640
• Best checkpoint: epoch 16, validation precision 0.700, recall 0.406, mAP50 0.369, mAP50-95 0.159

Coverage-aligned result

Both assets were rerun after the current coverage-aware QA logic was deployed. Each zone comparison uses the same evaluated reference population; older runs with different edge coverage are not used for the decision.

Zone	Active F1	Challenger F1	Delta
Mol Achterbos	0.6694	0.6850	+0.0156
Mol Donk	0.5894	0.6259	+0.0365
Mol Gompel	0.6564	0.6694	+0.0131
Mol Postel	0.4749	0.4753	+0.0005
Retie	0.6325	0.6491	+0.0166
Turnhout	0.5919	0.5988	+0.0069
Westerlo	0.6336	0.6700	+0.0364

Aggregate	Active	Challenger
Mean precision	0.6141	0.6251
Mean recall	0.6062	0.6293
Mean F1	0.6069	0.6248
Matches	5,711	5,919
False positives	3,568	3,506
False negatives	3,416	3,208

Promotion decision

Pure-empty control	Active detections	Challenger detections
Arendonk-heide	0	0
Lommel-heide	0	0
Postel-bos	0	2

Decision: retain the active model. The challenger improves every positive zone but violates the zero-detection Postel-bos gate. The formal promotion report therefore recommends the existing active key geointel-building-yolov8s-smallbld-minpx3-img640-ft30-pt|512|64|0.15. Positive-score gains do not override a failed fail-closed background control.

Persistent evidence is stored below /app/storage/operator-data/model-review/reviewed-accuracy-expansion/ and /app/storage/operator-evidence/mol-operational-validation/. Generated rasters, weights and large evidence JSON files remain outside Git.