Document stronger building model benchmark

CHANGELOG.md

@@ -7,6 +7,15 @@
 
 # Changelog
 
+## Sprint 128 Stronger building model runtime benchmark (2026-07-07)
+
+- Added `keremberke/yolov8s-building-segmentation` as an explicit Tower runtime model asset at `/mnt/user/appdata/geointel/models/yolov8s-building-segmentation.pt`; the file is not committed to Git.
+- Verified the live model catalog exposes `yolov8s-building-segmentation-pt` with `will_download_models=false` and SHA256 `a27af31654c6a4edbdc85581c33d93c13986b5919de7de410f8d85d801b3bb34`.
+- YOLO preflight loaded the model locally with `model_load_ok=true` and no automatic download.
+- Ran a 36-run Geel/Mol/Turnhout matrix comparing `yolov8n-building-segmentation-pt` and `yolov8s-building-segmentation-pt` across tile sizes `512`/`640` and thresholds `0.50`/`0.25`/`0.15`.
+- Best overall score was Mol with `yolov8s-building-segmentation-pt`, tile `640`, threshold `0.15`: 55 detections, 9 matches, 46 false positives, 365 false negatives, precision `0.16363636363636364`, recall `0.02406417112299465`, F1 `0.04195804195804196`.
+- Conclusion: `yolov8s` is cleaner than `yolov8n` on some samples, but still misses most GRB buildings; it is not a sufficient V1 default.
+
 ## Sprint 127 Multi-sample detection quality calibration tooling (2026-07-07)
 
 - Added `scripts/prepare_operator_real_data_samples.py` to prepare documented Geel, Mol and Turnhout orthophoto/GRB GBG building sample pairs as explicit runtime artifacts.

docs/CODEX_EXECUTION_LOG.md

@@ -1,3 +1,47 @@
+## Sprint 128 Stronger building model runtime benchmark (2026-07-07)
+
+Changed:
+- Added `keremberke/yolov8s-building-segmentation` as an explicit runtime model asset on Tower:
+  - path: `/mnt/user/appdata/geointel/models/yolov8s-building-segmentation.pt`
+  - source: `https://huggingface.co/keremberke/yolov8s-building-segmentation/resolve/main/best.pt`
+  - SHA256: `a27af31654c6a4edbdc85581c33d93c13986b5919de7de410f8d85d801b3bb34`
+- Did not commit model weights to Git and did not add app-side model download behavior.
+- Ran a stronger-candidate multi-sample matrix comparing:
+  - `yolov8n-building-segmentation-pt`
+  - `yolov8s-building-segmentation-pt`
+  - samples: Geel, Mol, Turnhout
+  - tile sizes: `512`, `640`
+  - overlap: `64`
+  - thresholds: `0.50`, `0.25`, `0.15`
+
+Tested:
+- Hugging Face API metadata check showed `keremberke/yolov8s-building-segmentation` is public, uses the Ultralytics library, exposes `best.pt`, and reports validation mAP@0.5 values for the `keremberke/satellite-building-segmentation` dataset.
+- Live model asset catalog reported `yolov8s-building-segmentation-pt`, `size_bytes=23814274`, `will_download_models=false`, SHA256 `a27af31654c6a4edbdc85581c33d93c13986b5919de7de410f8d85d801b3bb34`.
+- Live preflight passed with `model_asset_id=yolov8s-building-segmentation-pt`, `check_model_load=true`, `model_file_exists=true`, `model_load_ok=true`, `will_download_models=false` and `will_run_inference=false`.
+- Tower 36-run matrix passed:
+  - command: `OPERATOR_SAMPLE_MANIFEST_PATH=storage/operator-data/operator_samples_manifest.json QUALITY_MODEL_ASSET_IDS="yolov8n-building-segmentation-pt yolov8s-building-segmentation-pt" QUALITY_TILE_SIZES="512 640" QUALITY_TILE_OVERLAPS="64" QUALITY_THRESHOLDS="0.50 0.25 0.15" bash scripts/run_multi_sample_detection_quality_matrix.sh http://192.168.10.150:1202`
+  - output: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/20260707T025932Z/multi_sample_quality_summary.json`
+  - best overall score: Mol, `yolov8s-building-segmentation-pt`, tile `640`, threshold `0.15`, 55 detections, 9 matches, 46 false positives, 365 false negatives, precision `0.16363636363636364`, recall `0.02406417112299465`, F1 `0.04195804195804196`.
+  - best overall recall: Mol, `yolov8s-building-segmentation-pt`, tile `512`, threshold `0.15`, 77 detections, 9 matches, 68 false positives, 365 false negatives, precision `0.11688311688311688`, recall `0.02406417112299465`, F1 `0.03991130820399113`.
+  - best overall precision: Turnhout, `yolov8s-building-segmentation-pt`, tile `640`, threshold `0.25`, 20 detections, 8 matches, 12 false positives, 765 false negatives, precision `0.4`, recall `0.01034928848641656`, F1 `0.0201765447667087`.
+  - best Geel score: `yolov8s-building-segmentation-pt`, tile `640`, threshold `0.15`, F1 `0.0187207488299532`.
+  - best Mol score: `yolov8s-building-segmentation-pt`, tile `640`, threshold `0.15`, F1 `0.04195804195804196`.
+  - best Turnhout score remained `yolov8n-building-segmentation-pt`, tile `512`, threshold `0.15`, F1 `0.03934426229508197`.
+- Best-score evidence export passed for every sample in the new matrix:
+  - Geel evidence: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/20260707T025932Z/geel/calibration_evidence_review.html`, 641 features, 611 false negatives, 18 false positives, 6 matched detections and 6 matched references.
+  - Mol evidence: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/20260707T025932Z/mol/calibration_evidence_review.html`, 429 features, 365 false negatives, 46 false positives, 9 matched detections and 9 matched references.
+  - Turnhout evidence: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/20260707T025932Z/turnhout/calibration_evidence_review.html`, 915 features, 755 false negatives, 124 false positives, 18 matched detections and 18 matched references.
+
+Open:
+- None for adding and benchmarking the `yolov8s` candidate.
+
+Limitations:
+- `yolov8s` improves precision and F1 on Geel/Mol but still misses most reference buildings. Best recall is about 2.4%, which is not sufficient for a usable V1 building extraction default.
+- The model is a runtime artifact and remains outside Git.
+
+Next recommended pass:
+- Source or train a materially stronger aerial-building model. The GeoIntel pipeline is now good enough to benchmark candidates quickly, but the current public YOLO building-segmentation candidates are still too weak for the target Flemish orthophoto/GRB workflow.
+
 ## Sprint 127 Multi-sample detection quality calibration tooling (2026-07-07)
 
 Changed:

docs/TODO.md

@@ -100,7 +100,9 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Add real-data detection quality matrix tooling for model/tile/threshold comparison.
 - [x] Add reproducible Geel/Mol/Turnhout operator sample preparation and multi-sample quality matrix tooling.
 - [x] Run first Geel/Mol/Turnhout persisted detection quality baseline.
+- [x] Add and benchmark a stronger `yolov8s` building-segmentation runtime model candidate.
 - [ ] Calibrate confidence, IoU and model selection against persisted Geel/Mol/Turnhout detections and any additional local orthophoto/reference samples.
+- [ ] Find or train a materially stronger aerial/Kempen building model candidate; current `yolov8s` recall remains too low for a V1 default.
 
 ## Sprint 8 status