Record operator tile model benchmark results

CHANGELOG.md

@@ -13,6 +13,11 @@
 - Added readiness coverage for the tile exporter Python compile check.
 - Added regression coverage in `backend/tests/test_sprint130_operator_yolo_tile_dataset.py` for script contract, help behavior without GIS imports, edge-covering tile windows and deterministic negative-tile selection.
 - Updated operator documentation for tile-level dataset export and reuse of the existing local training wrapper.
+- Live Tower tile export produced `/app/storage/operator-data/yolo-building-tile-dataset` with 75 overlapping tiles and 5321 clipped building labels from the Geel/Mol/Turnhout operator samples.
+- Live Tower 30-epoch CPU training produced `/app/models/geointel-building-yolov8n-tile30.pt`; the model catalog exposes it as `geointel-building-yolov8n-tile30-pt` with SHA256 `b9e228202500d7c85836d12a72e320f4f2f0cef24cbb1b5bf7fa78a6778390af`.
+- Live YOLO preflight loaded `geointel-building-yolov8n-tile30-pt` successfully with `status=ready`, `model_load_ok=true`, `manifest_valid=true`, `tile_paths_exist=true`, `will_download_models=false` and `will_run_inference=false`.
+- Live 48-run Geel/Mol/Turnhout QA matrix compared `geointel-building-yolov8n-tile30-pt` with `yolov8s-building-segmentation-pt`; best overall score was Mol with the tile model, tile `640`, threshold `0.15`, precision `0.13602941176470587`, recall `0.09893048128342247`, F1 `0.11455108359133127`.
+- Result decision: the tile-trained local model is now the best tested candidate on Geel/Mol and best overall, but remains experimental and should not become the V1 default until more AOIs and negative/background samples materially improve recall and false-positive behavior.
 - No Training Studio UI, API contract change, provider fetching, model auto-provisioning or app-side model training behavior was introduced.
 
 ## Sprint 129 Operator YOLO training dataset tooling (2026-07-07)

docs/CODEX_EXECUTION_LOG.md

@@ -13,15 +13,39 @@ Tested:
 - `python -m pytest backend\tests\test_sprint130_operator_yolo_tile_dataset.py -q` passed.
 - `python scripts\export_operator_yolo_tile_dataset.py --help` passed without requiring local GIS dependencies.
 - `python -m py_compile scripts\export_operator_yolo_tile_dataset.py` passed.
+- Live Tower tile export passed:
+  - dataset: `/app/storage/operator-data/yolo-building-tile-dataset`
+  - samples: Geel, Mol and Turnhout
+  - tile size: `192`
+  - stride: `96`
+  - exported tiles: `75`
+  - positive tiles: `75`
+  - labels: `5321`
+  - validation split: Turnhout
+- Live Tower 30-epoch CPU tile training passed:
+  - output model: `/app/models/geointel-building-yolov8n-tile30.pt`
+  - catalog asset: `geointel-building-yolov8n-tile30-pt`
+  - SHA256: `b9e228202500d7c85836d12a72e320f4f2f0cef24cbb1b5bf7fa78a6778390af`
+  - final validation: precision `0.208`, recall `0.271`, mAP50 `0.122`, mAP50-95 `0.0308`
+- Live API preflight passed for `geointel-building-yolov8n-tile30-pt` with `status=ready`, `model_load_ok=true`, `manifest_valid=true`, `tile_paths_exist=true`, `will_download_models=false` and `will_run_inference=false`.
+- Live 48-run multi-sample QA matrix completed:
+  - output: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/20260707T190720Z/multi_sample_quality_summary.json`
+  - command compared `geointel-building-yolov8n-tile30-pt` with `yolov8s-building-segmentation-pt` over Geel, Mol and Turnhout, tile sizes `512`/`640`, overlap `64`, thresholds `0.50`/`0.25`/`0.15`/`0.05`.
+  - best overall score: Mol, `geointel-building-yolov8n-tile30-pt`, tile `640`, threshold `0.15`, 272 detections, 37 matches, 235 false positives, 337 false negatives, precision `0.13602941176470587`, recall `0.09893048128342247`, F1 `0.11455108359133127`.
+  - best overall recall: Mol, `geointel-building-yolov8n-tile30-pt`, tile `512`, threshold `0.05`, 544 detections, 44 matches, 500 false positives, 330 false negatives, precision `0.08088235294117647`, recall `0.11764705882352941`, F1 `0.09586056644880174`.
+  - best overall precision: Turnhout, `yolov8s-building-segmentation-pt`, tile `640`, threshold `0.25`, precision `0.4`, recall `0.01034928848641656`, F1 `0.0201765447667087`.
+  - per-sample score winners: Geel `geointel-building-yolov8n-tile30-pt` F1 `0.09671179883945842`; Mol `geointel-building-yolov8n-tile30-pt` F1 `0.11455108359133127`; Turnhout `yolov8s-building-segmentation-pt` F1 `0.09971777986829727`.
 
 Open:
-- Run the tile exporter inside the AI-enabled Tower runtime, train a local tile-level model, and benchmark it through the existing multi-sample Detection + QA matrix.
+- None for the tile exporter/training runtime proof itself.
 
 Limitations:
 - This remains operator tooling only. It does not add Training Studio, browser training controls, provider fetching, fake detections, model auto-provisioning or API contract changes.
+- `geointel-building-yolov8n-tile30-pt` is an experimental local candidate, not a V1 default. It improves the operator-trained baseline materially but still has low recall and many false positives on the current 3-sample corpus.
+- The current corpus is too small and all exported tiles were positive; the next model pass needs more AOIs and deliberate negative/background tiles.
 
 Next recommended pass:
-- Export the tile-level dataset on Tower with `tile-size=192`, `stride=96`, train a longer local model, and compare it against the current `yolov8s-building-segmentation-pt` baseline.
+- Expand the operator sample corpus beyond Geel/Mol/Turnhout, include negative/background AOIs, regenerate the tile dataset, then train a longer/larger local model candidate and rerun the same persisted QA matrix.
 
 ## Sprint 129 Operator YOLO training dataset tooling (2026-07-07)
 

docs/TODO.md

@@ -101,8 +101,10 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Add reproducible Geel/Mol/Turnhout operator sample preparation and multi-sample quality matrix tooling.
 - [x] Run first Geel/Mol/Turnhout persisted detection quality baseline.
 - [x] Add and benchmark a stronger `yolov8s` building-segmentation runtime model candidate.
-- [ ] Calibrate confidence, IoU and model selection against persisted Geel/Mol/Turnhout detections and any additional local orthophoto/reference samples.
+- [x] Add operator-only tile-level YOLO dataset export with overlapping windows and deterministic negative tile retention.
-- [ ] Find or train a materially stronger aerial/Kempen building model candidate; current `yolov8s` recall remains too low for a V1 default.
+- [x] Train and benchmark the first tile-level local YOLO candidate on Tower through the persisted QA/QC matrix.
+- [ ] Calibrate confidence, IoU and model selection against additional local orthophoto/reference samples beyond Geel/Mol/Turnhout.
+- [ ] Find or train a materially stronger aerial/Kempen building model candidate; `geointel-building-yolov8n-tile30-pt` is the best current overall candidate but still too weak for a V1 default.
 
 ## Sprint 8 status
 
@@ -407,5 +409,6 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Add operator-only YOLO dataset export and local training-smoke wrapper for real sample calibration.
 - [x] Train/evaluate a small local GeoIntel building-detector smoke from the current operator samples and reject it because QA/QC did not improve.
 - [x] Add operator-only tile-level YOLO dataset export with overlapping windows and deterministic negative tile retention.
-- [ ] Run tile-level training on Tower and accept/reject the resulting local model through the persisted QA/QC matrix.
+- [x] Run tile-level training on Tower and accept/reject the resulting local model through the persisted QA/QC matrix.
 - [ ] Add more AOIs after the tile-level baseline so the next local model attempt is not limited to Geel/Mol/Turnhout.
+- [ ] Add negative/background AOIs so the next tile dataset is not all positive tiles.