Document remote-sensing YOLO candidate benchmark

CHANGELOG.md

@@ -7,6 +7,18 @@
 
 # Changelog
 
+## Sprint 134 External remote-sensing YOLO candidate benchmark (2026-07-07)
+
+- Evaluated the Hugging Face `agademer/yolo-remote-sensing-photovoltaic` YOLOv8l detection checkpoint as an explicit operator-provided runtime model asset.
+- Downloaded `yolo-remote-sensing-photovoltaic-v8l-solar-farms-and-cities-v20260331-detect-1000_epochs.pt` to the Tower runtime as `/app/models/yolo-remote-sensing-photovoltaic-v8l-detect-1000.pt`; the model file is not committed to Git.
+- The model catalog exposes it as `yolo-remote-sensing-photovoltaic-v8l-detect-1000-pt` with SHA256 `242ff4ab889569278f0eb9fcd22eb2c4bf2a52e48d05d89cc7cfa7941165d203`.
+- Live YOLO preflight loaded the model successfully with `status=ready`, `model_load_ok=true`, `manifest_valid=true`, `tile_paths_exist=true`, `will_download_models=false` and `will_run_inference=false`.
+- Live 45-run dense QA matrix compared the external YOLOv8l candidate with `geointel-building-yolov8n-expanded160e50-pt` and `geointel-building-yolov8n-hardneg160r8e40-pt` across Geel, Mol, Turnhout, Retie and Kasterlee-bos.
+- Result: the external candidate was very conservative and missed most dense GRB buildings. It scored F1 `0.0` on Geel, `0.010582010582010581` on Mol, `0.019070321811680575` on Turnhout and `0.0` on Retie, while expanded160e50 remained the dense-AOI winner.
+- Live 27-run background matrix showed the external candidate was cleaner on Kasterlee-bos than local YOLOv8n candidates, with 1/2/5 detections at thresholds `0.25`/`0.15`/`0.05`, but it leaked 0/1/3 detections on Postel-bos and was therefore not uniformly cleaner than hardneg160r8e40.
+- Decision: keep the model as runtime evidence only. It should not become the V1 default because recall is too low for operational extraction. The next pass should train a higher-capacity local model, starting from a stronger base and using the existing dense plus hard-negative benchmark gates.
+- No API contract change, provider fetching, fake detections, model auto-provisioning, repository-stored weights or app-side model training behavior was introduced.
+
 ## Sprint 133 Hard-negative-balanced YOLO candidate (2026-07-07)
 
 - Added `--background-negative-repeat` / `OPERATOR_YOLO_BACKGROUND_NEGATIVE_REPEAT` support to `scripts/export_operator_yolo_tile_dataset.py` so train-split background-candidate negative tiles can be repeated deterministically without duplicating validation tiles.

docs/CODEX_EXECUTION_LOG.md

@@ -1,3 +1,45 @@
+## Sprint 134 External remote-sensing YOLO candidate benchmark (2026-07-07)
+
+Changed:
+- Added no production code and no repository-stored model weights.
+- Evaluated the Hugging Face `agademer/yolo-remote-sensing-photovoltaic` model card as a candidate because it explicitly targets remote-sensing imagery and includes `building` as a class.
+- Downloaded the YOLOv8l detection checkpoint manually as an operator/runtime asset:
+  - source model: `agademer/yolo-remote-sensing-photovoltaic`
+  - source file: `yolo-remote-sensing-photovoltaic-v8l-solar-farms-and-cities-v20260331-detect-1000_epochs.pt`
+  - runtime file: `/app/models/yolo-remote-sensing-photovoltaic-v8l-detect-1000.pt`
+  - catalog asset: `yolo-remote-sensing-photovoltaic-v8l-detect-1000-pt`
+  - SHA256: `242ff4ab889569278f0eb9fcd22eb2c4bf2a52e48d05d89cc7cfa7941165d203`
+- Updated `scripts/README.md`, `docs/TODO.md`, `docs/CODEX_EXECUTION_LOG.md` and `CHANGELOG.md` with benchmark evidence and the decision not to promote this model.
+
+Tested:
+- Live API preflight passed for `yolo-remote-sensing-photovoltaic-v8l-detect-1000-pt` with `status=ready`, `model_load_ok=true`, `manifest_valid=true`, `tile_paths_exist=true`, `will_download_models=false` and `will_run_inference=false`.
+- Live 45-run multi-sample QA matrix completed:
+  - output: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/remote-sensing-v8l1000-live/multi_sample_quality_summary.json`
+  - command compared `yolo-remote-sensing-photovoltaic-v8l-detect-1000-pt`, `geointel-building-yolov8n-expanded160e50-pt` and `geointel-building-yolov8n-hardneg160r8e40-pt` over Geel, Mol, Turnhout, Retie and Kasterlee-bos with tile `640`, overlap `64`, thresholds `0.25`/`0.15`/`0.05`.
+  - dense-sample results for the external candidate:
+    - Geel: best F1 `0.0`
+    - Mol: best F1 `0.010582010582010581`
+    - Turnhout: best F1 `0.019070321811680575`
+    - Retie: best F1 `0.0`
+  - Kasterlee-bos sparse result: best F1 `0.25` with 1 detection, 1 match and 0 false positives at thresholds `0.25`/`0.15`.
+  - dense-sample winner remained `geointel-building-yolov8n-expanded160e50-pt`.
+- Live 27-run hard-negative matrix completed:
+  - output: `/mnt/user/appdata/geointel/artifacts/detection-hard-negatives/remote-sensing-v8l1000-live/hard_negative_matrix_summary.json`
+  - Postel-bos: external candidate produced 0/1/3 detections at thresholds `0.25`/`0.15`/`0.05`.
+  - Lommel-heide: external candidate produced 0/0/0 detections.
+  - Kasterlee-bos: external candidate produced 1/2/5 detections, cleaner than expanded160e50 and hardneg160r8e40 on that sparse AOI.
+
+Open:
+- None for this benchmark pass.
+
+Limitations:
+- The external model is very conservative on the current Kempen orthophoto/GRB benchmark and misses most dense buildings.
+- It may be useful as evidence for a high-precision/sparse-review mode, but it is not a viable V1 default extraction model.
+- The runtime `.pt` remains outside Git and must be managed as an operator asset.
+
+Next recommended pass:
+- Train a higher-capacity local detector from a stronger base model using the existing operator tile dataset, then gate it with both dense QA and hard-negative matrices before any model default changes.
+
 ## Sprint 133 Hard-negative-balanced YOLO candidate (2026-07-07)
 
 Changed:

docs/TODO.md

@@ -107,7 +107,9 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Add negative/background AOIs to the operator sample corpus and train an expanded local tile-level YOLO candidate.
 - [x] Add a hard-negative model-quality pass with sparse/background AOIs and explicit false-positive scoring.
 - [x] Train a hard-negative-balanced YOLO candidate and rerun dense QA plus background false-positive matrices.
+- [x] Benchmark an external remote-sensing YOLOv8l building candidate as an explicit local model asset.
 - [ ] Find or train a materially stronger aerial/Kempen building model candidate; `geointel-building-yolov8n-expanded160e50-pt` is the best current dense-AOI candidate but still too weak and too noisy for a V1 default.
+- [ ] Train a higher-capacity local aerial-building detector with stronger positive recall while preserving the hard-negative false-positive gate.
 
 ## Sprint 8 status
 

scripts/README.md

@@ -416,6 +416,25 @@ Lommel-heide across all tested thresholds. It also regressed dense-AOI F1
 against `geointel-building-yolov8n-expanded160e50-pt`, so it is useful model
 quality evidence but not a V1 default.
 
+An external remote-sensing YOLOv8l candidate was also benchmarked as an
+operator-provided local model asset:
+
+```bash
+mkdir -p models
+curl -L --fail \
+  -o models/yolo-remote-sensing-photovoltaic-v8l-detect-1000.pt \
+  https://huggingface.co/agademer/yolo-remote-sensing-photovoltaic/resolve/main/yolo-remote-sensing-photovoltaic-v8l-solar-farms-and-cities-v20260331-detect-1000_epochs.pt
+```
+
+GeoIntel exposed the file as
+`yolo-remote-sensing-photovoltaic-v8l-detect-1000-pt` with SHA256
+`242ff4ab889569278f0eb9fcd22eb2c4bf2a52e48d05d89cc7cfa7941165d203`, and
+YOLO preflight loaded it without downloads. On the live dense matrix it missed
+most buildings and scored far below `geointel-building-yolov8n-expanded160e50-pt`
+on Geel, Mol, Turnhout and Retie. On Kasterlee-bos it was clean and precise,
+but that sparse-AOI behavior is not enough for V1 extraction. Keep it as
+benchmark evidence only, not as a default model.
+
 Export calibration QA evidence for visual review:
 
 ```bash