Document YOLOv8s benchmark status

CHANGELOG.md

@@ -1283,3 +1283,12 @@ Added:
 - Segmentation now distinguishes configured model state from UI-runnable state and blocks the explicit test/demo-only fixture segmenter in the normal run form.
 - Added regression coverage for AI Lab action guardrails and compact guardrail styling.
 - No API contracts, migrations, product capabilities, live provider fetching or AI/model dependency changes were introduced.
+
+## Operator YOLOv8s hard-negative benchmark (2026-07-08)
+
+- Trained a Tower-local YOLOv8s hard-negative building detector from the existing operator tile dataset.
+- Published the trained runtime artifact as `geointel-building-yolov8s-hardneg160r4e50-pt` in the live model asset catalog without adding application download behavior.
+- Reused persisted dense QA and hard-negative benchmark runs through the existing live API.
+- Observed dense QA F1 scores up to `0.6380` and safest current threshold behavior around `0.25`.
+- Kept the model inactive by default because the `kasterlee_bos` hard-negative sample still produced 10 detections at threshold `0.25`.
+- No repository code, API contracts, migrations, product behavior, provider fetching or AI dependency strategy changed in this benchmark pass.

docs/CODEX_EXECUTION_LOG.md

@@ -5000,6 +5000,55 @@ Limitations:
 Next recommended pass:
 - Continue with V1 usability work that reduces operator confusion without expanding frozen product scope.
 
+## Operator YOLOv8s hard-negative model benchmark (2026-07-08)
+
+Changed:
+- No repository code, API contract, migration, UI or application behavior was changed in this pass.
+- Downloaded the official Ultralytics YOLOv8s base model manually as an operator/runtime asset on Tower at `/mnt/user/appdata/geointel/models/yolov8s.pt`.
+- Trained a local hard-negative building detector on Tower from the existing exported operator tile dataset `yolo-building-tile-hardneg160r4`.
+- Produced the trained runtime model artifact `/mnt/user/appdata/geointel/models/geointel-building-yolov8s-hardneg160r4e50.pt`, mounted in the container as `/app/models/geointel-building-yolov8s-hardneg160r4e50.pt`.
+- Verified the live model catalog exposes the trained model as `geointel-building-yolov8s-hardneg160r4e50-pt` with SHA256 `9bf71ad4742048ac77f07060b677bacd9757b8d310497fcada334d543e320d19`, size `22473194`, `will_download_models=false` and `active=false`.
+- Reused existing persisted dense QA and hard-negative benchmark runs through the live API; no external provider fetching and no fake data were introduced.
+
+Training evidence:
+- Base model: `/app/models/yolov8s.pt`.
+- Base model SHA256: `1f47a78bf100391c2a140b7ac73a1caae18c32779be7d310658112f7ac9aa78a`.
+- Training run: `/app/storage/training/operator-yolo/geointel-building-yolov8s-hardneg160r4e50`.
+- Best checkpoint copied to: `/app/models/geointel-building-yolov8s-hardneg160r4e50.pt`.
+- Final validation from Ultralytics: precision `0.449`, recall `0.404`, mAP50 `0.322`, mAP50-95 `0.110`.
+
+Live benchmark evidence:
+- Created 24 persisted rescored detection runs with `model_asset_id=geointel-building-yolov8s-hardneg160r4e50-pt`.
+- Dense persisted QA matrix results at IoU `0.1`:
+  - Four dense benchmark scenes saturated at 300 detections across thresholds `0.05`, `0.15` and `0.25`.
+  - Dense F1 scores observed: `0.6380`, `0.5627`, `0.5247`, `0.4678`.
+  - Dense precision ranged from `0.7167` to `0.8600`; recall ranged from `0.3247` to `0.5749`.
+  - Mean IoU ranged from `0.3962` to `0.4465`.
+  - The sparse/forest scene improved as the threshold increased: `49` detections and F1 `0.1429` at `0.05`; `18` detections and F1 `0.3200` at `0.15`; `10` detections and F1 `0.4706` at `0.25`.
+- Hard-negative detection counts:
+  - `kasterlee_bos`: `49` at threshold `0.05`, `18` at `0.15`, `10` at `0.25`.
+  - `lommel_heide`: `0` at thresholds `0.05`, `0.15` and `0.25`.
+  - `postel_bos`: `1` at threshold `0.05`, `0` at `0.15` and `0.25`.
+
+Assessment:
+- The YOLOv8s hard-negative model is a materially better evaluation candidate than the previous tiny smoke models and is useful in dense building scenes.
+- The model should not be made the silent default yet because `kasterlee_bos` still produces 10 hard-negative detections at threshold `0.25`.
+- Threshold `0.25` is the safest observed operating point for the current candidate.
+- The next product step should be model-selection and threshold workflow hardening before any operator-facing default activation.
+
+Operational issue:
+- Tower root SSH became unavailable after training. The key `widefrog_unraid_deploy` was offered, but the server returned `Permission denied (publickey)`.
+- This blocks Codex-driven host-side deploy/sync until the Tower root `authorized_keys` entry is restored.
+- Live HTTP API verification on `http://192.168.10.150:1202` remained reachable.
+
+Open:
+- Restore root SSH access on Tower for `widefrog_unraid_deploy`.
+- Add an operator-facing model selection/activation flow or documented command that can promote a chosen local model deliberately, with visible SHA256 and threshold guidance.
+- Add more negative/background AOIs and a threshold calibration benchmark before activating this model by default.
+
+Next recommended pass:
+- Implement V1 model-catalog hardening: show available local model assets, make active model/threshold explicit, and prevent accidental silent default activation.
+
 ## Sprint 117 Safe local YOLO model activation (2026-07-06)
 
 Changed:

docs/TODO.md

@@ -415,5 +415,9 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Train/evaluate a small local GeoIntel building-detector smoke from the current operator samples and reject it because QA/QC did not improve.
 - [x] Add operator-only tile-level YOLO dataset export with overlapping windows and deterministic negative tile retention.
 - [x] Run tile-level training on Tower and accept/reject the resulting local model through the persisted QA/QC matrix.
+- [x] Train/evaluate a YOLOv8s hard-negative local building-detector candidate on Tower and keep it inactive because hard-negative false positives remain.
+- [ ] Restore Tower root SSH access for the `widefrog_unraid_deploy` key so Codex can deploy/sync again from the repo.
+- [ ] Add an operator-facing local model catalog/activation workflow with SHA256, active model status and explicit threshold guidance.
+- [ ] Add threshold calibration UX so detection runs do not silently rely on an unsafe default confidence.
 - [ ] Add more AOIs after the tile-level baseline so the next local model attempt is not limited to Geel/Mol/Turnhout.
 - [ ] Add negative/background AOIs so the next tile dataset is not all positive tiles.

scripts/README.md

@@ -486,6 +486,19 @@ files are present without `--model-file`. It writes only
 `GEOINTEL_INSTALL_AI=true`, `YOLO_ENABLED=true`, `YOLO_MODELS_DIR=/app/models`
 and the mounted `YOLO_MODEL_PATH`.
 
+Tower-local model evaluation status:
+
+- `geointel-building-yolov8s-hardneg160r4e50.pt` is available as an evaluated
+  local runtime artifact after the hard-negative YOLOv8s training pass.
+- The live catalog id is `geointel-building-yolov8s-hardneg160r4e50-pt`.
+- The model SHA256 is
+  `9bf71ad4742048ac77f07060b677bacd9757b8d310497fcada334d543e320d19`.
+- The current safest observed threshold is `0.25`, but the model remains an
+  evaluation candidate because one hard-negative forest sample still produced
+  false detections at that threshold.
+- Do not silently activate this model as a default. Apply it only as an explicit
+  operator choice until the model catalog/threshold workflow is hardened.
+
 Clean old offline demo export artifacts without touching uploaded source data:
 
 ```bash