Record AOI1024 clean YOLO candidate gate

CHANGELOG.md

@@ -7,6 +7,16 @@
 
 # Changelog
 
+## Sprint 153 AOI1024 clean-label YOLOv8s candidate gate (2026-07-09)
+
+- Audited AOI1024 label-quality variants after paged GRB reference regeneration and selected `/app/storage/operator-data/yolo-building-aoi1024-visible050-minpx8` for training because it passed the dataset audit while keeping the 512px runtime scale.
+- Trained inactive local model asset `geointel-building-yolov8s-aoi1024clean512e50-pt` from the cleaned 512px tile dataset. The Tower catalog reports SHA256 `7cfadb684dd56623d2e35ebd65593211d3051908c87121bef438b231d3e47cce`, `status=available`, `active=false` and `will_download_models=false`.
+- Ultralytics validation for the completed 50-epoch CPU run ended at approximately precision `0.441`, recall `0.380`, mAP50 `0.265` and mAP50-95 `0.096`.
+- Ran the full seven-reference AOI1024 persisted QA matrix at tile `512`, overlap `64` and thresholds `0.25`, `0.15` and `0.05`. Mean positive F1 improved materially: `0.4723513253430784` at threshold `0.05`, `0.4753322215541376` at `0.15` and `0.31021575247724875` at `0.25`.
+- Ran the full nine-sample hard-negative/background matrix. Background false-positive pressure still blocks default promotion: max detections were `137` at threshold `0.05`, `75` at `0.15` and `55` at `0.25`.
+- Generated the promotion report at `artifacts/detection-model-promotion/aoi1024clean512e50-full/detection_model_promotion_report.md`; recommendation remains `none` because every threshold fails `background_false_positive_pressure`.
+- No API contract, migration, provider fetching, fake detection data, model download behavior or active model default changed.
+
 ## Sprint 152 GRB reference paging for operator samples (2026-07-09)
 
 - Fixed the operator real-data sample preparer so GRB GBG reference GeoJSON is fetched through OGC API `rel=next` pagination links instead of stopping at the first `limit=1000` page.

docs/CODEX_EXECUTION_LOG.md

@@ -6041,3 +6041,58 @@ Open:
 ## Next recommended pass
 
 - Fix dense GRB reference completeness first: add provider-side paging or split dense AOIs so reference exports do not cap at 1000 features, then regenerate AOI1024 labels before another training run.
+
+# Sprint 153 - AOI1024 clean-label YOLOv8s candidate gate
+
+## What changed
+
+- Built and audited AOI1024 label-quality candidate exports on Tower after the paged GRB reference regeneration:
+  - baseline `yolo-building-aoi1024-visible025`: 144 tiles, 117 positive, 27 negative, 29,170 labels, audit `needs_attention`, median normalized box area `0.000762939453125`, small-box share `0.34744600617072335`.
+  - `yolo-building-aoi1024-visible050`: 144 tiles, 117 positive, 27 negative, 28,552 labels, audit `needs_attention`, small-box share `0.34463435135892406`.
+  - `yolo-building-aoi1024-visible025-tile640`: 45,529 labels, audit `needs_attention`, small-box share `0.4952667530584902`.
+  - `yolo-building-aoi1024-visible025-tile384`: 400 tiles, 311 positive, 89 negative, 46,423 labels, audit `ok`, small-box share `0.215281218361588`.
+  - `yolo-building-aoi1024-visible050-minpx8`: 144 tiles, 117 positive, 27 negative, 21,530 labels, audit `ok`, median normalized box area `0.0010299684375`, small-box share `0.14681839294008361`.
+- Selected `/app/storage/operator-data/yolo-building-aoi1024-visible050-minpx8` for the next controlled candidate because it passes the label audit while preserving the 512px runtime scale.
+- Trained Tower-local inactive model asset:
+  - model path: `/app/models/geointel-building-yolov8s-aoi1024clean512e50.pt`
+  - asset id: `geointel-building-yolov8s-aoi1024clean512e50-pt`
+  - SHA256: `7cfadb684dd56623d2e35ebd65593211d3051908c87121bef438b231d3e47cce`
+  - training summary: `/app/storage/training/operator-yolo/geointel-building-yolov8s-aoi1024clean512e50/training_summary.json`
+  - base model: `/app/models/yolov8s.pt`
+  - epochs: 50
+  - image size: 512
+  - batch size: 2
+  - device: CPU
+  - final Ultralytics validation on `best.pt`: precision `0.441`, recall `0.380`, mAP50 `0.265`, mAP50-95 `0.096`.
+- Verified the live model catalog reports the candidate as `status=available`, `active=false` and `will_download_models=false`.
+
+## What was tested
+
+- Tower training command:
+  - `docker exec -e OPERATOR_YOLO_DATASET_DIR=/app/storage/operator-data/yolo-building-aoi1024-visible050-minpx8 -e YOLO_BASE_MODEL_PATH=/app/models/yolov8s.pt -e TRAIN_OUTPUT_DIR=/app/storage/training/operator-yolo -e TRAIN_RUN_NAME=geointel-building-yolov8s-aoi1024clean512e50 -e TRAIN_MODEL_OUTPUT_PATH=/app/models/geointel-building-yolov8s-aoi1024clean512e50.pt -e TRAIN_EPOCHS=50 -e TRAIN_IMGSZ=512 -e TRAIN_BATCH=2 -e TRAIN_WORKERS=0 -e TRAIN_DEVICE=cpu geointel bash /app/scripts/train_operator_yolo_detector.sh`
+- Full seven-reference AOI1024 persisted positive QA matrix:
+  - command used `scripts/run_multi_sample_detection_quality_matrix.sh` with manifest `/mnt/user/appdata/geointel/storage/operator-data/operator-samples-1024/operator_samples_manifest.json`, samples `geel mol turnhout herentals balen retie westerlo`, model `geointel-building-yolov8s-aoi1024clean512e50-pt`, tile `512`, overlap `64`, thresholds `0.25 0.15 0.05` and IoU threshold `0.25`.
+  - output: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/aoi1024clean512e50-full/multi_sample_quality_summary.json`
+  - threshold `0.05`: 7 positive samples, mean F1 `0.4723513253430784`, mean precision `0.44861169484563357`, mean recall `0.5039340859731523`, min F1 `0.4442082890541977`.
+  - threshold `0.15`: 7 positive samples, mean F1 `0.4753322215541376`, mean precision `0.6673270868402211`, mean recall `0.37207511021815726`, min F1 `0.4055555555555556`.
+  - threshold `0.25`: 7 positive samples, mean F1 `0.31021575247724875`, mean precision `0.8163316419273472`, mean recall `0.19608909809008596`, min F1 `0.16430903155603915`.
+- Full nine-sample hard-negative/background matrix:
+  - command used `scripts/run_operator_hard_negative_detection_matrix.sh` with the same AOI1024 manifest, model `geointel-building-yolov8s-aoi1024clean512e50-pt`, tile `512`, overlap `64` and thresholds `0.25 0.15 0.05`.
+  - output: `/mnt/user/appdata/geointel/artifacts/detection-hard-negatives/aoi1024clean512e50-full/hard_negative_matrix_summary.json`
+  - threshold `0.05`: total background detections `488`, max sample detections `137`.
+  - threshold `0.15`: total background detections `269`, max sample detections `75`.
+  - threshold `0.25`: total background detections `205`, max sample detections `55`.
+- Promotion report:
+  - command used `scripts/build_detection_model_promotion_report.py` with `--min-positive-samples 7`, `--min-background-samples 9`, `--min-mean-f1 0.25` and `--max-background-detections-per-sample 0`.
+  - output: `/mnt/user/appdata/geointel/artifacts/detection-model-promotion/aoi1024clean512e50-full/detection_model_promotion_report.md`
+  - recommendation: `none`.
+
+## Known limitations
+
+- The clean-label model is materially stronger on positive reference AOIs than the previous AOI1024-visible025 candidate, but it is still not safe as a V1 default because hard-negative/background false positives remain high in mixed wooded AOIs.
+- The likely bottleneck has shifted from label-file integrity to discriminating buildings from visually similar background structures and wooded-edge artifacts.
+- The model remains inactive. No default model, API contract, migration, provider fetching or download behavior changed.
+
+## Next recommended pass
+
+- Build a background-aware training pass instead of another positive-only clean-label pass: combine the `visible050-minpx8` positives with stronger unique hard-negative/background retention, then re-run the same seven-positive/nine-background promotion gate before any activation.

docs/TODO.md

@@ -116,10 +116,12 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Rerun live dense-AOI calibration after redeploy with `YOLO_DUPLICATE_IOU_THRESHOLD=0.5`; Westerlo 0.25 improved to F1 `0.2537313432835821` and Turnhout 0.25 improved to F1 `0.14114114114114112`, but the candidate remains rejected.
 - [x] Add `OPERATOR_YOLO_MIN_LABEL_VISIBLE_RATIO` so the next overlapping-tile dataset can drop tiny clipped edge-fragment labels.
 - [x] Add operator-only larger-AOI sample prep flags so the next training dataset is not limited to one 512x512 tile per documented sample.
+- [x] Regenerate Tower AOI1024 operator samples with paged GRB references and verify no dense reference sample is capped at 1000 features.
+- [x] Export and audit AOI1024 clean-label variants; select `yolo-building-aoi1024-visible050-minpx8` as the first audit-passing 512px training candidate.
+- [x] Train and gate `geointel-building-yolov8s-aoi1024clean512e50-pt` through seven positive AOIs and nine hard-negative/background AOIs.
 - [ ] Find or train a materially stronger aerial/Kempen building model candidate; `geointel-building-yolov8n-expanded160e50-pt` is the best current dense-AOI candidate but still too weak and too noisy for a V1 default.
-- [ ] Train a higher-capacity local aerial-building detector with stronger positive recall while preserving the hard-negative false-positive gate.
-- [ ] Prepare `/app/storage/operator-data/operator-samples-1024` on Tower, then export and audit `yolo-building-aoi1024-visible025` before the next default-model training attempt.
-- [ ] Build the next candidate gate around better positive AOI coverage, label strategy and hard-negative retention.
+- [ ] Train a background-aware local aerial-building detector with stronger positive recall while reducing hard-negative false-positive pressure.
+- [ ] Build the next candidate gate around cleaner positive labels plus stronger unique hard-negative retention.
 
 ## Sprint 8 status