From c4a2e13d43e53dd370ea722eb32160e11e5b1d65 Mon Sep 17 00:00:00 2001 From: Codex Date: Thu, 9 Jul 2026 11:02:33 +0200 Subject: [PATCH] Record YOLO duplicate suppression calibration --- CHANGELOG.md | 1 + docs/CODEX_EXECUTION_LOG.md | 17 +++++++++++++++-- docs/TODO.md | 3 ++- 3 files changed, 18 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 52900fbd..8df97f37 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,7 @@ - Detection run result metadata now records raw candidate count, suppressed duplicate count and duplicate IoU threshold. - Calibration and quality matrix scripts now fetch detection run details and include raw/suppressed counts in summaries. - Updated Docker/Unraid env examples, API/AI/backend docs and detection pipeline notes. +- Redeployed Tower and reran Westerlo/Turnhout dense-AOI sweeps; duplicate suppression improved F1 but the AOI512 YOLOv8s candidate remains rejected for default use. - No model was activated, no detections were faked, and no migration changed. ## Sprint 148 YOLO max-detection cap hardening (2026-07-09) diff --git a/docs/CODEX_EXECUTION_LOG.md b/docs/CODEX_EXECUTION_LOG.md index 5c9e6704..242c4d48 100644 --- a/docs/CODEX_EXECUTION_LOG.md +++ b/docs/CODEX_EXECUTION_LOG.md @@ -26,9 +26,22 @@ Tested: - `bash -n scripts/run_detection_calibration_sweep.sh` - `bash -n scripts/run_detection_quality_matrix.sh` - `bash -n scripts/run_multi_sample_detection_quality_matrix.sh` +- `bash scripts/run_readiness_check.sh` (`426 passed`, frontend typecheck/build passed) +- Redeployed Tower all-in-one image with AI dependencies and verified `YOLO_MAX_DETECTIONS=1000` plus `YOLO_DUPLICATE_IOU_THRESHOLD=0.5` in the live container. +- Tower live migration smoke and browser runtime verification passed. +- Live Westerlo calibration with `geointel-building-yolov8s-aoi512e80-pt`: + - `0.25`: 202 persisted / 270 raw / 68 suppressed, F1 `0.2537313432835821` + - `0.15`: 328 persisted / 523 raw / 195 suppressed, F1 `0.22054380664652568` + - `0.05`: 586 persisted / 1000 raw / 414 suppressed, F1 `0.17173913043478262` +- Live Turnhout calibration with `geointel-building-yolov8s-aoi512e80-pt`: + - `0.25`: 559 persisted / 822 raw / 263 suppressed, F1 `0.14114114114114112` + - `0.15`: 659 persisted / 1000 raw / 341 suppressed, F1 `0.14385474860335196` + - `0.05`: 659 persisted / 1000 raw / 341 suppressed, F1 `0.14385474860335196` -Next: -- Run full readiness, then deploy Tower and rerun the dense Westerlo/Turnhout sweeps with duplicate suppression enabled to quantify quality impact versus the Sprint 148 uncapped baseline. +Conclusion: +- Cross-tile duplicate suppression reduces false-positive pressure and improves F1 versus the Sprint 148 uncapped baseline on the checked dense AOIs. +- The current AOI512 YOLOv8s candidate remains rejected for default use because recall/precision are still too low after post-processing. +- Next model work should focus on stronger training data/label strategy and a new candidate gate, not further default threshold lowering. ## Sprint 148 YOLO max-detection cap hardening (2026-07-09) diff --git a/docs/TODO.md b/docs/TODO.md index 6b7f91a5..c04c6a6f 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -113,10 +113,11 @@ This file now starts with the current implementation status. Older preparation/b - [x] Raise configured-YOLO `max_det` through `YOLO_MAX_DETECTIONS` so dense AOIs are not capped at 300 detections before QA/QC. - [x] Rerun live dense-AOI calibration after redeploy with `YOLO_MAX_DETECTIONS=1000`; Westerlo reached 523/1000 detections at lower thresholds and Turnhout reached 822/1000, confirming the old 300 cap is removed. - [x] Add configured-YOLO cross-tile duplicate suppression and raw/suppressed calibration evidence fields. +- [x] Rerun live dense-AOI calibration after redeploy with `YOLO_DUPLICATE_IOU_THRESHOLD=0.5`; Westerlo 0.25 improved to F1 `0.2537313432835821` and Turnhout 0.25 improved to F1 `0.14114114114114112`, but the candidate remains rejected. - [ ] Find or train a materially stronger aerial/Kempen building model candidate; `geointel-building-yolov8n-expanded160e50-pt` is the best current dense-AOI candidate but still too weak and too noisy for a V1 default. - [ ] Train a higher-capacity local aerial-building detector with stronger positive recall while preserving the hard-negative false-positive gate. - [ ] Add more diverse positive AOIs and revisit geometry-to-box label strategy before the next default-model training attempt. -- [ ] Rerun live dense-AOI calibration after redeploy with `YOLO_DUPLICATE_IOU_THRESHOLD=0.5` to measure post-processing impact versus the Sprint 148 uncapped baseline. +- [ ] Build the next candidate gate around better positive AOI coverage, label strategy and hard-negative retention. ## Sprint 8 status