diff --git a/CHANGELOG.md b/CHANGELOG.md index a2983804..9fd13cfb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,8 @@ - Wired the setting through `.env.example`, Docker Compose, Unraid env examples and the Dockerman run script. - Documented why dense building AOIs should not inherit the Ultralytics default cap of 300 detections before persisted QA/QC. - Added regression coverage for adapter forwarding and Docker/Unraid runtime exposure. +- Redeployed the Tower all-in-one runtime and verified live dense-AOI sweeps can exceed 300 persisted detection candidates: Westerlo reached 523/1000 detections and Turnhout reached 822/1000 at tested thresholds. +- The current AOI512 YOLOv8s candidate remains rejected for default use because persisted QA/QC F1 remains too low despite the runtime cap fix. - No model was activated, no detections were faked, and no API route or migration changed. ## Sprint 147 AOI512 YOLOv8s scale-match candidate gate (2026-07-09) diff --git a/docs/CODEX_EXECUTION_LOG.md b/docs/CODEX_EXECUTION_LOG.md index f084ff63..e25290fd 100644 --- a/docs/CODEX_EXECUTION_LOG.md +++ b/docs/CODEX_EXECUTION_LOG.md @@ -22,10 +22,22 @@ Tested: - `python -m pytest backend\tests\test_sprint8b_yolo_foundation.py::test_yolo_adapter_converts_single_band_tiles_to_rgb_before_prediction backend\tests\test_sprint8b_yolo_foundation.py::test_yolo_adapter_uses_configured_max_detections -q` (`2 passed`) - Red step: Docker runtime config tests failed before `.env.example` and Unraid runner exposed `YOLO_MAX_DETECTIONS`. - `python -m pytest backend\tests\test_docker_runtime_config.py::test_env_example_uses_runtime_env_names_read_by_backend_and_frontend backend\tests\test_docker_runtime_config.py::test_unraid_deploy_passes_ai_build_arg_and_yolo_runtime_env backend\tests\test_sprint8b_yolo_foundation.py -q` (`16 passed`) +- `bash scripts/run_readiness_check.sh` (`425 passed`, frontend typecheck/build passed) +- Redeployed Tower all-in-one image with AI dependencies and verified `YOLO_MAX_DETECTIONS=1000` in the live container. +- Tower live migration smoke passed against embedded PostGIS. +- Live Westerlo calibration with `geointel-building-yolov8s-aoi512e80-pt`: + - `0.25`: 270 detections, F1 `0.23509933774834438` + - `0.15`: 523 detections, F1 `0.19603267211201864` + - `0.05`: 1000 detections, F1 `0.13193403298350823` +- Live Turnhout calibration with `geointel-building-yolov8s-aoi512e80-pt`: + - `0.25`: 822 detections, F1 `0.1304075235109718` + - `0.15`: 1000 detections, F1 `0.13085166384658772` + - `0.05`: 1000 detections, F1 `0.13085166384658772` -Open: -- Rebuild/redeploy the Tower all-in-one image before rerunning live calibration so the container uses `YOLO_MAX_DETECTIONS=1000`. -- After deploy, rerun at least one high-density AOI calibration to confirm detection counts are no longer capped at 300. +Conclusion: +- The former 300-detection runtime cap is removed; dense AOIs can now persist more candidates. +- The current AOI512 YOLOv8s candidate remains rejected for operational use because precision/recall quality is still too low and low thresholds saturate the configured `1000` cap. +- Next model work should focus on training data coverage, label strategy and post-processing/NMS behavior rather than only threshold lowering. ## Sprint 147 AOI512 YOLOv8s scale-match candidate gate (2026-07-09) diff --git a/docs/TODO.md b/docs/TODO.md index 7da01f9f..0caae68a 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -111,10 +111,11 @@ This file now starts with the current implementation status. Older preparation/b - [x] Train and gate the `uniquehardneg160e50` YOLOv8s candidate through 7 positive AOIs and 9 hard-negative/background samples. - [x] Train and gate an AOI-scale `aoi512e80` YOLOv8s candidate to test the 160px training-scale hypothesis. - [x] Raise configured-YOLO `max_det` through `YOLO_MAX_DETECTIONS` so dense AOIs are not capped at 300 detections before QA/QC. +- [x] Rerun live dense-AOI calibration after redeploy with `YOLO_MAX_DETECTIONS=1000`; Westerlo reached 523/1000 detections at lower thresholds and Turnhout reached 822/1000, confirming the old 300 cap is removed. - [ ] Find or train a materially stronger aerial/Kempen building model candidate; `geointel-building-yolov8n-expanded160e50-pt` is the best current dense-AOI candidate but still too weak and too noisy for a V1 default. - [ ] Train a higher-capacity local aerial-building detector with stronger positive recall while preserving the hard-negative false-positive gate. - [ ] Add more diverse positive AOIs and revisit geometry-to-box label strategy before the next default-model training attempt. -- [ ] Rerun live dense-AOI calibration after redeploy with `YOLO_MAX_DETECTIONS=1000` to measure uncapped recall and false-positive pressure. +- [ ] Add operator-side duplicate suppression/post-processing analysis for dense overlapping tile detections before the next promotion gate. ## Sprint 8 status