Support multi-sample promotion evidence
GeoIntel CI / docs-smoke (push) Has been cancelled
GeoIntel CI / contract-smoke (push) Has been cancelled

This commit is contained in:
Codex
2026-07-09 14:52:06 +02:00
parent f4af3b9eb0
commit 006819ca51
5 changed files with 142 additions and 17 deletions
+12 -1
View File
@@ -5965,6 +5965,16 @@ Open:
- the previous Geel 1024px upload `413` no longer occurs
- the previous Geel low-threshold persisted YOLO/QA `504` no longer occurs
- Geel `threshold=0.05` completed with 2,612 detections, 5,091 raw candidates, 2,479 suppressed duplicates, precision `0.06316998468606431`, recall `0.165` and F1 `0.09136212624584718`
- Full AOI1024 positive persisted QA matrix:
- Geel best F1: `0.09136212624584718` at threshold `0.05`
- Turnhout best F1: `0.058721074894252295` at threshold `0.05`
- Retie best F1: `0.15621436716077539` at threshold `0.15`
- Westerlo best F1: `0.28703703703703703` at threshold `0.15`
- Full AOI1024 hard-negative/background matrix:
- pure empty Postel/Lommel/Arendonk samples stayed at or near zero detections
- mixed background candidates still produced false-positive pressure: max detections were `59` at threshold `0.25`, `107` at `0.15` and `226` at `0.05`
- Fixed `scripts/build_detection_model_promotion_report.py` after discovering it accepted `multi_sample_quality_summary.json` but counted positive samples as `0`. The report now supports both `samples[].runs[]` portfolios and `items[]` multi-sample summaries.
- AOI1024 promotion report path: `artifacts/detection-model-promotion/aoi1024visible025e50-full/detection_model_promotion_report.md`; recommendation remains `none`.
## Known limitations
@@ -5972,7 +5982,8 @@ Open:
- Several dense 1024 GRB reference exports reached the current 1000-feature source cap. Treat those samples as useful but potentially reference-capped until the provider query path supports paging or smaller dense AOIs are chosen.
- The trained model is intentionally inactive. It needs persisted detection QA/QC matrix evidence and background/hard-negative evidence before default promotion.
- The first completed persisted Geel QA run confirms the candidate is not promotion-ready: recall improves at low threshold, but false-positive pressure is too high.
- The complete positive/background evidence confirms the candidate is not promotion-ready. The bottleneck is label/source quality and sample design, not runtime configuration.
## Next recommended pass
- Run the remaining AOI1024 persisted QA samples and background/hard-negative checks, then decide whether label/source paging or additional AOI quality work comes before another training run.
- Fix dense GRB reference completeness first: add provider-side paging or split dense AOIs so reference exports do not cap at 1000 features, then regenerate AOI1024 labels before another training run.
+4 -1
View File
@@ -468,6 +468,9 @@ This file now starts with the current implementation status. Older preparation/b
- [x] Fix the all-in-one/compose Nginx upload limit after live 1024px GeoTIFF uploads hit `413 Request Entity Too Large`.
- [x] Fix the all-in-one/compose Nginx proxy timeout after low-threshold persisted YOLO/QA runs hit `504 Gateway Timeout`.
- [x] Rerun the previously failing Geel low-threshold persisted QA/QC path for `geointel-building-yolov8s-aoi1024visible025e50-pt` after redeploying upload/timeout fixes.
- [ ] Finish remaining AOI1024 positive-sample matrix coverage for `geointel-building-yolov8s-aoi1024visible025e50-pt`.
- [x] Finish remaining AOI1024 positive-sample matrix coverage for `geointel-building-yolov8s-aoi1024visible025e50-pt`.
- [x] Run AOI1024 background/hard-negative matrix for `geointel-building-yolov8s-aoi1024visible025e50-pt`.
- [x] Fix promotion-report parsing for `multi_sample_quality_summary.json` inputs.
- [x] Generate AOI1024 promotion report and keep recommended candidate as `none`.
- [ ] Add GRB paging or smaller dense AOI sampling before trusting 1000-feature-capped dense reference exports as full ground truth.
- [ ] Keep every local YOLO candidate inactive until positive-AOI and hard-negative promotion reports recommend default activation.