From 282f5b0d0c9a0aaf0a97c948dd5dbc3cf5cd3d72 Mon Sep 17 00:00:00 2001 From: Codex Date: Wed, 8 Jul 2026 16:43:00 +0200 Subject: [PATCH] Record live model promotion report --- CHANGELOG.md | 1 + docs/CODEX_EXECUTION_LOG.md | 21 +++++++++++++++++++-- 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 762bf49d..1abae8c7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,7 @@ - The report combines positive-AOI calibration evidence portfolios with hard-negative/background matrix summaries. - Candidate decisions are grouped by model asset, tile size, overlap and threshold, then gated by positive sample count, background sample count, mean F1 and maximum background detections per sample. - Added regression coverage for promoting a clean candidate and rejecting a candidate with background false-positive pressure. +- Ran the report on Tower against the regenerated 7-AOI positive portfolio and live hard-negative summaries; it evaluated 15 candidates and recommended none for default promotion. - No backend API, migration, frontend runtime, model weight, model download, inference or provider-fetching behavior changed. ## Sprint 141 Expanded positive-AOI matrix and portfolio metadata hardening (2026-07-08) diff --git a/docs/CODEX_EXECUTION_LOG.md b/docs/CODEX_EXECUTION_LOG.md index fa8842f0..6b93f2ff 100644 --- a/docs/CODEX_EXECUTION_LOG.md +++ b/docs/CODEX_EXECUTION_LOG.md @@ -15,10 +15,27 @@ Changed: Tested: - Red step: `python -m pytest backend\tests\test_sprint143_detection_model_promotion_report.py -q` failed because the report script did not exist. - `python -m pytest backend\tests\test_sprint143_detection_model_promotion_report.py -q` (`1 passed`) +- `python -m pytest backend\tests\test_sprint143_detection_model_promotion_report.py backend\tests\test_sprint139_multi_aoi_calibration_evidence_portfolio.py -q` (`2 passed`) +- `python -m py_compile scripts\build_detection_model_promotion_report.py` +- `python scripts\smoke_docs.py` +- `bash scripts/run_readiness_check.sh` (`419 passed`; frontend typecheck/build passed; Alembic head `202606120900`) +- Tower pulled commit `c2fba67` and generated: + - `/mnt/user/appdata/geointel/artifacts/detection-model-promotion/positive-expanded-vs-hard-negative-20260708/detection_model_promotion_report.json` + - `/mnt/user/appdata/geointel/artifacts/detection-model-promotion/positive-expanded-vs-hard-negative-20260708/detection_model_promotion_report.md` +- Live promotion report evaluated 15 model/tile/threshold candidates and recommended `none`. +- Best current positive candidate `geointel-building-yolov8n-expanded160e50-pt|640|64|0.15` was rejected by gates: + - `positive_mean_f1_below_gate` + - `background_false_positive_pressure` + - positive samples `7` + - background samples `3` + - mean F1 `0.19718007234060198` + - max background detections `46` + - total background detections `46` +- Tower container remained healthy on `0.0.0.0:1202->80/tcp`. Open: -- Run the report against the regenerated 7-AOI positive portfolio and live hard-negative summaries on Tower. -- Use the report as a promotion gate only; it must not mutate the active YOLO model configuration. +- No evaluated model/threshold is ready for default promotion. +- Next model work should improve positive recall/F1 while preserving a strict hard-negative false-positive gate. ## Sprint 141 Expanded positive-AOI matrix and portfolio metadata hardening (2026-07-08)