Harden detection model asset selection
GeoIntel CI / docs-smoke (push) Has been cancelled
GeoIntel CI / contract-smoke (push) Has been cancelled

This commit is contained in:
Codex
2026-07-08 04:03:50 +02:00
parent 48ffcd0991
commit 0ed71c6f1e
7 changed files with 153 additions and 9 deletions
+30
View File
@@ -5000,6 +5000,36 @@ Limitations:
Next recommended pass:
- Continue with V1 usability work that reduces operator confusion without expanding frozen product scope.
## Sprint 122 Detection model asset activation guardrails (2026-07-08)
Changed:
- Hardened Detection Lab local model handling so reported runtime model assets are read-only choices and are not auto-selected by the frontend hook.
- Configured YOLO run readiness now blocks submission when local model assets exist but no explicit `model_asset_id` has been selected.
- Added an explicit model asset section with active runtime env status, `will_download_models`, SHA-256 preview, file size and mounted model path.
- Surfaced the current benchmark candidate `geointel-building-yolov8s-hardneg160r4e50-pt` with recommended starting threshold `0.25`.
- Added compact UI guidance styling for the benchmark/threshold warning.
- Added regression coverage in `backend/tests/test_sprint122_model_asset_activation_guardrails.py`.
- Updated `CHANGELOG.md` and `docs/TODO.md`.
Tested:
- Red step: `python -m pytest backend\tests\test_sprint122_model_asset_activation_guardrails.py -q` failed on the previous auto-selection behavior and missing guardrail copy.
- `python -m pytest backend\tests\test_sprint122_model_asset_activation_guardrails.py -q` (`3 passed`)
- `python -m pytest backend\tests\test_sprint118_yolo_preflight_ui.py backend\tests\test_sprint103_ai_lab_run_readiness.py backend\tests\test_sprint104_ai_lab_action_guardrails.py backend\tests\test_model_asset_catalog.py backend\tests\test_sprint122_model_asset_activation_guardrails.py -q` (`16 passed`)
- `python -m compileall backend/app`
- `python -m pytest backend\tests -q` (`408 passed`)
- `cd frontend && npm run typecheck`
- `cd frontend && npm run build`
Open:
- Full threshold calibration comparison UI is still pending; this pass adds safe single-threshold guidance and explicit asset choice only.
Limitations:
- Frontend guardrail only; no backend API contracts, migrations, provider fetching, model downloads or model weight mutation behavior changed.
- The active runtime env model can still be configured by operators through existing deployment/env tooling, but the Detection Lab no longer silently chooses a local asset from the catalog for a run.
Next recommended pass:
- Add threshold calibration comparison UX over existing persisted runs so candidate models can be promoted with visible precision/recall/F1 and hard-negative counts.
## Operator YOLOv8s hard-negative model benchmark (2026-07-08)
Changed:
+3 -2
View File
@@ -416,7 +416,8 @@ This file now starts with the current implementation status. Older preparation/b
- [x] Add operator-only tile-level YOLO dataset export with overlapping windows and deterministic negative tile retention.
- [x] Run tile-level training on Tower and accept/reject the resulting local model through the persisted QA/QC matrix.
- [x] Train/evaluate a YOLOv8s hard-negative local building-detector candidate on Tower and keep it inactive because hard-negative false positives remain.
- [ ] Add an operator-facing local model catalog/activation workflow with SHA256, active model status and explicit threshold guidance.
- [ ] Add threshold calibration UX so detection runs do not silently rely on an unsafe default confidence.
- [x] Add an operator-facing local model catalog/activation workflow with SHA256, active model status and explicit threshold guidance.
- [x] Block silent local model asset auto-selection in Detection Lab.
- [ ] Add full threshold calibration comparison UX so detection runs can compare candidate thresholds before promotion.
- [ ] Add more AOIs after the tile-level baseline so the next local model attempt is not limited to Geel/Mol/Turnhout.
- [ ] Add negative/background AOIs so the next tile dataset is not all positive tiles.