Jens/geointel
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add detection threshold calibration comparison
GeoIntel CI / docs-smoke (push) Has been cancelled
Details
GeoIntel CI / contract-smoke (push) Has been cancelled
Details

Browse Source
This commit is contained in:
Codex
2026-07-08 12:33:07 +02:00
parent 46204edbf4
commit 0e182ad69d
7 changed files with 333 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/CODEX_EXECUTION_LOG.md
+24
View File
@@ -5103,6 +5103,30 @@ Limitations:
Next recommended pass:
- Add threshold calibration comparison UX so an operator can compare candidate thresholds before promoting a local model.
## Sprint 133 Detection threshold calibration UX (2026-07-08)
Changed:
- Added a Detection Lab calibration comparison panel that combines existing persisted `DetectionRunRead` rows with existing persisted `QualityCheckRead`/metric rows.
- The panel shows confidence threshold, model, local model asset id, detection count, precision, recall, F1, false positives, false negatives and linked quality-check id.
- Added summary cards for best F1 candidate, best precision candidate and lowest false-positive pressure.
- Added a promotion guardrail that keeps model/threshold acceptance tied to QA evidence across AOIs instead of a single run.
- Passed project-level `qualityChecks` into Detection Lab without adding API routes, migrations or new AI execution behavior.
- Added regression coverage in `backend/tests/test_sprint133_detection_threshold_calibration_ux.py`.
- Marked the threshold calibration UX item complete in `docs/TODO.md`.
Tested:
- Red step: `python -m pytest backend\tests\test_sprint133_detection_threshold_calibration_ux.py -q` failed while the persisted calibration comparison UI was absent.
- `python -m pytest backend\tests\test_sprint133_detection_threshold_calibration_ux.py -q` (`1 passed`)
- `python -m pytest backend\tests\test_sprint122_model_asset_activation_guardrails.py backend\tests\test_sprint123_raster_detection_handoff_operational.py backend\tests\test_sprint133_detection_threshold_calibration_ux.py -q` (`7 passed`)
- `cd frontend && npm run typecheck`
- `cd frontend && npm run build`
Limitations:
- This is a persisted-run comparison surface only. It does not launch batch calibration sweeps from the browser and does not auto-promote model assets or thresholds.
Next recommended pass:
- Add a guided in-app calibration runner that can queue a small explicit threshold set for one selected raster/reference pair, reusing the existing detection and QA APIs.
## Sprint 117 Safe local YOLO model activation (2026-07-06)
Changed:
docs/TODO.md
+1 -1
View File
@@ -419,6 +419,6 @@ This file now starts with the current implementation status. Older preparation/b
- [x] Add an operator-facing local model catalog/activation workflow with SHA256, active model status and explicit threshold guidance.
- [x] Block silent local model asset auto-selection in Detection Lab.
- [x] Add structured raster tile manifest handoff into Detection Lab with linked preflight visibility.
- [ ] Add full threshold calibration comparison UX so detection runs can compare candidate thresholds before promotion.
- [x] Add full threshold calibration comparison UX so detection runs can compare candidate thresholds before promotion.
- [ ] Add more AOIs after the tile-level baseline so the next local model attempt is not limited to Geel/Mol/Turnhout.
- [ ] Add negative/background AOIs so the next tile dataset is not all positive tiles.