Add detection operator profiles
This commit is contained in:
@@ -260,6 +260,14 @@ as false-positive pressure. It does not run QA/QC or invent reference metrics
|
||||
for empty/sparse background AOIs. The first expanded local model improved dense
|
||||
AOI F1, but Kasterlee-bos false positives block default promotion.
|
||||
|
||||
The current inactive AOI1024 background-aware local model asset,
|
||||
`geointel-building-yolov8s-aoi1024bg512r3e50-pt`, is exposed in Detection Lab
|
||||
only through deliberate operator profiles. `balanced-review` applies threshold
|
||||
`0.15` for the strongest positive-AOI F1 observed so far; `conservative-review`
|
||||
applies threshold `0.35` for higher precision review. Both profiles remain
|
||||
candidate-only, not default-approved, because the promotion recommendation is
|
||||
still `none` and background false-positive pressure has not passed the gate.
|
||||
|
||||
To compare the same model/tile/threshold grid across all prepared operator
|
||||
samples, use:
|
||||
|
||||
|
||||
@@ -6168,3 +6168,32 @@ Open:
|
||||
|
||||
- Build a V1 operator detection profile layer in the UI/docs: expose `balanced` (`threshold=0.15`) and `conservative review` (`threshold=0.35`) as explicit choices for local model assets, with clear warning that the model is not a default-approved detector.
|
||||
- Clean the background corpus classification: separate pure-empty AOIs from sparse-building contextual AOIs, then retrain or recalibrate against that cleaner gate.
|
||||
|
||||
# Sprint 155 - Detection operator profiles
|
||||
|
||||
## What changed
|
||||
|
||||
- Added `frontend/src/components/detection/detectionProfiles.ts` with explicit operator profiles for the inactive `geointel-building-yolov8s-aoi1024bg512r3e50-pt` local model asset.
|
||||
- Exposed two deliberate Detection Lab actions:
|
||||
- `balanced-review`: confidence threshold `0.15`, positive-AOI F1 `0.5074022485589402`, precision `0.636639`, recall `0.424258`, max background detections `103`.
|
||||
- `conservative-review`: confidence threshold `0.35`, positive-AOI F1 `0.32086574003576274`, precision `0.840006`, recall `0.202135`, max background detections `55`.
|
||||
- Applying a profile selects `yolo-configured`, the local model asset id and the profile threshold. It does not auto-select assets on model catalog load and does not promote the candidate as a default detector.
|
||||
- Detection Lab now marks both profiles as `Candidate only - not default-approved` because the promotion recommendation remains `none`.
|
||||
- Updated frontend, AI pipeline and TODO documentation.
|
||||
|
||||
## What was tested
|
||||
|
||||
- Added regression coverage in `backend/tests/test_sprint155_detection_operator_profiles.py`.
|
||||
- Ran `python -m pytest tests/test_sprint155_detection_operator_profiles.py tests/test_sprint122_model_asset_activation_guardrails.py -q`.
|
||||
- Ran `python -m pytest` in `backend`: 435 passed.
|
||||
- Ran `python -m compileall backend/app`.
|
||||
- Ran `cd frontend && npm run typecheck`.
|
||||
- Ran `cd frontend && npm run build`.
|
||||
- Ran `bash scripts/run_readiness_check.sh`.
|
||||
- Ran `cd backend && python -m alembic heads` and `cd backend && python -m alembic upgrade head --sql`.
|
||||
- Ran `bash -n scripts/live_migration_smoke.sh`.
|
||||
|
||||
## Known limitations
|
||||
|
||||
- The profiles are review/demo aids only. The background corpus still needs to be split into pure-empty negatives and sparse-building contextual AOIs before retraining or recalibrating for a default detector decision.
|
||||
- No backend API contract, migration, provider fetching, fake detection output, model download behavior or active runtime default changed.
|
||||
|
||||
+1
-1
@@ -120,7 +120,7 @@ This file now starts with the current implementation status. Older preparation/b
|
||||
- [x] Export and audit AOI1024 clean-label variants; select `yolo-building-aoi1024-visible050-minpx8` as the first audit-passing 512px training candidate.
|
||||
- [x] Train and gate `geointel-building-yolov8s-aoi1024clean512e50-pt` through seven positive AOIs and nine hard-negative/background AOIs.
|
||||
- [x] Train and gate background-aware `geointel-building-yolov8s-aoi1024bg512r3e50-pt`; it is the strongest positive-AOI candidate so far but remains inactive because full background-candidate false-positive pressure still blocks default promotion.
|
||||
- [ ] Add explicit operator detection profiles for local model assets: balanced review around threshold `0.15` and conservative high-precision review around threshold `0.35`, both clearly marked as non-default-approved until promotion gates pass.
|
||||
- [x] Add explicit operator detection profiles for local model assets: balanced review around threshold `0.15` and conservative high-precision review around threshold `0.35`, both clearly marked as non-default-approved until promotion gates pass.
|
||||
- [ ] Split the background corpus into pure-empty negatives and sparse-building contextual AOIs, then retrain or recalibrate against the cleaner gate.
|
||||
- [ ] Promote a V1 default building detector only after it passes seven positive AOIs, clean hard-negative/background gates and persisted QA/QC evidence without fake detections or model downloads.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user