Add guarded promoted YOLO activation
GeoIntel CI / docs-smoke (push) Has been cancelled
GeoIntel CI / contract-smoke (push) Has been cancelled

This commit is contained in:
Codex
2026-07-11 10:45:06 +02:00
parent 340be960e9
commit b1a4074cc8
13 changed files with 550 additions and 23 deletions
+27 -5
View File
@@ -335,13 +335,35 @@ hard-negative gates, then run `sparse_building_context` as a separate review
matrix. The first expanded local model improved dense AOI F1, but Kasterlee-bos
false positives block default promotion.
The current inactive AOI1024 background-aware local model asset,
The AOI1024 background-aware local model asset,
`geointel-building-yolov8s-aoi1024bg512r3e50-pt`, is exposed in Detection Lab
only through deliberate operator profiles. `balanced-review` applies threshold
`0.15` for the strongest positive-AOI F1 observed so far; `conservative-review`
applies threshold `0.35` for higher precision review. Both profiles remain
candidate-only, not default-approved, because the promotion recommendation is
still `none` and background false-positive pressure has not passed the gate.
`0.15` for the strongest positive-AOI F1 observed so far, but remains
candidate-only because pure-empty false-positive pressure failed at that
threshold. `conservative-review` applies threshold `0.35` and is marked as the
promoted candidate after the split-background report passed the strict
pure-empty gate. Sparse-context detections remain review-only evidence, not a
default-promotion blocker.
To update a Tower/Unraid `.env` from a promoted report, use the guarded
activation helper. It validates the exact report candidate key, verifies that
the candidate has `promotion_status=promote_candidate`, resolves the local model
asset under the mounted models directory, and writes environment updates only
when `--apply` is supplied:
```bash
python scripts/activate_promoted_yolo_candidate.py \
--promotion-report artifacts/detection-model-promotion/split-aware/aoi1024bg512r3e50-high-threshold-split-20260710T222934Z/detection_model_promotion_report.json \
--candidate-key 'geointel-building-yolov8s-aoi1024bg512r3e50-pt|512|64|0.35' \
--models-dir /mnt/user/appdata/geointel/models \
--env-file /mnt/user/appdata/geointel/.env \
--json
```
Re-run with `--apply` only after reviewing the emitted env updates. The helper
does not download weights, load a model or run inference. Restart or rebuild the
runtime after applying because `YOLO_MODEL_PATH` is read from environment
configuration.
To compare the same model/tile/threshold grid across all prepared operator
samples, use:
+41
View File
@@ -6532,3 +6532,44 @@ Open:
## Next recommended pass
- Add a guarded model activation/operator-selection workflow that can mark a promoted candidate as active only after the report artifact and candidate key are explicitly supplied.
# Sprint 163 - Guarded promoted YOLO activation workflow
## What changed
- Added `scripts/activate_promoted_yolo_candidate.py`.
- The helper validates:
- the promotion report file exists and is valid JSON;
- the exact supplied `candidate_key` matches the report recommended candidate;
- the candidate has `promotion_status=promote_candidate`;
- positive sample count, background sample count, mean F1 and max background detections still satisfy report gates;
- the candidate `model_asset_id` resolves to an existing local model file under the mounted models directory.
- The helper emits `.env` updates in dry-run mode by default and writes them only when `--apply` is supplied.
- Updated Detection Lab operator profiles:
- `balanced-review` at threshold `0.15` remains candidate-only because pure-empty false-positive pressure failed.
- `conservative-review` at threshold `0.35` is marked as promoted/default-approved based on the split-background pure-empty gate.
- Added docs for the guarded activation command in `docs/AI_PIPELINES.md`, `scripts/README.md`, `backend/README.md` and `frontend/README.md`.
- Added readiness coverage for compiling the new helper.
- No API contract, database migration, provider fetching, fake detection path, model file mutation, model download or automatic runtime activation was introduced in code.
## What was tested locally
- RED: `python -m pytest tests/test_sprint162_promoted_model_activation.py -q` failed while `scripts/activate_promoted_yolo_candidate.py` was absent.
- RED: `python -m pytest tests/test_sprint155_detection_operator_profiles.py -q` failed before `conservative-review` was marked promoted.
- RED: `python -m pytest tests/test_sprint162_promoted_model_activation.py::test_readiness_gate_compiles_promoted_activation_script -q` failed before readiness compiled the helper.
- Ran `python -m pytest tests/test_sprint162_promoted_model_activation.py tests/test_sprint155_detection_operator_profiles.py -q`: 7 passed.
- Ran `python -m py_compile scripts/activate_promoted_yolo_candidate.py`.
- Ran `python -m compileall backend/app`.
- Ran `python -m pytest` in `backend`: 454 passed, 17 existing Pydantic protected-namespace warnings.
- Ran `npm run typecheck` in `frontend`.
- Ran `npm run build` in `frontend`.
- Ran `bash scripts/run_readiness_check.sh`: passed.
## Known limitations
- The helper updates runtime environment only; a container restart or rebuild is still required for `YOLO_MODEL_PATH` changes to take effect.
- The promoted threshold is represented in the operator profile and promotion report. The backend detection endpoint still requires clients to submit the intended confidence threshold explicitly.
## Next recommended pass
- Push this helper to Tower, run it first as dry-run against the high-threshold promotion report, then apply and redeploy/restart only if the emitted `YOLO_MODEL_PATH` matches the promoted local asset.
+4 -3
View File
@@ -121,14 +121,15 @@ This file now starts with the current implementation status. Older preparation/b
- [x] Export and audit AOI1024 clean-label variants; select `yolo-building-aoi1024-visible050-minpx8` as the first audit-passing 512px training candidate.
- [x] Train and gate `geointel-building-yolov8s-aoi1024clean512e50-pt` through seven positive AOIs and nine hard-negative/background AOIs.
- [x] Train and gate background-aware `geointel-building-yolov8s-aoi1024bg512r3e50-pt`; it is the strongest positive-AOI candidate so far but remains inactive because full background-candidate false-positive pressure still blocks default promotion.
- [x] Add explicit operator detection profiles for local model assets: balanced review around threshold `0.15` and conservative high-precision review around threshold `0.35`, both clearly marked as non-default-approved until promotion gates pass.
- [x] Add explicit operator detection profiles for local model assets: balanced review around threshold `0.15` remains candidate-only, while conservative high-precision review around threshold `0.35` is marked promoted after the pure-empty split-background gate passed.
- [x] Add pure-empty versus sparse-building contextual background corpus classification to operator manifests, hard-negative matrix filters and YOLO tile provenance.
- [x] Add a split background-corpus matrix runner and report builder that runs pure-empty and sparse-context matrices separately.
- [x] Teach the model promotion report to consume split background summaries so only `pure_empty_negative` blocks default promotion and `sparse_building_context` stays review-only.
- [x] Add one-command operator workflow to run split background matrices and immediately build the split-aware promotion report.
- [x] Add preflight-only validation for the split-background promotion workflow before long runtime matrices.
- [ ] Rerun split background matrices on Tower after rebuild, then retrain or recalibrate against the cleaner pure-empty gate plus separate sparse-context inspection matrix.
- [ ] Promote a V1 default building detector only after it passes seven positive AOIs, clean hard-negative/background gates and persisted QA/QC evidence without fake detections or model downloads.
- [x] Rerun split background matrices on Tower after rebuild, then recalibrate against the cleaner pure-empty gate plus separate sparse-context inspection matrix.
- [x] Add guarded promoted-candidate activation helper requiring a promotion report path and exact candidate key before `.env` can be changed.
- [ ] Apply promoted V1 default building detector only after explicit operator review of the emitted `.env` updates, followed by rebuild/restart and browser/runtime smoke.
## Sprint 8 status