Add guarded promoted YOLO activation
This commit is contained in:
+27
-5
@@ -335,13 +335,35 @@ hard-negative gates, then run `sparse_building_context` as a separate review
|
||||
matrix. The first expanded local model improved dense AOI F1, but Kasterlee-bos
|
||||
false positives block default promotion.
|
||||
|
||||
The current inactive AOI1024 background-aware local model asset,
|
||||
The AOI1024 background-aware local model asset,
|
||||
`geointel-building-yolov8s-aoi1024bg512r3e50-pt`, is exposed in Detection Lab
|
||||
only through deliberate operator profiles. `balanced-review` applies threshold
|
||||
`0.15` for the strongest positive-AOI F1 observed so far; `conservative-review`
|
||||
applies threshold `0.35` for higher precision review. Both profiles remain
|
||||
candidate-only, not default-approved, because the promotion recommendation is
|
||||
still `none` and background false-positive pressure has not passed the gate.
|
||||
`0.15` for the strongest positive-AOI F1 observed so far, but remains
|
||||
candidate-only because pure-empty false-positive pressure failed at that
|
||||
threshold. `conservative-review` applies threshold `0.35` and is marked as the
|
||||
promoted candidate after the split-background report passed the strict
|
||||
pure-empty gate. Sparse-context detections remain review-only evidence, not a
|
||||
default-promotion blocker.
|
||||
|
||||
To update a Tower/Unraid `.env` from a promoted report, use the guarded
|
||||
activation helper. It validates the exact report candidate key, verifies that
|
||||
the candidate has `promotion_status=promote_candidate`, resolves the local model
|
||||
asset under the mounted models directory, and writes environment updates only
|
||||
when `--apply` is supplied:
|
||||
|
||||
```bash
|
||||
python scripts/activate_promoted_yolo_candidate.py \
|
||||
--promotion-report artifacts/detection-model-promotion/split-aware/aoi1024bg512r3e50-high-threshold-split-20260710T222934Z/detection_model_promotion_report.json \
|
||||
--candidate-key 'geointel-building-yolov8s-aoi1024bg512r3e50-pt|512|64|0.35' \
|
||||
--models-dir /mnt/user/appdata/geointel/models \
|
||||
--env-file /mnt/user/appdata/geointel/.env \
|
||||
--json
|
||||
```
|
||||
|
||||
Re-run with `--apply` only after reviewing the emitted env updates. The helper
|
||||
does not download weights, load a model or run inference. Restart or rebuild the
|
||||
runtime after applying because `YOLO_MODEL_PATH` is read from environment
|
||||
configuration.
|
||||
|
||||
To compare the same model/tile/threshold grid across all prepared operator
|
||||
samples, use:
|
||||
|
||||
@@ -6532,3 +6532,44 @@ Open:
|
||||
## Next recommended pass
|
||||
|
||||
- Add a guarded model activation/operator-selection workflow that can mark a promoted candidate as active only after the report artifact and candidate key are explicitly supplied.
|
||||
|
||||
# Sprint 163 - Guarded promoted YOLO activation workflow
|
||||
|
||||
## What changed
|
||||
|
||||
- Added `scripts/activate_promoted_yolo_candidate.py`.
|
||||
- The helper validates:
|
||||
- the promotion report file exists and is valid JSON;
|
||||
- the exact supplied `candidate_key` matches the report recommended candidate;
|
||||
- the candidate has `promotion_status=promote_candidate`;
|
||||
- positive sample count, background sample count, mean F1 and max background detections still satisfy report gates;
|
||||
- the candidate `model_asset_id` resolves to an existing local model file under the mounted models directory.
|
||||
- The helper emits `.env` updates in dry-run mode by default and writes them only when `--apply` is supplied.
|
||||
- Updated Detection Lab operator profiles:
|
||||
- `balanced-review` at threshold `0.15` remains candidate-only because pure-empty false-positive pressure failed.
|
||||
- `conservative-review` at threshold `0.35` is marked as promoted/default-approved based on the split-background pure-empty gate.
|
||||
- Added docs for the guarded activation command in `docs/AI_PIPELINES.md`, `scripts/README.md`, `backend/README.md` and `frontend/README.md`.
|
||||
- Added readiness coverage for compiling the new helper.
|
||||
- No API contract, database migration, provider fetching, fake detection path, model file mutation, model download or automatic runtime activation was introduced in code.
|
||||
|
||||
## What was tested locally
|
||||
|
||||
- RED: `python -m pytest tests/test_sprint162_promoted_model_activation.py -q` failed while `scripts/activate_promoted_yolo_candidate.py` was absent.
|
||||
- RED: `python -m pytest tests/test_sprint155_detection_operator_profiles.py -q` failed before `conservative-review` was marked promoted.
|
||||
- RED: `python -m pytest tests/test_sprint162_promoted_model_activation.py::test_readiness_gate_compiles_promoted_activation_script -q` failed before readiness compiled the helper.
|
||||
- Ran `python -m pytest tests/test_sprint162_promoted_model_activation.py tests/test_sprint155_detection_operator_profiles.py -q`: 7 passed.
|
||||
- Ran `python -m py_compile scripts/activate_promoted_yolo_candidate.py`.
|
||||
- Ran `python -m compileall backend/app`.
|
||||
- Ran `python -m pytest` in `backend`: 454 passed, 17 existing Pydantic protected-namespace warnings.
|
||||
- Ran `npm run typecheck` in `frontend`.
|
||||
- Ran `npm run build` in `frontend`.
|
||||
- Ran `bash scripts/run_readiness_check.sh`: passed.
|
||||
|
||||
## Known limitations
|
||||
|
||||
- The helper updates runtime environment only; a container restart or rebuild is still required for `YOLO_MODEL_PATH` changes to take effect.
|
||||
- The promoted threshold is represented in the operator profile and promotion report. The backend detection endpoint still requires clients to submit the intended confidence threshold explicitly.
|
||||
|
||||
## Next recommended pass
|
||||
|
||||
- Push this helper to Tower, run it first as dry-run against the high-threshold promotion report, then apply and redeploy/restart only if the emitted `YOLO_MODEL_PATH` matches the promoted local asset.
|
||||
|
||||
+4
-3
@@ -121,14 +121,15 @@ This file now starts with the current implementation status. Older preparation/b
|
||||
- [x] Export and audit AOI1024 clean-label variants; select `yolo-building-aoi1024-visible050-minpx8` as the first audit-passing 512px training candidate.
|
||||
- [x] Train and gate `geointel-building-yolov8s-aoi1024clean512e50-pt` through seven positive AOIs and nine hard-negative/background AOIs.
|
||||
- [x] Train and gate background-aware `geointel-building-yolov8s-aoi1024bg512r3e50-pt`; it is the strongest positive-AOI candidate so far but remains inactive because full background-candidate false-positive pressure still blocks default promotion.
|
||||
- [x] Add explicit operator detection profiles for local model assets: balanced review around threshold `0.15` and conservative high-precision review around threshold `0.35`, both clearly marked as non-default-approved until promotion gates pass.
|
||||
- [x] Add explicit operator detection profiles for local model assets: balanced review around threshold `0.15` remains candidate-only, while conservative high-precision review around threshold `0.35` is marked promoted after the pure-empty split-background gate passed.
|
||||
- [x] Add pure-empty versus sparse-building contextual background corpus classification to operator manifests, hard-negative matrix filters and YOLO tile provenance.
|
||||
- [x] Add a split background-corpus matrix runner and report builder that runs pure-empty and sparse-context matrices separately.
|
||||
- [x] Teach the model promotion report to consume split background summaries so only `pure_empty_negative` blocks default promotion and `sparse_building_context` stays review-only.
|
||||
- [x] Add one-command operator workflow to run split background matrices and immediately build the split-aware promotion report.
|
||||
- [x] Add preflight-only validation for the split-background promotion workflow before long runtime matrices.
|
||||
- [ ] Rerun split background matrices on Tower after rebuild, then retrain or recalibrate against the cleaner pure-empty gate plus separate sparse-context inspection matrix.
|
||||
- [ ] Promote a V1 default building detector only after it passes seven positive AOIs, clean hard-negative/background gates and persisted QA/QC evidence without fake detections or model downloads.
|
||||
- [x] Rerun split background matrices on Tower after rebuild, then recalibrate against the cleaner pure-empty gate plus separate sparse-context inspection matrix.
|
||||
- [x] Add guarded promoted-candidate activation helper requiring a promotion report path and exact candidate key before `.env` can be changed.
|
||||
- [ ] Apply promoted V1 default building detector only after explicit operator review of the emitted `.env` updates, followed by rebuild/restart and browser/runtime smoke.
|
||||
|
||||
## Sprint 8 status
|
||||
|
||||
|
||||
Reference in New Issue
Block a user