Promote expanded building detector profile
This commit is contained in:
+13
-11
@@ -335,15 +335,17 @@ hard-negative gates, then run `sparse_building_context` as a separate review
|
||||
matrix. The first expanded local model improved dense AOI F1, but Kasterlee-bos
|
||||
false positives block default promotion.
|
||||
|
||||
The AOI1024 background-aware local model asset,
|
||||
`geointel-building-yolov8s-aoi1024bg512r3e50-pt`, is exposed in Detection Lab
|
||||
only through deliberate operator profiles. `balanced-review` applies threshold
|
||||
`0.15` for the strongest positive-AOI F1 observed so far, but remains
|
||||
candidate-only because pure-empty false-positive pressure failed at that
|
||||
threshold. `conservative-review` applies threshold `0.35` and is marked as the
|
||||
promoted candidate after the split-background report passed the strict
|
||||
pure-empty gate. Sparse-context detections remain review-only evidence, not a
|
||||
default-promotion blocker.
|
||||
The expanded-AOI local model asset,
|
||||
`geointel-building-yolov8s-aoi1024expandedminpx4vis035e50-pt`, is the current
|
||||
recommended Detection Lab operator profile. Use tile size `512`, overlap `64`
|
||||
and confidence threshold `0.15`. Persisted QA/QC across seven positive AOIs
|
||||
measured mean precision `0.6471`, recall `0.4700` and F1 `0.5433`; the strict
|
||||
three-sample pure-empty background gate produced zero detections. The previous
|
||||
`geointel-building-yolov8s-aoi1024bg512r3e50-pt` model remains available as a
|
||||
legacy conservative `0.35` review profile. Sparse-context detections remain
|
||||
review-only evidence, not a default-promotion blocker. Persistent misses are
|
||||
concentrated in small buildings, so every production-like run still requires
|
||||
persisted QA/QC against suitable reference data.
|
||||
|
||||
To update a Tower/Unraid `.env` from a promoted report, use the guarded
|
||||
activation helper. It validates the exact report candidate key, verifies that
|
||||
@@ -353,8 +355,8 @@ when `--apply` is supplied:
|
||||
|
||||
```bash
|
||||
python scripts/activate_promoted_yolo_candidate.py \
|
||||
--promotion-report artifacts/detection-model-promotion/split-aware/aoi1024bg512r3e50-high-threshold-split-20260710T222934Z/detection_model_promotion_report.json \
|
||||
--candidate-key 'geointel-building-yolov8s-aoi1024bg512r3e50-pt|512|64|0.35' \
|
||||
--promotion-report artifacts/detection-model-promotion/split-aware/aoi1024expandedminpx4vis035e50-split/detection_model_promotion_report.json \
|
||||
--candidate-key 'geointel-building-yolov8s-aoi1024expandedminpx4vis035e50-pt|512|64|0.15' \
|
||||
--models-dir /mnt/user/appdata/geointel/models \
|
||||
--env-file /mnt/user/appdata/geointel/.env \
|
||||
--json
|
||||
|
||||
@@ -7039,3 +7039,25 @@ Open:
|
||||
- The expanded live export exposed that Arendonk-heide remained configured as a holdout while all of its low-variance tiles were correctly filtered out.
|
||||
- Added `retained_validation_sample_slugs` and `empty_validation_sample_slugs` to tile dataset summaries so configured and actual validation coverage cannot be confused.
|
||||
- Added a focused regression test and kept filtering behavior unchanged; no blank tile was reintroduced.
|
||||
|
||||
# Sprint 173 - Expanded building model promotion
|
||||
|
||||
## Runtime evidence
|
||||
|
||||
- Completed inactive 50-epoch CPU training for `geointel-building-yolov8s-aoi1024expandedminpx4vis035e50.pt` from the expanded 20-source real-data corpus.
|
||||
- Trained-model SHA256: `a8a79cf5b0bdc19a0245acc322cf77232c335e222bd5f3c00a17d5f29402c196`.
|
||||
- Training summary recorded 45,892 labels across 171 retained tiles; final training metrics were precision `0.55558`, recall `0.35206`, mAP50 `0.27440` and mAP50-95 `0.10107`.
|
||||
- The persisted seven-AOI positive matrix recommended tile size `512`, overlap `64` and threshold `0.15`: mean precision `0.6470590036`, recall `0.4699913837`, F1 `0.5432865391`, and minimum per-AOI F1 `0.4897494305`.
|
||||
- The strict pure-empty gate covered Postel, Lommel and Arendonk across all tested thresholds and produced zero detections. Sparse-building contextual AOIs remained review-only evidence because real GRB buildings are present there.
|
||||
- Fixed-threshold persisted false-negative comparison used identical reference feature populations and reduced the false-negative rate in all seven positive AOIs versus the previous active `0.35` profile.
|
||||
- The guarded activation helper first returned `ready_to_apply`, resolved the exact local model asset and reported no downloads or inference. The reviewed `--apply` pass updated only `GEOINTEL_INSTALL_AI`, `YOLO_ENABLED`, `YOLO_MODELS_DIR` and `YOLO_MODEL_PATH` in the Tower environment.
|
||||
|
||||
## UI and operator behavior
|
||||
|
||||
- Detection Lab now recommends the promoted expanded-AOI model at threshold `0.15` and surfaces its seven-AOI coverage plus QA metrics.
|
||||
- The previous AOI1024 background-aware model remains mounted as an explicit legacy high-precision `0.35` review profile.
|
||||
- Profile selection remains deliberate: it selects a mounted local asset and run threshold only; it does not mutate runtime environment, download a model or start inference automatically.
|
||||
|
||||
## Remaining limitation
|
||||
|
||||
- Persistent small-building misses remain the weakest quality bucket. Continue with targeted evidence review and data coverage before considering another training run; do not infer production accuracy from aggregate F1 alone.
|
||||
|
||||
+4
-3
@@ -121,7 +121,7 @@ This file now starts with the current implementation status. Older preparation/b
|
||||
- [x] Export and audit AOI1024 clean-label variants; select `yolo-building-aoi1024-visible050-minpx8` as the first audit-passing 512px training candidate.
|
||||
- [x] Train and gate `geointel-building-yolov8s-aoi1024clean512e50-pt` through seven positive AOIs and nine hard-negative/background AOIs.
|
||||
- [x] Train and gate background-aware `geointel-building-yolov8s-aoi1024bg512r3e50-pt`; it is the strongest positive-AOI candidate so far but remains inactive because full background-candidate false-positive pressure still blocks default promotion.
|
||||
- [x] Add explicit operator detection profiles for local model assets: balanced review around threshold `0.15` remains candidate-only, while conservative high-precision review around threshold `0.35` is marked promoted after the pure-empty split-background gate passed.
|
||||
- [x] Add explicit operator detection profiles for local model assets and promote the expanded-AOI balanced review profile at threshold `0.15` after positive and pure-empty split-background gates passed.
|
||||
- [x] Add pure-empty versus sparse-building contextual background corpus classification to operator manifests, hard-negative matrix filters and YOLO tile provenance.
|
||||
- [x] Add a split background-corpus matrix runner and report builder that runs pure-empty and sparse-context matrices separately.
|
||||
- [x] Teach the model promotion report to consume split background summaries so only `pure_empty_negative` blocks default promotion and `sparse_building_context` stays review-only.
|
||||
@@ -135,8 +135,9 @@ This file now starts with the current implementation status. Older preparation/b
|
||||
- [x] Regenerate the AOI1024 cleanpx YOLO dataset with low-variance negative filtering and rerun visual contact-sheet QA before training.
|
||||
- [x] Train one inactive candidate from the filtered AOI1024 cleanpx YOLO dataset and gate it through the positive-AOI plus split-background promotion workflow; reject it because mean positive F1 remains below gate.
|
||||
- [x] Add deterministic dataset/base/trained-model SHA256 provenance to future operator training summaries.
|
||||
- [ ] Review per-AOI false-negative evidence for the weakest AOIs and improve positive sample/label geometry coverage before another training candidate.
|
||||
- [ ] Apply promoted V1 default building detector only after explicit operator review of the emitted `.env` updates, followed by rebuild/restart and browser/runtime smoke.
|
||||
- [x] Review per-AOI false-negative evidence, expand positive sample/label coverage and verify the resulting candidate improves false-negative rate in every validated AOI.
|
||||
- [ ] Complete rebuild/restart and browser/runtime smoke for the guarded promoted V1 building detector activation.
|
||||
- [ ] Expand focused small-building training evidence only after reviewing persistent false negatives from the promoted model; do not start another blind training run.
|
||||
|
||||
## Sprint 8 status
|
||||
|
||||
|
||||
Reference in New Issue
Block a user