Record per-sample YOLO audit diagnostics
GeoIntel CI / docs-smoke (push) Has been cancelled
GeoIntel CI / contract-smoke (push) Has been cancelled

This commit is contained in:
Codex
2026-07-11 20:56:11 +02:00
parent 7ccabf5fed
commit 8cb435361d
2 changed files with 61 additions and 0 deletions
+59
View File
@@ -6747,3 +6747,62 @@ Open:
## Next recommended pass ## Next recommended pass
- Add building-size/visibility diagnostics per AOI and use them to choose or generate better positive samples before another training run. Do not spend more CPU on the current cleanpx dataset without changing the sample/label strategy. - Add building-size/visibility diagnostics per AOI and use them to choose or generate better positive samples before another training run. Do not spend more CPU on the current cleanpx dataset without changing the sample/label strategy.
# Sprint 166 - Per-sample YOLO dataset audit diagnostics
## What changed
- Extended `scripts/audit_operator_yolo_dataset_quality.py` with per-sample YOLO label diagnostics in `sample_summaries`:
- parsed label count;
- invalid label count;
- missing label file count;
- median/mean normalized box area;
- median normalized width/height;
- small-box count/share;
- sample-specific quality warning codes.
- Kept the existing global audit gates unchanged.
- Updated the Markdown audit output so each AOI/sample line includes the new label-quality signals.
- Added regression assertions in `backend/tests/test_sprint146_operator_yolo_dataset_quality_audit.py`.
- Updated `scripts/README.md` to document per-sample diagnostics.
## Local validation
- RED: `python -m pytest backend/tests/test_sprint146_operator_yolo_dataset_quality_audit.py::test_operator_yolo_dataset_quality_audit_reports_dataset_risks -q` failed with `KeyError: 'parsed_label_count'` before sample diagnostics existed.
- GREEN: same targeted test passed after adding sample label stats.
- Ran `python -m pytest backend/tests/test_sprint146_operator_yolo_dataset_quality_audit.py backend/tests/test_sprint130_operator_yolo_tile_dataset.py backend/tests/test_sprint156_background_corpus_classification.py -q`: 12 passed.
- Ran `bash scripts/run_readiness_check.sh`: 457 backend tests passed, frontend typecheck passed, frontend build passed, readiness passed.
## Tower runtime evidence
- Pushed commit `7ccabf5` and redeployed the all-in-one Tower runtime at `http://192.168.10.150:1202`.
- Deploy validation passed:
- live migration smoke passed;
- browser runtime verification passed;
- container exposed `0.0.0.0:1202->80/tcp`.
- Re-ran the clean AOI1024 audit with sample diagnostics:
- report: `/app/artifacts/operator-yolo-dataset-audit/aoi1024-cleanpx12vis035-sample-diagnostics/operator_yolo_dataset_quality_audit.json`
- status: `ok`
- tile count: `144`
- positive tiles: `114`
- negative tiles: `30`
- labels: `14632`
- parsed labels: `14632`
- median normalized box area: `0.001373291016`
- small-box share: `0.0`
- sample count: `16`
- All samples reported `quality_warnings=[]` under the current gates.
- Lowest positive/context label counts:
- `ravels_bos`: 22 parsed labels, median box area `0.001140594385`
- `geel_bel`: 56 parsed labels, median box area `0.001411437891`
- `kasterlee_bos`: 73 parsed labels, median box area `0.001522064229`
- `dessel_heide`: 77 parsed labels, median box area `0.001495361328`
- `meerhout_bos`: 99 parsed labels, median box area `0.001232147197`
## Known limitations
- Numeric label diagnostics now pass, but the rejected `aoi1024cleanpx12vis035e50` model proves that numeric gates alone are insufficient.
- The next unknown is visual alignment and semantic fit: whether GRB building boxes line up well enough with the orthophoto tiles and whether sparse/context AOIs represent the false-positive modes seen during promotion.
## Next recommended pass
- Design and add a visual YOLO label QA artifact generator: deterministic contact sheets of selected train/val tiles with YOLO boxes overlaid on imagery, grouped by AOI/sample and label density. Use it before another training run.
+2
View File
@@ -129,6 +129,8 @@ This file now starts with the current implementation status. Older preparation/b
- [x] Add preflight-only validation for the split-background promotion workflow before long runtime matrices. - [x] Add preflight-only validation for the split-background promotion workflow before long runtime matrices.
- [x] Rerun split background matrices on Tower after rebuild, then recalibrate against the cleaner pure-empty gate plus separate sparse-context inspection matrix. - [x] Rerun split background matrices on Tower after rebuild, then recalibrate against the cleaner pure-empty gate plus separate sparse-context inspection matrix.
- [x] Add guarded promoted-candidate activation helper requiring a promotion report path and exact candidate key before `.env` can be changed. - [x] Add guarded promoted-candidate activation helper requiring a promotion report path and exact candidate key before `.env` can be changed.
- [x] Add per-sample YOLO dataset audit diagnostics for parsed labels, median box area, small-box share and AOI-specific warning codes.
- [ ] Add deterministic visual YOLO label QA contact sheets before spending more CPU on another training run.
- [ ] Apply promoted V1 default building detector only after explicit operator review of the emitted `.env` updates, followed by rebuild/restart and browser/runtime smoke. - [ ] Apply promoted V1 default building detector only after explicit operator review of the emitted `.env` updates, followed by rebuild/restart and browser/runtime smoke.
## Sprint 8 status ## Sprint 8 status