Record per-sample YOLO audit diagnostics

2026-07-11 20:56:11 +02:00
parent 7ccabf5fed
commit 8cb435361d
2 changed files with 61 additions and 0 deletions
@@ -6747,3 +6747,62 @@ Open:
 ## Next recommended pass

 - Add building-size/visibility diagnostics per AOI and use them to choose or generate better positive samples before another training run. Do not spend more CPU on the current cleanpx dataset without changing the sample/label strategy.
+
+# Sprint 166 - Per-sample YOLO dataset audit diagnostics
+
+## What changed
+
+- Extended `scripts/audit_operator_yolo_dataset_quality.py` with per-sample YOLO label diagnostics in `sample_summaries`:
+  - parsed label count;
+  - invalid label count;
+  - missing label file count;
+  - median/mean normalized box area;
+  - median normalized width/height;
+  - small-box count/share;
+  - sample-specific quality warning codes.
+- Kept the existing global audit gates unchanged.
+- Updated the Markdown audit output so each AOI/sample line includes the new label-quality signals.
+- Added regression assertions in `backend/tests/test_sprint146_operator_yolo_dataset_quality_audit.py`.
+- Updated `scripts/README.md` to document per-sample diagnostics.
+
+## Local validation
+
+- RED: `python -m pytest backend/tests/test_sprint146_operator_yolo_dataset_quality_audit.py::test_operator_yolo_dataset_quality_audit_reports_dataset_risks -q` failed with `KeyError: 'parsed_label_count'` before sample diagnostics existed.
+- GREEN: same targeted test passed after adding sample label stats.
+- Ran `python -m pytest backend/tests/test_sprint146_operator_yolo_dataset_quality_audit.py backend/tests/test_sprint130_operator_yolo_tile_dataset.py backend/tests/test_sprint156_background_corpus_classification.py -q`: 12 passed.
+- Ran `bash scripts/run_readiness_check.sh`: 457 backend tests passed, frontend typecheck passed, frontend build passed, readiness passed.
+
+## Tower runtime evidence
+
+- Pushed commit `7ccabf5` and redeployed the all-in-one Tower runtime at `http://192.168.10.150:1202`.
+- Deploy validation passed:
+  - live migration smoke passed;
+  - browser runtime verification passed;
+  - container exposed `0.0.0.0:1202->80/tcp`.
+- Re-ran the clean AOI1024 audit with sample diagnostics:
+  - report: `/app/artifacts/operator-yolo-dataset-audit/aoi1024-cleanpx12vis035-sample-diagnostics/operator_yolo_dataset_quality_audit.json`
+  - status: `ok`
+  - tile count: `144`
+  - positive tiles: `114`
+  - negative tiles: `30`
+  - labels: `14632`
+  - parsed labels: `14632`
+  - median normalized box area: `0.001373291016`
+  - small-box share: `0.0`
+  - sample count: `16`
+- All samples reported `quality_warnings=[]` under the current gates.
+- Lowest positive/context label counts:
+  - `ravels_bos`: 22 parsed labels, median box area `0.001140594385`
+  - `geel_bel`: 56 parsed labels, median box area `0.001411437891`
+  - `kasterlee_bos`: 73 parsed labels, median box area `0.001522064229`
+  - `dessel_heide`: 77 parsed labels, median box area `0.001495361328`
+  - `meerhout_bos`: 99 parsed labels, median box area `0.001232147197`
+
+## Known limitations
+
+- Numeric label diagnostics now pass, but the rejected `aoi1024cleanpx12vis035e50` model proves that numeric gates alone are insufficient.
+- The next unknown is visual alignment and semantic fit: whether GRB building boxes line up well enough with the orthophoto tiles and whether sparse/context AOIs represent the false-positive modes seen during promotion.
+
+## Next recommended pass
+
+- Design and add a visual YOLO label QA artifact generator: deterministic contact sheets of selected train/val tiles with YOLO boxes overlaid on imagery, grouped by AOI/sample and label density. Use it before another training run.