Record YOLO visual label QA evidence

2026-07-11 22:12:35 +02:00
parent 5688feec35
commit 2a3472b919
2 changed files with 62 additions and 1 deletions
@@ -6806,3 +6806,63 @@ Open:
 ## Next recommended pass

 - Design and add a visual YOLO label QA artifact generator: deterministic contact sheets of selected train/val tiles with YOLO boxes overlaid on imagery, grouped by AOI/sample and label density. Use it before another training run.
+
+# Sprint 167 - Operator YOLO visual label QA contact sheets
+
+## What changed
+
+- Added `scripts/render_operator_yolo_label_qa_contact_sheets.py` to render deterministic operator-only contact sheets from existing YOLO tile datasets.
+- The script reads the existing `yolo_tile_dataset_summary.json`, image tiles and label files, then writes:
+  - `operator_yolo_label_qa_summary.json`;
+  - `operator_yolo_label_qa_contact_sheet.md`;
+  - one or more PNG contact sheets with YOLO labels drawn over the tile imagery.
+- Added explicit checks for missing images, missing label files, invalid YOLO rows and blank-looking/low-variance tiles.
+- Added regression coverage in `backend/tests/test_sprint167_operator_yolo_label_qa_contact_sheets.py`.
+- Updated the all-in-one Dockerfile so operator QA scripts are copied after the expensive dependency layer, keeping future script-only rebuilds cache-friendlier.
+- Updated Docker runtime tests so the all-in-one image keeps packaging the operator scripts needed on Tower.
+- No inference, training, provider fetch, database mutation, model activation or fake detection path was introduced.
+
+## Local validation
+
+- RED: `python -m pytest backend/tests/test_sprint167_operator_yolo_label_qa_contact_sheets.py -q` failed before the contact-sheet script existed.
+- GREEN: same targeted test passed after adding the renderer.
+- RED: the low-variance regression failed before `low_visual_variance_tile_count` existed.
+- GREEN: same targeted test passed after adding low-variance tile reporting.
+- Ran `python -m pytest backend/tests/test_docker_runtime_config.py::test_all_in_one_dockerfile_copies_operator_scripts_for_runtime_use backend/tests/test_docker_runtime_config.py::test_all_in_one_dockerfile_copies_operator_scripts_after_dependency_install backend/tests/test_sprint167_operator_yolo_label_qa_contact_sheets.py -q`: 3 passed.
+- Ran `bash scripts/run_readiness_check.sh`: 459 backend tests passed, frontend typecheck passed, frontend build passed, readiness passed.
+
+## Tower runtime evidence
+
+- Pushed commits `fbccf83` and `5688fee`, then redeployed the all-in-one Tower runtime at `http://192.168.10.150:1202`.
+- Docker storage had filled during the first build attempt. Cleaned Docker build cache and dangling images only; application volumes and appdata were not pruned. Docker reclaimed `98.38GB`.
+- Deploy validation for commit `5688fee` passed:
+  - live migration smoke passed;
+  - browser runtime verification passed;
+  - container exposed `0.0.0.0:1202->80/tcp`;
+  - Unraid icon check remained OK.
+- Rendered the clean AOI1024 visual label QA artifact inside the live container:
+  - summary: `/app/artifacts/operator-yolo-label-qa/aoi1024-cleanpx12vis035/operator_yolo_label_qa_summary.json`
+  - Markdown: `/app/artifacts/operator-yolo-label-qa/aoi1024-cleanpx12vis035/operator_yolo_label_qa_contact_sheet.md`
+  - contact sheet: `/app/artifacts/operator-yolo-label-qa/aoi1024-cleanpx12vis035/contact_sheet_001.png`
+- Runtime summary:
+  - status: `ok`
+  - selected tiles: `32`
+  - rendered tiles: `32`
+  - valid labels: `7670`
+  - invalid labels: `0`
+  - missing images: `0`
+  - missing label files: `0`
+  - low-variance tiles: `6`
+- Visual inspection confirmed:
+  - dense positive tiles show yellow YOLO boxes over real orthophoto imagery;
+  - the six low-variance tiles are `arendonk_heide` pure-empty negative validation tiles with no labels and blank-looking imagery.
+
+## Known limitations
+
+- The contact sheet now makes visual label inspection possible, but it also proves that the current clean AOI1024 dataset still contains blank-looking pure-empty negative tiles.
+- Those blank/low-variance negatives should not be used blindly for the next training run. They can distort the background corpus and do not represent realistic aerial false-positive pressure.
+- The current inactive model/promotion state remains unchanged.
+
+## Next recommended pass
+
+- Add no-data/low-variance filtering to the operator YOLO tile export path, regenerate the clean AOI1024 dataset, rerun the contact-sheet QA, and only then consider another training attempt.