From 2a3472b919cbb754a9188c51e5da671412202373 Mon Sep 17 00:00:00 2001 From: Codex Date: Sat, 11 Jul 2026 22:12:35 +0200 Subject: [PATCH] Record YOLO visual label QA evidence --- docs/CODEX_EXECUTION_LOG.md | 60 +++++++++++++++++++++++++++++++++++++ docs/TODO.md | 3 +- 2 files changed, 62 insertions(+), 1 deletion(-) diff --git a/docs/CODEX_EXECUTION_LOG.md b/docs/CODEX_EXECUTION_LOG.md index 22c5bd1f..977b52be 100644 --- a/docs/CODEX_EXECUTION_LOG.md +++ b/docs/CODEX_EXECUTION_LOG.md @@ -6806,3 +6806,63 @@ Open: ## Next recommended pass - Design and add a visual YOLO label QA artifact generator: deterministic contact sheets of selected train/val tiles with YOLO boxes overlaid on imagery, grouped by AOI/sample and label density. Use it before another training run. + +# Sprint 167 - Operator YOLO visual label QA contact sheets + +## What changed + +- Added `scripts/render_operator_yolo_label_qa_contact_sheets.py` to render deterministic operator-only contact sheets from existing YOLO tile datasets. +- The script reads the existing `yolo_tile_dataset_summary.json`, image tiles and label files, then writes: + - `operator_yolo_label_qa_summary.json`; + - `operator_yolo_label_qa_contact_sheet.md`; + - one or more PNG contact sheets with YOLO labels drawn over the tile imagery. +- Added explicit checks for missing images, missing label files, invalid YOLO rows and blank-looking/low-variance tiles. +- Added regression coverage in `backend/tests/test_sprint167_operator_yolo_label_qa_contact_sheets.py`. +- Updated the all-in-one Dockerfile so operator QA scripts are copied after the expensive dependency layer, keeping future script-only rebuilds cache-friendlier. +- Updated Docker runtime tests so the all-in-one image keeps packaging the operator scripts needed on Tower. +- No inference, training, provider fetch, database mutation, model activation or fake detection path was introduced. + +## Local validation + +- RED: `python -m pytest backend/tests/test_sprint167_operator_yolo_label_qa_contact_sheets.py -q` failed before the contact-sheet script existed. +- GREEN: same targeted test passed after adding the renderer. +- RED: the low-variance regression failed before `low_visual_variance_tile_count` existed. +- GREEN: same targeted test passed after adding low-variance tile reporting. +- Ran `python -m pytest backend/tests/test_docker_runtime_config.py::test_all_in_one_dockerfile_copies_operator_scripts_for_runtime_use backend/tests/test_docker_runtime_config.py::test_all_in_one_dockerfile_copies_operator_scripts_after_dependency_install backend/tests/test_sprint167_operator_yolo_label_qa_contact_sheets.py -q`: 3 passed. +- Ran `bash scripts/run_readiness_check.sh`: 459 backend tests passed, frontend typecheck passed, frontend build passed, readiness passed. + +## Tower runtime evidence + +- Pushed commits `fbccf83` and `5688fee`, then redeployed the all-in-one Tower runtime at `http://192.168.10.150:1202`. +- Docker storage had filled during the first build attempt. Cleaned Docker build cache and dangling images only; application volumes and appdata were not pruned. Docker reclaimed `98.38GB`. +- Deploy validation for commit `5688fee` passed: + - live migration smoke passed; + - browser runtime verification passed; + - container exposed `0.0.0.0:1202->80/tcp`; + - Unraid icon check remained OK. +- Rendered the clean AOI1024 visual label QA artifact inside the live container: + - summary: `/app/artifacts/operator-yolo-label-qa/aoi1024-cleanpx12vis035/operator_yolo_label_qa_summary.json` + - Markdown: `/app/artifacts/operator-yolo-label-qa/aoi1024-cleanpx12vis035/operator_yolo_label_qa_contact_sheet.md` + - contact sheet: `/app/artifacts/operator-yolo-label-qa/aoi1024-cleanpx12vis035/contact_sheet_001.png` +- Runtime summary: + - status: `ok` + - selected tiles: `32` + - rendered tiles: `32` + - valid labels: `7670` + - invalid labels: `0` + - missing images: `0` + - missing label files: `0` + - low-variance tiles: `6` +- Visual inspection confirmed: + - dense positive tiles show yellow YOLO boxes over real orthophoto imagery; + - the six low-variance tiles are `arendonk_heide` pure-empty negative validation tiles with no labels and blank-looking imagery. + +## Known limitations + +- The contact sheet now makes visual label inspection possible, but it also proves that the current clean AOI1024 dataset still contains blank-looking pure-empty negative tiles. +- Those blank/low-variance negatives should not be used blindly for the next training run. They can distort the background corpus and do not represent realistic aerial false-positive pressure. +- The current inactive model/promotion state remains unchanged. + +## Next recommended pass + +- Add no-data/low-variance filtering to the operator YOLO tile export path, regenerate the clean AOI1024 dataset, rerun the contact-sheet QA, and only then consider another training attempt. diff --git a/docs/TODO.md b/docs/TODO.md index d559ce59..693382a2 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -130,7 +130,8 @@ This file now starts with the current implementation status. Older preparation/b - [x] Rerun split background matrices on Tower after rebuild, then recalibrate against the cleaner pure-empty gate plus separate sparse-context inspection matrix. - [x] Add guarded promoted-candidate activation helper requiring a promotion report path and exact candidate key before `.env` can be changed. - [x] Add per-sample YOLO dataset audit diagnostics for parsed labels, median box area, small-box share and AOI-specific warning codes. -- [ ] Add deterministic visual YOLO label QA contact sheets before spending more CPU on another training run. +- [x] Add deterministic visual YOLO label QA contact sheets before spending more CPU on another training run. +- [ ] Filter no-data/low-variance pure-empty negative tiles from operator YOLO exports before the next training run. - [ ] Apply promoted V1 default building detector only after explicit operator review of the emitted `.env` updates, followed by rebuild/restart and browser/runtime smoke. ## Sprint 8 status