From 34f09fe323ef216537ed94fdac1f64e6a3bf02f0 Mon Sep 17 00:00:00 2001 From: Codex Date: Thu, 9 Jul 2026 01:47:08 +0200 Subject: [PATCH] Record operator YOLO dataset audit results --- CHANGELOG.md | 1 + docs/CODEX_EXECUTION_LOG.md | 8 +++++++- docs/TODO.md | 4 +++- scripts/README.md | 7 +++++++ 4 files changed, 18 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5b548451..dc63552f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1413,3 +1413,4 @@ Added: - Added `scripts/audit_operator_yolo_dataset_quality.py`, an operator-only YOLO tile dataset quality audit that produces JSON and Markdown reports for sample coverage, split coverage, repeated hard-negative pressure and label-size integrity before further training runs. - Added pytest coverage and readiness syntax checking for the new operator YOLO dataset audit script. +- Recorded live Tower audit results showing `yolo-building-tile-expanded160` as the clean current baseline and r4/r8 hard-negative datasets as repeat-heavy evidence sets that need more unique background AOIs before further hard-negative training. diff --git a/docs/CODEX_EXECUTION_LOG.md b/docs/CODEX_EXECUTION_LOG.md index 09b554e1..48175065 100644 --- a/docs/CODEX_EXECUTION_LOG.md +++ b/docs/CODEX_EXECUTION_LOG.md @@ -5655,6 +5655,12 @@ Open: ## What was tested - `python -m pytest backend\tests\test_sprint146_operator_yolo_dataset_quality_audit.py -q` +- `bash scripts/run_readiness_check.sh` +- Tower live audit after pulling commit `5898e54`: + - `yolo-building-tile-dataset`: `needs_attention`; only 3 positive samples and no background negatives. + - `yolo-building-tile-expanded160`: `ok`; 10 samples, 8 positive samples, 360 tiles, 11,213 labels, no missing/invalid label rows. + - `yolo-building-tile-hardneg160r4`: `needs_attention`; repeated background negatives are 91.1% of negative tiles. + - `yolo-building-tile-hardneg160r8`: `needs_attention`; repeated background negatives are 95.4% of negative tiles. ## Known limitations @@ -5663,4 +5669,4 @@ Open: ## Next recommended pass -- Run the audit against the existing Tower tile datasets and use the results to decide the next training-data expansion pass. +- Add more unique hard-negative/background AOIs before another hard-negative training run. The current label files are clean, so the bottleneck is dataset diversity and balance rather than label-file corruption. diff --git a/docs/TODO.md b/docs/TODO.md index 2289fda0..5ff894a3 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -441,4 +441,6 @@ This file now starts with the current implementation status. Older preparation/b - [x] Add a dataset/label-quality audit for generated operator YOLO tile datasets. - [x] Report sample coverage, validation coverage, repeated hard-negative pressure and YOLO label area integrity. - [x] Wire the audit script into the readiness syntax gate. -- [ ] Use live audit output to decide whether the next model pass needs more positive AOIs, label cleanup or unique hard negatives. +- [x] Use live audit output to decide whether the next model pass needs more positive AOIs, label cleanup or unique hard negatives. +- [ ] Add more unique background/hard-negative AOIs before repeating hard-negative-balanced YOLO training. +- [ ] Keep `yolo-building-tile-expanded160` as the clean current training baseline; avoid promoting r4/r8 repeat-heavy datasets as defaults. diff --git a/scripts/README.md b/scripts/README.md index 356f81a7..575d1c4a 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -345,6 +345,13 @@ missing or invalid label rows and normalized box-area signals. Treat next action is usually more positive AOIs, better validation coverage or more unique hard negatives rather than simply extending epochs. +Current Tower audit status: + +- `yolo-building-tile-expanded160`: clean baseline; no missing/invalid labels. +- `yolo-building-tile-hardneg160r4` and `yolo-building-tile-hardneg160r8`: + repeat-heavy hard-negative variants; useful evidence, but add more unique + background AOIs before training another hard-negative-balanced candidate. + For hard-negative-balanced experiments, repeat only train-split negative tiles from samples marked `sample_role=background_candidate`: