Filter low variance YOLO negative tiles

2026-07-11 22:57:59 +02:00
parent 2a3472b919
commit a159370a11
7 changed files with 306 additions and 2 deletions
@@ -0,0 +1,56 @@
+# YOLO Low-Variance Negative Filter Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Add opt-in filtering for blank/low-variance negative tiles in the operator YOLO tile exporter.
+
+**Architecture:** Keep the behavior inside `scripts/export_operator_yolo_tile_dataset.py` because this is operator-only dataset construction, not application inference. Compute low-variance from the raster window image array, skip only negative tiles when explicitly enabled, and expose all decisions in `yolo_tile_dataset_summary.json`.
+
+**Tech Stack:** Python, pytest, rasterio/Pillow runtime helpers already used by the exporter.
+
+---
+
+### Task 1: Regression Test
+
+**Files:**
+- Modify: `backend/tests/test_sprint130_operator_yolo_tile_dataset.py`
+
+- [x] Add a test that monkeypatches `image_array_from_raster_window` and verifies `export_sample_tiles` skips only low-variance negative tiles when `drop_low_variance_negatives=True`.
+- [x] Run the targeted test and confirm it fails because the exporter does not yet accept/report the new filter fields.
+
+### Task 2: Exporter Implementation
+
+**Files:**
+- Modify: `scripts/export_operator_yolo_tile_dataset.py`
+
+- [x] Add CLI/env options:
+  - `--drop-low-variance-negatives`
+  - `--blank-range-threshold`
+- [x] Add a helper to detect low visual variance from an image array.
+- [x] Thread the options into `export_sample_tiles`.
+- [x] Skip only negative low-variance tiles when the option is enabled.
+- [x] Add `low_visual_variance` to kept tile records.
+- [x] Add skipped records with `skip_reason="low_visual_variance_negative"`.
+- [x] Add summary fields for the option, threshold and skipped count.
+
+### Task 3: Documentation
+
+**Files:**
+- Modify: `scripts/README.md`
+- Modify: `docs/CODEX_EXECUTION_LOG.md`
+- Modify: `docs/TODO.md`
+
+- [x] Document the new operator export flags and intended Tower command.
+- [x] Record local validation and known limitation.
+- [x] Mark the low-variance export filter item as implemented after validation.
+
+### Task 4: Validation and Handoff
+
+**Files:**
+- No additional file changes expected.
+
+- [x] Run targeted pytest for the exporter/contact-sheet tests.
+- [x] Run `bash scripts/run_readiness_check.sh`.
+- [ ] Commit and push.
+- [ ] Rebuild/deploy Tower if code changed.
+- [ ] Regenerate the AOI1024 dataset with the new filter enabled, then render contact sheets and record the result.