Filter low variance YOLO negative tiles
GeoIntel CI / docs-smoke (push) Has been cancelled
GeoIntel CI / contract-smoke (push) Has been cancelled

This commit is contained in:
Codex
2026-07-11 22:57:59 +02:00
parent 2a3472b919
commit a159370a11
7 changed files with 306 additions and 2 deletions
@@ -0,0 +1,34 @@
# YOLO Low-Variance Negative Filter Design
## Goal
Prevent blank/no-data pure-empty negative tiles from entering the next operator YOLO training dataset while keeping the exporter conservative and auditable.
## Scope
This pass changes only operator-side YOLO tile export tooling. It does not change backend APIs, database schema, frontend behavior, model activation, training behavior, provider fetching or inference.
## Approach
Add an opt-in filter to `scripts/export_operator_yolo_tile_dataset.py` that evaluates raster tile image variance before writing negative tiles. The filter applies only after labels are computed and only when a tile is negative. Positive tiles are never dropped by this gate, even if visually low-variance, because dropping labeled data silently would be a worse failure mode.
The filter will use the same simple max-min grayscale range heuristic as the contact-sheet QA script. A tile with range at or below `OPERATOR_YOLO_BLANK_RANGE_THRESHOLD` is treated as low-variance. When `--drop-low-variance-negatives` is enabled, that negative tile is skipped and recorded in summary fields instead of being written into `images/` and `labels/`.
## Reporting
The dataset summary must include:
- `drop_low_variance_negatives`
- `blank_range_threshold`
- `skipped_low_variance_negative_tile_count`
- skipped tile records with `skip_reason="low_visual_variance_negative"`
Kept tile records should include `low_visual_variance` so downstream contact-sheet and audit tooling can expose the signal.
## Validation
Add a regression test that constructs a blank negative raster window and a patterned negative raster window, enables the filter, and verifies that only the blank negative is skipped for `low_visual_variance_negative`.
## Known Limitation
The heuristic is intentionally simple. It identifies blank/no-data tiles, not semantic quality. Operator visual QA remains required before another training run.