Jens/geointel
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Filter low variance YOLO negative tiles
GeoIntel CI / docs-smoke (push) Has been cancelled
Details
GeoIntel CI / contract-smoke (push) Has been cancelled
Details

Browse Source
This commit is contained in:
Codex
2026-07-11 22:57:59 +02:00
parent 2a3472b919
commit a159370a11
7 changed files with 306 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/CODEX_EXECUTION_LOG.md
+38
View File
@@ -6866,3 +6866,41 @@ Open:
## Next recommended pass
- Add no-data/low-variance filtering to the operator YOLO tile export path, regenerate the clean AOI1024 dataset, rerun the contact-sheet QA, and only then consider another training attempt.
# Sprint 168 - Operator YOLO low-variance negative filtering
## What changed
- Added opt-in low-variance negative filtering to `scripts/export_operator_yolo_tile_dataset.py`.
- Added CLI/env controls:
- `--drop-low-variance-negatives` / `OPERATOR_YOLO_DROP_LOW_VARIANCE_NEGATIVES`;
- `--blank-range-threshold` / `OPERATOR_YOLO_BLANK_RANGE_THRESHOLD`.
- The filter evaluates the rendered raster tile image and skips only negative tiles when enabled.
- Positive/labeled tiles are never removed by this variance gate.
- Kept tile records now include `low_visual_variance`.
- Skipped blank/no-data negative records use `skip_reason="low_visual_variance_negative"`.
- Dataset summaries now include:
- `drop_low_variance_negatives`;
- `blank_range_threshold`;
- `skipped_low_variance_negative_tile_count`.
- Updated operator documentation with the refreshed AOI1024 cleanpx export command.
- Added design and execution plan docs under `docs/superpowers/`.
## Local validation
- RED: `python -m pytest backend/tests/test_sprint130_operator_yolo_tile_dataset.py::test_export_can_skip_low_variance_negative_tiles -q` failed because `export_sample_tiles()` did not accept `drop_low_variance_negatives`.
- GREEN: same targeted test passed after adding the filter.
- Ran `python -m pytest backend/tests/test_sprint130_operator_yolo_tile_dataset.py -q`: 8 passed.
- Ran `python -m pytest backend/tests/test_sprint130_operator_yolo_tile_dataset.py backend/tests/test_sprint167_operator_yolo_label_qa_contact_sheets.py backend/tests/test_docker_runtime_config.py::test_all_in_one_dockerfile_copies_operator_scripts_for_runtime_use backend/tests/test_docker_runtime_config.py::test_all_in_one_dockerfile_copies_operator_scripts_after_dependency_install -q`: 11 passed.
- Ran `python scripts/export_operator_yolo_tile_dataset.py --help`: the CLI exposes `--drop-low-variance-negatives`, `--no-drop-low-variance-negatives` and `--blank-range-threshold` without loading GIS dependencies.
- Ran `bash scripts/run_readiness_check.sh`: 460 backend tests passed, frontend typecheck passed, frontend build passed, readiness passed.
## Known limitations
- The low-variance gate is deliberately simple and only identifies visually blank/no-data-looking negative tiles.
- It is opt-in to avoid silently changing historical dataset exports.
- Operator visual contact-sheet review remains required before any new training run.
## Next recommended pass
- Redeploy Tower, regenerate the AOI1024 cleanpx dataset with `--drop-low-variance-negatives`, rerun dataset audit and contact-sheet QA, then decide whether the filtered dataset is suitable for another training run.