Add hard-negative balanced YOLO tile export

2026-07-07 23:01:51 +02:00
parent 558c17129b
commit 9bd6752128
6 changed files with 187 additions and 25 deletions
@@ -1,3 +1,57 @@
+## Sprint 133 Hard-negative-balanced YOLO candidate (2026-07-07)
+
+Changed:
+- Hardened `scripts/export_operator_yolo_tile_dataset.py` with deterministic train-only background-negative repetition through `--background-negative-repeat` and `OPERATOR_YOLO_BACKGROUND_NEGATIVE_REPEAT`.
+- Background-negative repetition applies only when `is_negative=true`, `sample_role=background_candidate` and `split=train`; validation tiles, positive tiles and normal reference samples are not duplicated.
+- Added tile-level provenance fields `sample_role`, `repeat_index` and `is_repeated_background_negative`.
+- Added regression coverage in `backend/tests/test_sprint130_operator_yolo_tile_dataset.py`.
+- Updated `scripts/README.md`, `docs/TODO.md`, `docs/CODEX_EXECUTION_LOG.md` and `CHANGELOG.md`.
+
+Tested:
+- RED: `python -m pytest backend\tests\test_sprint130_operator_yolo_tile_dataset.py -q` failed before `--background-negative-repeat` and `background_negative_repeat_count` existed.
+- `python -m pytest backend\tests\test_sprint130_operator_yolo_tile_dataset.py -q` passed.
+- `python -m py_compile scripts\export_operator_yolo_tile_dataset.py` passed.
+- `python scripts\export_operator_yolo_tile_dataset.py --help` passed.
+- Live Tower hard-negative-balanced tile export passed:
+  - dataset: `/app/storage/operator-data/yolo-building-tile-hardneg160r8`
+  - source samples: 10
+  - tile size: `160`
+  - stride: `80`
+  - negative keep ratio: `1.0`
+  - background negative repeat: `8`
+  - exported tiles: `864`
+  - positive tiles: `260`
+  - negative tiles: `604`
+  - labels: `11213`
+  - train tiles: `756`
+  - validation tiles: `108`
+- Live Tower 40-epoch CPU training passed:
+  - output model: `/app/models/geointel-building-yolov8n-hardneg160r8e40.pt`
+  - catalog asset: `geointel-building-yolov8n-hardneg160r8e40-pt`
+  - SHA256: `7a77bd9f68e4c3927ffc8a8cd978a81067b02f42cffe77ada5334b5f8dbb6b50`
+  - final validation: precision `0.403`, recall `0.378`, mAP50 `0.301`, mAP50-95 `0.0944`
+- Live API preflight passed for `geointel-building-yolov8n-hardneg160r8e40-pt` with `status=ready`, `model_load_ok=true`, `manifest_valid=true`, `tile_paths_exist=true`, `will_download_models=false` and `will_run_inference=false`.
+- Live 60-run multi-sample QA matrix completed:
+  - output: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/hardneg160r8e40-live/multi_sample_quality_summary.json`
+  - command compared `geointel-building-yolov8n-hardneg160r8e40-pt`, `geointel-building-yolov8n-expanded160e50-pt`, `geointel-building-yolov8n-tile30-pt` and `yolov8s-building-segmentation-pt` over Geel, Mol, Turnhout, Retie and Kasterlee-bos with tile `640`, overlap `64`, thresholds `0.25`/`0.15`/`0.05`.
+  - best overall score and recall remained Geel with `geointel-building-yolov8n-expanded160e50-pt`, precision `0.30333333333333334`, recall `0.14748784440842788`, F1 `0.1984732824427481`.
+  - hardneg160r8e40 dense F1 lagged expanded160e50 on Geel (`0.14394765539803708` vs `0.1984732824427481`), Mol (`0.11572700296735906` vs `0.1651651651651652`), Turnhout (`0.14911463187325258` vs `0.1938490214352283`) and Retie (`0.10538116591928251` vs `0.1569506726457399`).
+- Live 36-run hard-negative matrix completed:
+  - output: `/mnt/user/appdata/geointel/artifacts/detection-hard-negatives/hardneg160r8e40-live/hard_negative_matrix_summary.json`
+  - Postel-bos: hardneg160r8e40 produced 0/0/0 detections at thresholds `0.25`/`0.15`/`0.05`; expanded160e50 produced 0/0/1.
+  - Lommel-heide: hardneg160r8e40 produced 0/0/0 detections; expanded160e50 produced 0/0/10.
+  - Kasterlee-bos: hardneg160r8e40 produced 5/9/25 detections; expanded160e50 produced 38/46/76.
+
+Open:
+- None for the hard-negative-balanced tile export contract itself.
+
+Limitations:
+- `geointel-building-yolov8n-hardneg160r8e40-pt` reduced false-positive pressure but regressed dense-AOI recall/F1. It should not become the V1 default.
+- This remains operator tooling only. It does not add Training Studio, browser training controls, provider fetching, fake detections, model auto-provisioning or API contract changes.
+
+Next recommended pass:
+- Train or import a materially stronger aerial/Kempen building model candidate, then benchmark it against the same dense QA and hard-negative matrices before changing default model selection.
+
 ## Sprint 132 Operator hard-negative detection matrix (2026-07-07)

 Changed: