Add hard-negative balanced YOLO tile export
This commit is contained in:
@@ -1,3 +1,57 @@
|
||||
## Sprint 133 Hard-negative-balanced YOLO candidate (2026-07-07)
|
||||
|
||||
Changed:
|
||||
- Hardened `scripts/export_operator_yolo_tile_dataset.py` with deterministic train-only background-negative repetition through `--background-negative-repeat` and `OPERATOR_YOLO_BACKGROUND_NEGATIVE_REPEAT`.
|
||||
- Background-negative repetition applies only when `is_negative=true`, `sample_role=background_candidate` and `split=train`; validation tiles, positive tiles and normal reference samples are not duplicated.
|
||||
- Added tile-level provenance fields `sample_role`, `repeat_index` and `is_repeated_background_negative`.
|
||||
- Added regression coverage in `backend/tests/test_sprint130_operator_yolo_tile_dataset.py`.
|
||||
- Updated `scripts/README.md`, `docs/TODO.md`, `docs/CODEX_EXECUTION_LOG.md` and `CHANGELOG.md`.
|
||||
|
||||
Tested:
|
||||
- RED: `python -m pytest backend\tests\test_sprint130_operator_yolo_tile_dataset.py -q` failed before `--background-negative-repeat` and `background_negative_repeat_count` existed.
|
||||
- `python -m pytest backend\tests\test_sprint130_operator_yolo_tile_dataset.py -q` passed.
|
||||
- `python -m py_compile scripts\export_operator_yolo_tile_dataset.py` passed.
|
||||
- `python scripts\export_operator_yolo_tile_dataset.py --help` passed.
|
||||
- Live Tower hard-negative-balanced tile export passed:
|
||||
- dataset: `/app/storage/operator-data/yolo-building-tile-hardneg160r8`
|
||||
- source samples: 10
|
||||
- tile size: `160`
|
||||
- stride: `80`
|
||||
- negative keep ratio: `1.0`
|
||||
- background negative repeat: `8`
|
||||
- exported tiles: `864`
|
||||
- positive tiles: `260`
|
||||
- negative tiles: `604`
|
||||
- labels: `11213`
|
||||
- train tiles: `756`
|
||||
- validation tiles: `108`
|
||||
- Live Tower 40-epoch CPU training passed:
|
||||
- output model: `/app/models/geointel-building-yolov8n-hardneg160r8e40.pt`
|
||||
- catalog asset: `geointel-building-yolov8n-hardneg160r8e40-pt`
|
||||
- SHA256: `7a77bd9f68e4c3927ffc8a8cd978a81067b02f42cffe77ada5334b5f8dbb6b50`
|
||||
- final validation: precision `0.403`, recall `0.378`, mAP50 `0.301`, mAP50-95 `0.0944`
|
||||
- Live API preflight passed for `geointel-building-yolov8n-hardneg160r8e40-pt` with `status=ready`, `model_load_ok=true`, `manifest_valid=true`, `tile_paths_exist=true`, `will_download_models=false` and `will_run_inference=false`.
|
||||
- Live 60-run multi-sample QA matrix completed:
|
||||
- output: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/hardneg160r8e40-live/multi_sample_quality_summary.json`
|
||||
- command compared `geointel-building-yolov8n-hardneg160r8e40-pt`, `geointel-building-yolov8n-expanded160e50-pt`, `geointel-building-yolov8n-tile30-pt` and `yolov8s-building-segmentation-pt` over Geel, Mol, Turnhout, Retie and Kasterlee-bos with tile `640`, overlap `64`, thresholds `0.25`/`0.15`/`0.05`.
|
||||
- best overall score and recall remained Geel with `geointel-building-yolov8n-expanded160e50-pt`, precision `0.30333333333333334`, recall `0.14748784440842788`, F1 `0.1984732824427481`.
|
||||
- hardneg160r8e40 dense F1 lagged expanded160e50 on Geel (`0.14394765539803708` vs `0.1984732824427481`), Mol (`0.11572700296735906` vs `0.1651651651651652`), Turnhout (`0.14911463187325258` vs `0.1938490214352283`) and Retie (`0.10538116591928251` vs `0.1569506726457399`).
|
||||
- Live 36-run hard-negative matrix completed:
|
||||
- output: `/mnt/user/appdata/geointel/artifacts/detection-hard-negatives/hardneg160r8e40-live/hard_negative_matrix_summary.json`
|
||||
- Postel-bos: hardneg160r8e40 produced 0/0/0 detections at thresholds `0.25`/`0.15`/`0.05`; expanded160e50 produced 0/0/1.
|
||||
- Lommel-heide: hardneg160r8e40 produced 0/0/0 detections; expanded160e50 produced 0/0/10.
|
||||
- Kasterlee-bos: hardneg160r8e40 produced 5/9/25 detections; expanded160e50 produced 38/46/76.
|
||||
|
||||
Open:
|
||||
- None for the hard-negative-balanced tile export contract itself.
|
||||
|
||||
Limitations:
|
||||
- `geointel-building-yolov8n-hardneg160r8e40-pt` reduced false-positive pressure but regressed dense-AOI recall/F1. It should not become the V1 default.
|
||||
- This remains operator tooling only. It does not add Training Studio, browser training controls, provider fetching, fake detections, model auto-provisioning or API contract changes.
|
||||
|
||||
Next recommended pass:
|
||||
- Train or import a materially stronger aerial/Kempen building model candidate, then benchmark it against the same dense QA and hard-negative matrices before changing default model selection.
|
||||
|
||||
## Sprint 132 Operator hard-negative detection matrix (2026-07-07)
|
||||
|
||||
Changed:
|
||||
|
||||
+1
-1
@@ -106,7 +106,7 @@ This file now starts with the current implementation status. Older preparation/b
|
||||
- [x] Calibrate confidence, IoU and model selection against additional local orthophoto/reference samples beyond Geel/Mol/Turnhout.
|
||||
- [x] Add negative/background AOIs to the operator sample corpus and train an expanded local tile-level YOLO candidate.
|
||||
- [x] Add a hard-negative model-quality pass with sparse/background AOIs and explicit false-positive scoring.
|
||||
- [ ] Train a hard-negative-balanced YOLO candidate and rerun dense QA plus background false-positive matrices.
|
||||
- [x] Train a hard-negative-balanced YOLO candidate and rerun dense QA plus background false-positive matrices.
|
||||
- [ ] Find or train a materially stronger aerial/Kempen building model candidate; `geointel-building-yolov8n-expanded160e50-pt` is the best current dense-AOI candidate but still too weak and too noisy for a V1 default.
|
||||
|
||||
## Sprint 8 status
|
||||
|
||||
Reference in New Issue
Block a user