Expand operator samples for YOLO hard negatives

docs/AI_PIPELINES.md

@@ -171,12 +171,17 @@ Documented operator samples can be prepared inside the all-in-one runtime
 container:
 
 ```bash
-docker exec -it geointel python /app/scripts/prepare_operator_real_data_samples.py --samples geel,mol,turnhout
+docker exec -it geointel python3 /app/scripts/prepare_operator_real_data_samples.py
 ```
 
 The helper fetches explicit Digitaal Vlaanderen orthophoto/GRB GBG sample pairs
 for the documented AOIs only and writes `operator_samples_manifest.json`. The
-application itself still does not perform live provider fetching.
+default corpus includes dense reference AOIs for Geel, Mol, Turnhout, Herentals,
+Balen, Retie and Westerlo plus explicitly marked background candidates for
+Postel-bos, Lommel-heide and Kasterlee-bos. Background candidates may persist
+empty GRB FeatureCollections for negative-tile training; normal reference AOIs
+still fail on empty GRB responses. The application itself still does not perform
+live provider fetching.
 
 For confidence-threshold calibration, use the sweep wrapper:
 

docs/CODEX_EXECUTION_LOG.md

@@ -1,3 +1,54 @@
+## Sprint 131 Operator sample expansion and negative-tile YOLO candidate (2026-07-07)
+
+Changed:
+- Extended `scripts/prepare_operator_real_data_samples.py` with `sample_role` and `allow_empty_reference`.
+- Added reference AOIs for Herentals, Balen, Retie and Westerlo.
+- Added background-candidate AOIs for Postel-bos, Lommel-heide and Kasterlee-bos. Background candidates can persist empty GRB FeatureCollections for negative-tile training, while normal reference samples still fail on empty GRB results.
+- Added regression coverage in `backend/tests/test_sprint131_operator_sample_expansion.py`.
+- Updated `scripts/README.md`, `backend/README.md`, `docs/AI_PIPELINES.md`, `docs/TODO.md` and `CHANGELOG.md`.
+
+Tested:
+- RED: `python -m pytest backend\tests\test_sprint131_operator_sample_expansion.py -q` failed before the new sample metadata and background candidates existed.
+- `python -m pytest backend\tests\test_sprint131_operator_sample_expansion.py -q` passed.
+- `python -m py_compile scripts\prepare_operator_real_data_samples.py` passed.
+- `python scripts\prepare_operator_real_data_samples.py --help` passed.
+- Live Tower operator sample preparation passed:
+  - manifest: `/app/storage/operator-data/operator_samples_manifest.json`
+  - samples: Geel `617`, Mol `374`, Turnhout `773`, Herentals `665`, Balen `309`, Retie `592`, Westerlo `334`, Postel-bos `0`, Lommel-heide `0`, Kasterlee-bos `7` reference features.
+- Live Tower expanded tile export passed:
+  - dataset: `/app/storage/operator-data/yolo-building-tile-expanded160`
+  - tile size: `160`
+  - stride: `80`
+  - exported tiles: `360`
+  - positive tiles: `260`
+  - negative tiles: `100`
+  - labels: `11213`
+  - train tiles: `252`
+  - validation tiles: `108`
+- Live Tower 50-epoch CPU training passed:
+  - output model: `/app/models/geointel-building-yolov8n-expanded160e50.pt`
+  - catalog asset: `geointel-building-yolov8n-expanded160e50-pt`
+  - SHA256: `bf6a5e8d25a62d784ee53764ea11d7ce89c4e7aeeac7588010e497b8d7dafb2b`
+  - final validation: precision `0.428`, recall `0.389`, mAP50 `0.318`, mAP50-95 `0.106`
+- Live API preflight passed for `geointel-building-yolov8n-expanded160e50-pt` with `status=ready`, `model_load_ok=true`, `manifest_valid=true`, `tile_paths_exist=true`, `will_download_models=false` and `will_run_inference=false`.
+- Live 45-run multi-sample QA matrix completed:
+  - output: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/expanded160e50-live/multi_sample_quality_summary.json`
+  - command compared `geointel-building-yolov8n-expanded160e50-pt`, `geointel-building-yolov8n-tile30-pt` and `yolov8s-building-segmentation-pt` over Geel, Mol, Turnhout, Retie and Kasterlee-bos with tile `640`, overlap `64`, thresholds `0.25`/`0.15`/`0.05`.
+  - best overall score and recall: Geel, `geointel-building-yolov8n-expanded160e50-pt`, tile `640`, threshold `0.05`, 300 detections, 91 matches, 209 false positives, 526 false negatives, precision `0.30333333333333334`, recall `0.14748784440842788`, F1 `0.1984732824427481`.
+  - dense-sample score winners: Geel, Mol, Turnhout and Retie all selected `geointel-building-yolov8n-expanded160e50-pt`.
+  - hard-negative/sparse-sample winner: Kasterlee-bos selected `yolov8s-building-segmentation-pt`, threshold `0.25`, F1 `0.16666666666666666`; the expanded local model produced too many false positives there.
+
+Open:
+- None for the sample-preparation and expanded-training runtime proof itself.
+
+Limitations:
+- This remains operator tooling only. It does not add Training Studio, browser training controls, provider fetching, fake detections, model auto-provisioning or API contract changes.
+- `geointel-building-yolov8n-expanded160e50-pt` is the best tested candidate on dense operator AOIs, but it is still experimental and should not become the V1 default until hard-negative false positives improve.
+- The next model pass should add more sparse/background AOIs, tune confidence/NMS/max-detection settings and compare a stronger architecture or longer run against the same persisted QA matrix.
+
+Next recommended pass:
+- Build a hard-negative model-quality pass: expand sparse/background AOIs, export a balanced tile dataset, train a stronger candidate, and rerun the multi-sample QA matrix with dense and background samples scored separately.
+
 ## Sprint 130 Operator YOLO tile-level dataset tooling (2026-07-07)
 
 Changed:

docs/TODO.md

@@ -103,8 +103,10 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Add and benchmark a stronger `yolov8s` building-segmentation runtime model candidate.
 - [x] Add operator-only tile-level YOLO dataset export with overlapping windows and deterministic negative tile retention.
 - [x] Train and benchmark the first tile-level local YOLO candidate on Tower through the persisted QA/QC matrix.
-- [ ] Calibrate confidence, IoU and model selection against additional local orthophoto/reference samples beyond Geel/Mol/Turnhout.
-- [ ] Find or train a materially stronger aerial/Kempen building model candidate; `geointel-building-yolov8n-tile30-pt` is the best current overall candidate but still too weak for a V1 default.
+- [x] Calibrate confidence, IoU and model selection against additional local orthophoto/reference samples beyond Geel/Mol/Turnhout.
+- [x] Add negative/background AOIs to the operator sample corpus and train an expanded local tile-level YOLO candidate.
+- [ ] Add a hard-negative model-quality pass with more sparse/background AOIs, balanced tile export and explicit false-positive scoring.
+- [ ] Find or train a materially stronger aerial/Kempen building model candidate; `geointel-building-yolov8n-expanded160e50-pt` is the best current dense-AOI candidate but still too weak and too noisy for a V1 default.
 
 ## Sprint 8 status