Jens/geointel
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add operator hard-negative detection matrix
GeoIntel CI / docs-smoke (push) Has been cancelled
Details
GeoIntel CI / contract-smoke (push) Has been cancelled
Details

Browse Source
This commit is contained in:
Codex
2026-07-07 22:07:56 +02:00
parent 89c5729d33
commit 558c17129b
9 changed files with 648 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/CODEX_EXECUTION_LOG.md
+37
View File
@@ -1,3 +1,40 @@
## Sprint 132 Operator hard-negative detection matrix (2026-07-07)
Changed:
- Added `scripts/run_operator_hard_negative_detection_matrix.sh`.
- The script reads `operator_samples_manifest.json`, selects samples marked `background_candidate` or `allow_empty_reference`, uploads only the raster, generates a tile manifest, checks configured-YOLO preflight, runs `POST /api/v1/detection/run` and counts persisted detections.
- It intentionally does not upload reference vectors and does not call detection QA/QC endpoints, because background AOIs have no meaningful reference target.
- Added readiness shell-syntax coverage for the new script.
- Added regression coverage in `backend/tests/test_sprint132_operator_hard_negative_matrix.py`.
- Updated `scripts/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
Tested:
- RED: `python -m pytest backend\tests\test_sprint132_operator_hard_negative_matrix.py -q` failed while `scripts/run_operator_hard_negative_detection_matrix.sh` did not exist.
- `python -m pytest backend\tests\test_sprint132_operator_hard_negative_matrix.py -q` passed.
- `bash -n scripts/run_operator_hard_negative_detection_matrix.sh` passed.
- Live Tower 27-run hard-negative matrix completed:
- output: `/mnt/user/appdata/geointel/artifacts/detection-hard-negatives/expanded160e50-live/hard_negative_matrix_summary.json`
- samples: Postel-bos, Lommel-heide and Kasterlee-bos
- models: `geointel-building-yolov8n-expanded160e50-pt`, `geointel-building-yolov8n-tile30-pt`, `yolov8s-building-segmentation-pt`
- tile size: `640`
- overlap: `64`
- thresholds: `0.25`, `0.15`, `0.05`
- Live false-positive pressure results:
- Postel-bos: expanded160e50 produced 0 detections at `0.25`/`0.15`, 1 at `0.05`; tile30 produced 0/0/1; yolov8s produced 0/3/6.
- Lommel-heide: expanded160e50 produced 0 detections at `0.25`/`0.15`, 10 at `0.05`; tile30 produced 0/0/3; yolov8s produced 0/0/0.
- Kasterlee-bos: expanded160e50 produced 38/46/76 detections at `0.25`/`0.15`/`0.05`; tile30 produced 15/22/42; yolov8s produced 5/6/7.
Open:
- None for the hard-negative matrix tooling itself.
Limitations:
- Background matrix scores false-positive pressure from detection counts only. It does not calculate precision/recall/F1 because background candidates intentionally do not provide a full reference target.
- Kasterlee-bos still has 7 GRB features and is best interpreted as a sparse/hard-negative AOI, not a purely empty background tile.
- `geointel-building-yolov8n-expanded160e50-pt` should not be promoted to default model while Kasterlee-bos false-positive pressure remains high.
Next recommended pass:
- Train a hard-negative-balanced candidate: oversample sparse/background tiles, lower the dense-AOI max-detection bias, and rerun both dense QA matrix and hard-negative matrix before changing any default model selection.
## Sprint 131 Operator sample expansion and negative-tile YOLO candidate (2026-07-07)
Changed: