Classify operator background corpus

docs/AI_PIPELINES.md

@@ -247,6 +247,7 @@ considered as a default:
 
 ```bash
 OPERATOR_SAMPLE_MANIFEST_PATH=storage/operator-data/operator_samples_manifest.json \
+OPERATOR_BACKGROUND_CATEGORIES="pure_empty_negative" \
 OPERATOR_BACKGROUND_SAMPLE_SLUGS="postel_bos lommel_heide kasterlee_bos" \
 QUALITY_MODEL_ASSET_IDS="geointel-building-yolov8n-expanded160e50-pt geointel-building-yolov8n-tile30-pt yolov8s-building-segmentation-pt" \
 QUALITY_TILE_SIZES="640" \
@@ -257,8 +258,13 @@ bash scripts/run_operator_hard_negative_detection_matrix.sh http://192.168.10.15
 
 The hard-negative matrix uploads only background rasters and counts detections
 as false-positive pressure. It does not run QA/QC or invent reference metrics
-for empty/sparse background AOIs. The first expanded local model improved dense
-AOI F1, but Kasterlee-bos false positives block default promotion.
+for empty/sparse background AOIs. Operator manifests classify background
+samples as `pure_empty_negative` when GRB returns zero reference buildings and
+`sparse_building_context` when contextual buildings are present. Use
+`OPERATOR_BACKGROUND_CATEGORIES="pure_empty_negative"` for default-promotion
+hard-negative gates, then run `sparse_building_context` as a separate review
+matrix. The first expanded local model improved dense AOI F1, but Kasterlee-bos
+false positives block default promotion.
 
 The current inactive AOI1024 background-aware local model asset,
 `geointel-building-yolov8s-aoi1024bg512r3e50-pt`, is exposed in Detection Lab

docs/CODEX_EXECUTION_LOG.md

@@ -6197,3 +6197,41 @@ Open:
 
 - The profiles are review/demo aids only. The background corpus still needs to be split into pure-empty negatives and sparse-building contextual AOIs before retraining or recalibrating for a default detector decision.
 - No backend API contract, migration, provider fetching, fake detection output, model download behavior or active runtime default changed.
+
+# Sprint 156 - Background corpus classification
+
+## What changed
+
+- Added explicit background category classification to operator sample preparation:
+  - `pure_empty_negative` when a background candidate has zero GRB reference buildings.
+  - `sparse_building_context` when a background candidate has one or more GRB reference buildings.
+  - `reference_aoi` for normal positive reference samples.
+- Persisted `background_category` into generated operator sample manifests and reference GeoJSON metadata.
+- Added `OPERATOR_BACKGROUND_CATEGORIES` to `scripts/run_operator_hard_negative_detection_matrix.sh` so the strict default-promotion hard-negative gate can run only on `pure_empty_negative` samples, while `sparse_building_context` samples can be reviewed separately.
+- Preserved `background_category` in YOLO tile export metadata so negative-tile provenance survives training dataset audits.
+- Updated operator pipeline docs, TODO and changelog.
+
+## What was tested
+
+- Added regression coverage in `backend/tests/test_sprint156_background_corpus_classification.py`.
+- Ran `python -m pytest tests/test_sprint156_background_corpus_classification.py -q`.
+- Ran `python -m pytest tests/test_sprint156_background_corpus_classification.py tests/test_sprint131_operator_sample_expansion.py tests/test_sprint132_operator_hard_negative_matrix.py tests/test_sprint130_operator_yolo_tile_dataset.py -q`: 17 passed.
+- Ran `python -m compileall backend/app`.
+- Ran `python -m pytest` in `backend`: 439 passed.
+- Ran `cd frontend && npm run typecheck`.
+- Ran `cd frontend && npm run build`.
+- Ran `bash scripts/run_readiness_check.sh`.
+- Ran `cd backend && python -m alembic heads` and `cd backend && python -m alembic upgrade head --sql`.
+- Ran `bash -n scripts/live_migration_smoke.sh` and `bash -n scripts/run_operator_hard_negative_detection_matrix.sh`.
+
+## Known limitations
+
+- This pass adds the cleaner corpus/gate contract only. It does not regenerate Tower manifests, retrain YOLO, rerun the live hard-negative matrices or change any model default.
+- No backend API contract, database migration, provider fetching, fake detection output or model download behavior changed.
+
+## Next recommended pass
+
+- Redeploy/rebuild the runtime scripts, regenerate the operator sample manifest, then run:
+  - `OPERATOR_BACKGROUND_CATEGORIES="pure_empty_negative"` for the strict default-promotion false-positive gate.
+  - `OPERATOR_BACKGROUND_CATEGORIES="sparse_building_context"` for contextual review evidence.
+- Retrain or recalibrate the inactive AOI1024 local model candidate only after those two matrices are available.

docs/TODO.md

@@ -121,7 +121,8 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Train and gate `geointel-building-yolov8s-aoi1024clean512e50-pt` through seven positive AOIs and nine hard-negative/background AOIs.
 - [x] Train and gate background-aware `geointel-building-yolov8s-aoi1024bg512r3e50-pt`; it is the strongest positive-AOI candidate so far but remains inactive because full background-candidate false-positive pressure still blocks default promotion.
 - [x] Add explicit operator detection profiles for local model assets: balanced review around threshold `0.15` and conservative high-precision review around threshold `0.35`, both clearly marked as non-default-approved until promotion gates pass.
-- [ ] Split the background corpus into pure-empty negatives and sparse-building contextual AOIs, then retrain or recalibrate against the cleaner gate.
+- [x] Add pure-empty versus sparse-building contextual background corpus classification to operator manifests, hard-negative matrix filters and YOLO tile provenance.
+- [ ] Retrain or recalibrate against the cleaner pure-empty gate plus separate sparse-context inspection matrix.
 - [ ] Promote a V1 default building detector only after it passes seven positive AOIs, clean hard-negative/background gates and persisted QA/QC evidence without fake detections or model downloads.
 
 ## Sprint 8 status