Promote focused small-building detector

docs/AI_PIPELINES.md

@@ -335,17 +335,22 @@ hard-negative gates, then run `sparse_building_context` as a separate review
 matrix. The first expanded local model improved dense AOI F1, but Kasterlee-bos
 false positives block default promotion.
 
-The expanded-AOI local model asset,
-`geointel-building-yolov8s-aoi1024expandedminpx4vis035e50-pt`, is the current
+The focused small-building local model asset,
+`geointel-building-yolov8s-smallbld-minpx3-img640-ft30-pt`, is the current
 recommended Detection Lab operator profile. Use tile size `512`, overlap `64`
-and confidence threshold `0.15`. Persisted QA/QC across seven positive AOIs
-measured mean precision `0.6471`, recall `0.4700` and F1 `0.5433`; the strict
-three-sample pure-empty background gate produced zero detections. The previous
-`geointel-building-yolov8s-aoi1024bg512r3e50-pt` model remains available as a
-legacy conservative `0.35` review profile. Sparse-context detections remain
-review-only evidence, not a default-promotion blocker. Persistent misses are
-concentrated in small buildings, so every production-like run still requires
-persisted QA/QC against suitable reference data.
+and confidence threshold `0.15`. Persisted QA/QC at match IoU `0.25` across
+seven positive AOIs measured mean precision `0.5898`, recall `0.5770` and F1
+`0.5825`; minimum per-AOI F1 was `0.5528`. The strict three-sample pure-empty
+background gate produced zero detections. Compared with the previous balanced
+profile, the same persisted reference populations contain 1,571 fewer false
+negatives, including 745 fewer misses in the 25-100 m2 bucket and 181 fewer
+below 25 m2. This recall gain increases the false-positive review load, so the
+previous `geointel-building-yolov8s-aoi1024expandedminpx4vis035e50-pt` profile
+remains available as a higher-precision legacy `0.15` choice. The older
+`geointel-building-yolov8s-aoi1024bg512r3e50-pt` remains the conservative
+`0.35` profile. Sparse-context detections remain review-only evidence, not a
+default-promotion blocker. Every production-like run still requires persisted
+QA/QC against suitable reference data.
 
 To update a Tower/Unraid `.env` from a promoted report, use the guarded
 activation helper. It validates the exact report candidate key, verifies that
@@ -355,8 +360,8 @@ when `--apply` is supplied:
 
 ```bash
 python scripts/activate_promoted_yolo_candidate.py \
-  --promotion-report artifacts/detection-model-promotion/split-aware/aoi1024expandedminpx4vis035e50-split/detection_model_promotion_report.json \
-  --candidate-key 'geointel-building-yolov8s-aoi1024expandedminpx4vis035e50-pt|512|64|0.15' \
+  --promotion-report storage/operator-data/model-review/small-building-candidate/promotion/detection_model_promotion_report.json \
+  --candidate-key 'geointel-building-yolov8s-smallbld-minpx3-img640-ft30-pt|512|64|0.15' \
   --models-dir /mnt/user/appdata/geointel/models \
   --env-file /mnt/user/appdata/geointel/.env \
   --json
@@ -440,6 +445,13 @@ and records the positive/negative tile counts. This gives the training smoke
 more image samples while preserving the same explicit operator-data and QA/QC
 validation boundary.
 
+Focused small-building experiments use Beerse, Rijkevorsel, Hoogstraten and
+Vorselaar as training AOIs, with Vosselaar and Grobbendonk retained as
+independent validation AOIs. The exporter accepts an explicit `--samples`
+subset and records `source_manifest_sample_count`, `selected_sample_slugs` and
+`excluded_sample_slugs` in its summary. Manifest-backed validation samples
+cannot silently enter training.
+
 For visual error inspection, export the persisted QA evidence from a calibration
 summary:
 

docs/CODEX_EXECUTION_LOG.md

@@ -7071,3 +7071,68 @@ Open:
 - Embedded PostGIS live migration smoke passed with PostGIS `3.6`, required tables/indexes, database collation and the single Alembic head `202606120900`.
 - Browser validation confirmed that the recommended profile selects the exact active asset and threshold `0.15`, while live preflight displays CPU dependency/model readiness and the expected missing-manifest guard before dataset handoff.
 - Browser console warnings/errors: `0`.
+
+# Sprint 174 - Focused small-building recovery and promotion
+
+## Data and training evidence
+
+- Converted the Sprint 173 persistent false-negative audit into one focused real-data experiment instead of extending the same corpus blindly.
+- Added Beerse, Rijkevorsel, Hoogstraten and Vorselaar as training AOIs and Vosselaar/Grobbendonk as independent tile-level validation AOIs.
+- Kept Turnhout, Retie and Westerlo outside the tile corpus as operation-level holdouts.
+- Exported `/app/storage/operator-data/yolo-building-aoi1024-smallbld-minpx3vis035` from an explicit 23-sample manifest subset:
+  - 198 retained tiles;
+  - 180 positive and 18 negative tiles;
+  - 58,820 real GRB-derived labels;
+  - 48 visually reviewed tiles;
+  - zero invalid labels, missing images, missing label files or low-variance review selections.
+- The accepted `min-label-px=3` corpus retained 1,228 more genuine small-building labels than the comparable `min-label-px=4` export.
+- Trained one inactive 30-epoch CPU candidate from the previous active local model:
+  - model: `geointel-building-yolov8s-smallbld-minpx3-img640-ft30.pt`;
+  - model SHA256: `a9088b8491dfae36694b53e9e9406cb4e3511d334a5712fa34f75078a47759c1`;
+  - dataset-summary SHA256: `49b2a07d2105d08356431757b83eafc1498eaf1fb76965b1efe05b776824942a`;
+  - dataset-YAML SHA256: `3a2ea97c35a18072a1ab6738cd673c0ecec5344b19461c91d72a15e138d46e8d`;
+  - no model download and no fake training or QA data.
+
+## Persisted promotion evidence
+
+- Evaluated the exact fixed profile `tile=512`, `overlap=64`, `confidence=0.15` with QA match IoU explicitly fixed at `0.25`.
+- Seven positive AOIs produced:
+  - mean precision `0.5898197518`;
+  - mean recall `0.5769921004`;
+  - mean F1 `0.5824578632`;
+  - minimum per-AOI F1 `0.5527837436`.
+- Every AOI improved F1 relative to the previous balanced model. Turnhout improved from `0.4897494305` to `0.5527837436`.
+- The strict pure-empty gate covered Postel, Lommel and Arendonk and produced zero detections for every sample.
+- The formal promotion report recommended the exact key `geointel-building-yolov8s-smallbld-minpx3-img640-ft30-pt|512|64|0.15`.
+- Fixed-reference object evidence used identical GRB feature identities and reduced false negatives from `7,753` to `6,182`:
+  - 1,571 fewer total false negatives;
+  - 745 fewer misses in the 25-100 m2 bucket;
+  - 181 fewer misses below 25 m2;
+  - all seven AOIs improved.
+- Remaining persistent misses total `5,838`, concentrated in Turnhout, Herentals and Geel and still dominated by small buildings.
+- Mean precision decreased from `0.6470590036` to `0.5898197518`. The new profile is therefore a recall-balanced operator default with a higher false-positive review load, not ground truth.
+
+## Repository hardening
+
+- Added explicit `--samples` / `OPERATOR_YOLO_SAMPLES` corpus selection with selected/excluded sample provenance and unknown-sample rejection.
+- Added persistent false-negative area statistics, size buckets and combined GeoJSON review evidence.
+- Copied the complete operator evaluation/promotion toolchain into the all-in-one image and added a regression that rejects every Docker `COPY scripts/...` source that does not exist.
+- Removed Pydantic protected-namespace warnings for legitimate `model_*` API fields while preserving all schema field names and response contracts.
+- Updated Detection Lab profiles: the new small-building profile is recommended, the previous expanded profile remains the higher-precision legacy choice, and the background-aware `0.35` profile remains conservative.
+- Guarded activation first returned `ready_to_apply`; the reviewed `--apply` pass updated only `GEOINTEL_INSTALL_AI`, `YOLO_ENABLED`, `YOLO_MODELS_DIR` and `YOLO_MODEL_PATH` in the Tower environment.
+
+## Local validation
+
+- `python -m compileall backend/app`: passed.
+- `python -m pytest`: 472 passed.
+- `python -m ruff check` for all changed Python modules/tests: passed.
+- `npm run typecheck`: passed.
+- `npm run build`: passed; app bundle `215.64 kB`, MapLibre bundle `801.82 kB` before gzip.
+- `bash scripts/run_readiness_check.sh`: passed with 472 tests.
+- `python -m alembic heads`: one head, `202606120900`.
+- `python -m alembic upgrade head --sql`: complete migration chain rendered successfully.
+- Shell syntax checks passed for live migration and the full operator evaluation/promotion chain.
+
+## Next recommended pass
+
+- After redeploy, verify the active model SHA, local model-load preflight, live PostGIS migration smoke and browser profile selection. Then review false-positive evidence and the remaining 5,838 persistent misses before any further training.

docs/TODO.md

@@ -137,7 +137,8 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Add deterministic dataset/base/trained-model SHA256 provenance to future operator training summaries.
 - [x] Review per-AOI false-negative evidence, expand positive sample/label coverage and verify the resulting candidate improves false-negative rate in every validated AOI.
 - [x] Complete rebuild/restart and browser/runtime smoke for the guarded promoted V1 building detector activation.
-- [ ] Expand focused small-building training evidence only after reviewing persistent false negatives from the promoted model; do not start another blind training run.
+- [x] Expand focused small-building training evidence after reviewing persistent false negatives, train one inactive candidate and pass it through positive, pure-empty and fixed-reference promotion evidence before guarded activation.
+- [ ] Review the remaining 5,838 persistent false negatives and the increased false-positive load before any further model training; do not start another blind run.
 
 ## Sprint 8 status
 
@@ -494,7 +495,7 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Regenerate Tower AOI1024 operator samples with paged GRB references, then re-export and audit labels before any new training attempt.
 - [x] Improve AOI1024 label quality before retraining: `yolo-building-aoi1024-cleanpx12vis035` now audits `ok` with 14,632 labels, `min_label_px=12`, `min_label_visible_ratio=0.35`, median normalized box area `0.001373291015625` and small-box share `0.0`.
 - [x] Train and reject `geointel-building-yolov8s-aoi1024cleanpx12vis035e50-pt` through the positive/background promotion gate.
-- [ ] Keep every local YOLO candidate inactive until positive-AOI and hard-negative promotion reports recommend default activation.
+- [x] Keep every local YOLO candidate inactive until positive-AOI and hard-negative promotion reports recommend default activation.
 
 # Sprint 171 - Positive AOI expansion and small-building recovery
 
@@ -505,4 +506,6 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Refresh the full AOI1024 operator manifest on Tower and fetch only missing AOIs.
 - [x] Export and audit a low-minimum-label dataset without changing the active model.
 - [x] Render and inspect a sample-balanced label contact sheet before training.
-- [ ] Finish the inactive expanded-minpx4 candidate and run the full promotion gate.
+- [x] Finish the inactive expanded-minpx4 candidate and run the full promotion gate.
+- [x] Export the focused 23-sample minpx3 corpus with independent Vosselaar/Grobbendonk validation and external Turnhout/Retie/Westerlo holdouts.
+- [x] Train, audit and guarded-activate the focused small-building candidate only after all persisted promotion gates passed.