diff --git a/CHANGELOG.md b/CHANGELOG.md index 446d52d3..d93a5490 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,14 @@ # Changelog +## Sprint 162 Split-background promotion runtime pass (2026-07-11) + +- Hardened split-background preflight compatibility for legacy operator manifests by deriving missing background categories from `reference_feature_count`. +- Ran Tower split-background promotion evidence against `http://192.168.10.150:1202`. +- Produced a high-threshold promote candidate: `geointel-building-yolov8s-aoi1024bg512r3e50-pt|512|64|0.35`. +- Verified the strict pure-empty background gate: 3 samples, 9 runs, 0 detections. +- Preserved the current model default; no automatic activation, provider fetching, fake outputs, model downloads, API contracts or migrations changed. + ## Sprint 161 Widescreen workbench support (2026-07-10) - Added dedicated `1800px` and `2200px` frontend layout breakpoints for wide and ultrawide monitors. diff --git a/docs/CODEX_EXECUTION_LOG.md b/docs/CODEX_EXECUTION_LOG.md index 3b2f27be..e46d7b1e 100644 --- a/docs/CODEX_EXECUTION_LOG.md +++ b/docs/CODEX_EXECUTION_LOG.md @@ -6467,3 +6467,68 @@ Open: - Manually refresh the already-open `http://192.168.10.150:1202` browser tab on the actual widescreen monitor to observe the new layout. - Continue with the split-background promotion preflight/matrix workflow now that Tower is rebuilt with the latest frontend and AI runtime. + +# Sprint 162 - Split-background promotion runtime pass + +## What changed + +- Hardened `scripts/run_split_background_promotion_workflow.sh` preflight so legacy operator manifests without explicit `background_category` are handled consistently with `run_operator_hard_negative_detection_matrix.sh`. +- Missing background categories are now derived from `reference_feature_count`: `0` becomes `pure_empty_negative`, and background samples with references become `sparse_building_context`. +- Added a regression test for the derived-category preflight path. +- Updated operator docs and changelog. No API contract, migration, provider fetching, model default, fake output or model download behavior changed. + +## What was tested locally + +- Ran `python -m pytest tests/test_sprint159_split_promotion_workflow.py -q`: 4 passed. +- Ran `python -m pytest tests/test_sprint157_background_split_matrix_runner.py tests/test_sprint158_promotion_report_split_background.py tests/test_sprint159_split_promotion_workflow.py -q`: 9 passed. +- Ran `bash -n scripts/run_split_background_promotion_workflow.sh`. +- Ran `python -m compileall backend/app`. +- Ran `python -m pytest` in `backend`: 450 passed, 17 existing Pydantic protected-namespace warnings. +- Ran `npm run typecheck` in `frontend`. +- Ran `npm run build` in `frontend`. +- Ran `bash scripts/run_readiness_check.sh`: passed. + +## Tower runtime evidence + +- Pushed commit `a499c5f` and fast-forwarded the Tower checkout. +- Ran split-background preflight against `http://192.168.10.150:1202` with: + - `PROMOTION_POSITIVE_PORTFOLIO_PATH=artifacts/detection-quality-matrix/multi-sample/aoi1024bg512r3e50-full/multi_sample_quality_summary.json` + - `OPERATOR_SAMPLE_MANIFEST_PATH=storage/operator-data/operator-samples-1024/operator_samples_manifest.json` + - result: passed. +- Ran the full low/mid-threshold split workflow into: + - background split: `artifacts/detection-hard-negatives/background-split/aoi1024bg512r3e50-split-20260710T222641Z` + - promotion report: `artifacts/detection-model-promotion/split-aware/aoi1024bg512r3e50-split-20260710T222641Z/detection_model_promotion_report.json` + - result: no recommended candidate because `0.15` produced 46 pure-empty detections on `postel_bos`, while `0.35` had no matching positive evidence in that portfolio. +- Ran high-threshold preflight and full split workflow using: + - `PROMOTION_POSITIVE_PORTFOLIO_PATH=artifacts/detection-quality-matrix/multi-sample/aoi1024bg512r3e50-high-threshold/multi_sample_quality_summary.json` + - `QUALITY_THRESHOLDS="0.35 0.45 0.6"` + - background split: `artifacts/detection-hard-negatives/background-split/aoi1024bg512r3e50-high-threshold-split-20260710T222934Z` + - promotion report: `artifacts/detection-model-promotion/split-aware/aoi1024bg512r3e50-high-threshold-split-20260710T222934Z/detection_model_promotion_report.json` +- High-threshold strict pure-empty gate passed: + - 3 pure-empty samples: `arendonk_heide`, `lommel_heide`, `postel_bos` + - 9 pure-empty runs + - total detections: `0` + - max detections per sample/threshold run: `0` +- Recommended candidate: + - `geointel-building-yolov8s-aoi1024bg512r3e50-pt|512|64|0.35` + - positive samples: `7` + - pure-empty background samples: `3` + - mean F1: `0.32086574003576274` + - mean precision: `0.8400057773951873` + - mean recall: `0.20213514285308795` + - max pure-empty detections: `0` + - promotion status: `promote_candidate` +- Sparse-context review remained review-only: + - 6 sparse-context samples + - 18 sparse-context runs + - total detections: `422` + - max sparse-context detections: `55` + +## Known limitations + +- The recommended candidate has conservative recall (`0.2021`) at threshold `0.35`; it is suitable as an operator-review candidate, not as an automatically activated production default. +- The model default was not changed. Activation should remain a separate explicit operator decision after reviewing the high-threshold report. + +## Next recommended pass + +- Add a guarded model activation/operator-selection workflow that can mark a promoted candidate as active only after the report artifact and candidate key are explicitly supplied.