Document live multi-AOI calibration portfolio

docs/CODEX_EXECUTION_LOG.md

@@ -1,3 +1,42 @@
+## Sprint 140 Live multi-AOI calibration portfolio run (2026-07-08)
+
+Changed:
+- Created a Tower-local `calibration-evidence-portfolio-manifest.json` for existing persisted multi-sample quality matrix summaries:
+  - Geel: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/20260707T025932Z/geel/quality_matrix_summary.json`
+  - Mol: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/20260707T025932Z/mol/quality_matrix_summary.json`
+  - Turnhout: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/20260707T025932Z/turnhout/quality_matrix_summary.json`
+- Ran `scripts/assemble_detection_calibration_evidence_portfolio.sh` against the live Tower app at `http://127.0.0.1:1202`.
+- Produced the first real multi-AOI calibration evidence handoff:
+  - `/mnt/user/appdata/geointel/artifacts/detection-calibration-portfolio/live-20260708/output/calibration_evidence_portfolio.json`
+  - `/mnt/user/appdata/geointel/artifacts/detection-calibration-portfolio/live-20260708/output/calibration_evidence_portfolio.md`
+  - per-sample `calibration_evidence.geojson`, `calibration_evidence_summary.json` and `calibration_evidence_review.html` artifacts for Geel, Mol and Turnhout.
+
+Tested:
+- Tower manifest validation passed with `python3 -m json.tool`.
+- Live portfolio assembly passed against persisted QA evidence from the running app.
+- Output validation passed: portfolio JSON/Markdown and all per-sample evidence GeoJSON/summary/review files exist and are non-empty.
+- Tower runtime remained healthy on `0.0.0.0:1202->80/tcp`.
+
+Evidence:
+- Portfolio sample count: `3`.
+- Total evidence features: `5509`.
+- Combined role counts: `false_negative=5239`, `false_positive=164`, `match_candidate=53`, `match_reference=53`.
+- Best sample by score: `mol`.
+- Best per-sample scores:
+  - Geel: threshold `0.15`, F1/score `0.0187207488299532`, precision `0.25`, recall `0.009724473257698542`, evidence features `1885`.
+  - Mol: threshold `0.15`, F1/score `0.04195804195804196`, precision `0.16363636363636364`, recall `0.02406417112299465`, evidence features `1223`.
+  - Turnhout: threshold `0.15`, F1/score `0.03597122302158273`, precision `0.2459016393442623`, recall `0.019404915912031046`, evidence features `2401`.
+
+Open:
+- The live portfolio confirms the evidence pipeline works, but the evaluated model/threshold set is not yet operational-quality for building extraction due to very low recall and a dominant false-negative count.
+
+Limitations:
+- This pass produced operator artifacts only. It did not rerun inference, mutate application data, change database state, add endpoints, change frontend runtime behavior, download models, train models or rebuild the Docker app.
+- The portfolio input came from the existing `20260707T025932Z` quality matrix; the next model review should use fresh matrices when new AOIs or model candidates are added.
+
+Next recommended pass:
+- Add more positive and hard-negative AOIs, then run a fresh multi-sample quality matrix and portfolio for the next model candidate. Do not promote the current evaluated model as a V1 default.
+
 ## Sprint 139 Multi-AOI calibration evidence portfolio (2026-07-08)
 
 Changed:

docs/TODO.md

@@ -426,5 +426,6 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Allow the evidence bundle script to consume Detection Lab calibration summary exports.
 - [x] Add a local browser-summary QA evidence bundle smoke using mocked canonical evidence responses.
 - [x] Add a multi-AOI calibration evidence portfolio convention for model-review handoff.
+- [x] Run the first live multi-AOI calibration evidence portfolio on Tower for Geel, Mol and Turnhout.
 - [ ] Add more AOIs after the tile-level baseline so the next local model attempt is not limited to Geel/Mol/Turnhout.
 - [ ] Add negative/background AOIs so the next tile dataset is not all positive tiles.