Add multi-AOI calibration evidence portfolio
This commit is contained in:
@@ -1,3 +1,36 @@
|
||||
## Sprint 139 Multi-AOI calibration evidence portfolio (2026-07-08)
|
||||
|
||||
Changed:
|
||||
- Added `scripts/assemble_detection_calibration_evidence_portfolio.sh` for packaging multiple AOI calibration summaries and their persisted QA evidence bundles into one model-review portfolio.
|
||||
- The assembler reads `calibration-evidence-portfolio-manifest.json`, copies each AOI summary into a sample folder, runs the existing `scripts/export_detection_calibration_evidence.sh` exporter per sample and writes:
|
||||
- `calibration_evidence_portfolio.json`
|
||||
- `calibration_evidence_portfolio.md`
|
||||
- Added optional `CURL_BIN` support to `scripts/export_detection_calibration_evidence.sh` so operator smokes/tests can inject a deterministic endpoint mock while defaulting to normal `curl`.
|
||||
- Added readiness syntax coverage and operator docs for the manifest convention.
|
||||
- Updated `scripts/README.md`, `CHANGELOG.md` and `docs/TODO.md`.
|
||||
- Added regression coverage in `backend/tests/test_sprint139_multi_aoi_calibration_evidence_portfolio.py`.
|
||||
|
||||
Tested:
|
||||
- Red step: `python -m pytest backend\tests\test_sprint139_multi_aoi_calibration_evidence_portfolio.py -q` failed while `scripts/assemble_detection_calibration_evidence_portfolio.sh` was absent.
|
||||
- `python -m pytest backend\tests\test_sprint139_multi_aoi_calibration_evidence_portfolio.py -q` (`1 passed`)
|
||||
- `python -m pytest backend\tests\test_sprint139_multi_aoi_calibration_evidence_portfolio.py backend\tests\test_sprint138_calibration_evidence_bundle_smoke.py backend\tests\test_sprint137_browser_calibration_summary_evidence_script.py backend\tests\test_sprint125_detection_calibration_evidence_bundle.py -q` (`4 passed`)
|
||||
- `bash -n scripts/assemble_detection_calibration_evidence_portfolio.sh`
|
||||
- `bash scripts/assemble_detection_calibration_evidence_portfolio.sh --help`
|
||||
- `bash -n scripts/export_detection_calibration_evidence.sh`
|
||||
- `bash scripts/export_detection_calibration_evidence.sh --help`
|
||||
- `python -m compileall backend/app`
|
||||
- `bash scripts/run_readiness_check.sh` (`418 passed`; frontend typecheck/build passed; Alembic head `202606120900`; shell syntax gates passed)
|
||||
|
||||
Open:
|
||||
- None for this pass.
|
||||
|
||||
Limitations:
|
||||
- This is local/operator evidence packaging only. It does not run inference, call live production data by itself, mutate application data, add backend endpoints, change migrations, create QA metrics, promote thresholds, download models, add provider fetching or change frontend runtime behavior.
|
||||
- The regression test uses mocked canonical QA evidence responses; real persisted QA evidence remains validated by running the portfolio assembler against live Detection Lab or calibration-sweep summaries.
|
||||
|
||||
Next recommended pass:
|
||||
- Run the portfolio assembler against the existing Tower calibration summaries for at least two real AOIs, then use the portfolio JSON/Markdown as the first model-review handoff artifact before any further training or threshold promotion.
|
||||
|
||||
## Sprint 138 Browser calibration evidence bundle smoke (2026-07-08)
|
||||
|
||||
Changed:
|
||||
|
||||
Reference in New Issue
Block a user