Document calibration evidence export results
This commit is contained in:
@@ -12,6 +12,7 @@
|
|||||||
- Added `scripts/export_detection_calibration_evidence.sh` to export persisted QA evidence from a detection calibration summary.
|
- Added `scripts/export_detection_calibration_evidence.sh` to export persisted QA evidence from a detection calibration summary.
|
||||||
- The script writes combined `calibration_evidence.geojson`, `calibration_evidence_summary.json` and a standalone `calibration_evidence_review.html` SVG artifact for matched detections, matched references, false positives and false negatives.
|
- The script writes combined `calibration_evidence.geojson`, `calibration_evidence_summary.json` and a standalone `calibration_evidence_review.html` SVG artifact for matched detections, matched references, false positives and false negatives.
|
||||||
- Added readiness syntax coverage and regression coverage for the evidence bundle contract.
|
- Added readiness syntax coverage and regression coverage for the evidence bundle contract.
|
||||||
|
- Ran the export on Tower for the latest Geel calibration sweep; the bundle contained 2555 evidence features: 2460 false negatives, 79 false positives, 8 matched detections and 8 matched references.
|
||||||
- No inference, model dependency, provider fetching, fake data, API contract or frontend runtime behavior changed.
|
- No inference, model dependency, provider fetching, fake data, API contract or frontend runtime behavior changed.
|
||||||
|
|
||||||
## Sprint 124 Detection calibration sweep tooling (2026-07-07)
|
## Sprint 124 Detection calibration sweep tooling (2026-07-07)
|
||||||
|
|||||||
@@ -17,16 +17,32 @@ Validation:
|
|||||||
- `bash -n scripts/export_detection_calibration_evidence.sh` passed.
|
- `bash -n scripts/export_detection_calibration_evidence.sh` passed.
|
||||||
- `bash scripts/export_detection_calibration_evidence.sh --help` passed.
|
- `bash scripts/export_detection_calibration_evidence.sh --help` passed.
|
||||||
- Missing-input guard printed usage and did not fetch evidence.
|
- Missing-input guard printed usage and did not fetch evidence.
|
||||||
|
- `bash scripts/run_readiness_check.sh` passed: 389 backend tests, Alembic head check, frontend typecheck/build and shell syntax checks.
|
||||||
|
- Tower pulled commit `ea8dcb2` with `git pull --ff-only origin main`.
|
||||||
|
- Tower evidence export passed against `/mnt/user/appdata/geointel/artifacts/detection-calibration/20260707T002103Z/calibration_summary.json`.
|
||||||
|
- Evidence artifacts written on Tower:
|
||||||
|
- `/mnt/user/appdata/geointel/artifacts/detection-calibration/20260707T002103Z/calibration_evidence.geojson`
|
||||||
|
- `/mnt/user/appdata/geointel/artifacts/detection-calibration/20260707T002103Z/calibration_evidence_summary.json`
|
||||||
|
- `/mnt/user/appdata/geointel/artifacts/detection-calibration/20260707T002103Z/calibration_evidence_review.html`
|
||||||
|
- Evidence counts from the Geel calibration export:
|
||||||
|
- total evidence features: 2555
|
||||||
|
- false negatives: 2460
|
||||||
|
- false positives: 79
|
||||||
|
- matched detections: 8
|
||||||
|
- matched references: 8
|
||||||
|
- Copied `calibration_evidence_review.html` and `calibration_evidence_summary.json` to the local ignored `artifacts/detection-calibration/20260707T002103Z/` directory for inspection.
|
||||||
|
- Internal browser validation passed through a temporary local static server: the review page loaded with title `GeoIntel Detection Calibration Evidence`, 1 SVG, 2555 SVG paths, 4 calibration table rows and 0 console errors.
|
||||||
|
|
||||||
Open:
|
Open:
|
||||||
- Pull the script to Tower and export evidence for the latest Geel calibration summary.
|
- None for evidence export tooling.
|
||||||
|
|
||||||
Limitations:
|
Limitations:
|
||||||
- This is operator tooling only. It does not rerun inference, change API behavior, add UI behavior, fetch providers, seed demo data or download models.
|
- This is operator tooling only. It does not rerun inference, change API behavior, add UI behavior, fetch providers, seed demo data or download models.
|
||||||
- The SVG review is a lightweight geometry overview, not a replacement for full MapLibre evidence review in the workbench.
|
- The SVG review is a lightweight geometry overview, not a replacement for full MapLibre evidence review in the workbench.
|
||||||
|
- The evidence distribution confirms the current active model misses most reference buildings on this sample. This points to model suitability and/or tiling strategy as the next bottleneck, not runtime plumbing.
|
||||||
|
|
||||||
Next recommended pass:
|
Next recommended pass:
|
||||||
- Run the evidence export on Tower, inspect the role distribution and use the artifacts to decide whether the current model is worth further threshold tuning or should be replaced.
|
- Add a model-quality decision pass: compare the current evaluation model against another building/aerial model or adjusted tile/overlap settings on at least two additional local orthophoto/reference samples.
|
||||||
|
|
||||||
## Sprint 124 Detection calibration sweep tooling (2026-07-07)
|
## Sprint 124 Detection calibration sweep tooling (2026-07-07)
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user