Add detection calibration sweep
GeoIntel CI / docs-smoke (push) Has been cancelled
GeoIntel CI / contract-smoke (push) Has been cancelled

This commit is contained in:
Codex
2026-07-07 02:19:28 +02:00
parent 673efd7a9d
commit 590e5975f5
9 changed files with 348 additions and 0 deletions
+15
View File
@@ -374,6 +374,21 @@ manifests generated for AI handoff include source CRS metadata so pixel-space
model outputs can be transformed to WGS84 GeoJSON coordinates. Current V1 upload
support is limited to GeoTIFF-style rasters and GeoJSON/JSON reference vectors.
For model-quality calibration, run the confidence sweep wrapper:
```bash
REAL_RASTER_PATH=/mnt/user/appdata/geointel/storage/operator-data/geel_orthophoto_wms_512.tif \
REAL_REFERENCE_VECTOR_PATH=/mnt/user/appdata/geointel/storage/operator-data/geel_grb_gbg_buildings.geojson \
CALIBRATION_THRESHOLDS="0.50 0.35 0.25 0.15" \
bash scripts/run_detection_calibration_sweep.sh http://192.168.10.150:1202
```
The sweep creates one real persisted workflow run per threshold, fetches the
persisted `QualityCheck`/`Metric` rows and writes a `calibration_summary.json`
with detection count, score, precision, recall, F1, mean IoU and false
positive/negative counts. It is intended to tune confidence/IoU/model choices,
not to add new inference behavior.
### Run backend
```bash
@@ -0,0 +1,29 @@
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
def test_detection_calibration_sweep_reuses_real_data_workflow_and_reports_qa_metrics() -> None:
script_path = ROOT / "scripts" / "run_detection_calibration_sweep.sh"
readiness = (ROOT / "scripts" / "run_readiness_check.sh").read_text(encoding="utf-8")
assert script_path.exists()
script = script_path.read_text(encoding="utf-8")
assert "bash -n scripts/run_detection_calibration_sweep.sh" in readiness
assert "verify_real_data_detection_qa_workflow.sh" in script
assert "CALIBRATION_THRESHOLDS" in script
assert "REAL_CONFIDENCE_THRESHOLD" in script
assert "REAL_RASTER_PATH" in script
assert "REAL_REFERENCE_VECTOR_PATH" in script
assert "/api/v1/projects/${project_id}/quality-checks" in script
assert "quality_check_id" in script
assert "false_positives" in script
assert "false_negatives" in script
assert "quality_score" in script
assert "best_by_score" in script
assert "calibration_summary.json" in script
assert "demo/workflow" not in script
assert "fixture_mode" not in script
assert "will_download_models" not in script