calibrate a confidence threshold from one inference pass

Threshold calibration ran the model over every tile once per threshold — three
GPU passes to compare 0.50, 0.25 and 0.15 on a hundred-tile raster. The answer
is already in a single run at the lowest value: detections above a higher cut
are a subset of it, and duplicate suppression walks candidates in descending
confidence, so a lower-confidence box can never displace a higher-confidence
one. The kept set above any cut is identical whichever threshold the run used,
which is what makes one pass sufficient rather than merely cheaper.

QA now takes calibration_thresholds and reads each operating point off the same
precision/recall walk it already performs, marking the F1-optimal cut. The lab
runs inference once and fills its table from the sweep.

The contract test asserted the per-threshold loop by name, pinning the waste it
was meant to describe. It now states what calibration owes an operator: a row
per requested threshold, from one run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Jens
2026-08-22 19:37:19 +02:00
co-authored by Claude Opus 5
parent 8a26007281
commit 2cd2c49389
11 changed files with 303 additions and 52 deletions
+12
View File
@@ -377,6 +377,7 @@ class DetectionService:
iou_threshold: float = 0.5,
class_name: str | None = None,
min_confidence: float | None = None,
calibration_thresholds: list[float] | None = None,
) -> dict[str, Any]:
run = db.get(AnalysisRun, analysis_run_id)
if not run or run.analysis_type != "detection":
@@ -563,6 +564,15 @@ class DetectionService:
reference_geometries,
iou_threshold=iou_threshold,
)
# Every requested confidence cut, answered from that one matching pass.
# Re-running inference per threshold spends N GPU passes to reproduce
# numbers already present here: suppression walks candidates in
# descending confidence, so the kept set above a cut does not depend on
# the threshold the run itself used.
calibration_sweep = DetectionMetricsService.calibration_sweep(
precision_recall_curve,
thresholds=list(calibration_thresholds or []),
)
mean_iou = None if not evidence.match_iou_values else sum(evidence.match_iou_values) / len(evidence.match_iou_values)
precision = evidence.matches / (evidence.matches + evidence.false_positives) if evidence.matches + evidence.false_positives > 0 else None
recall = evidence.matches / (evidence.matches + evidence.false_negatives) if evidence.matches + evidence.false_negatives > 0 else None
@@ -598,6 +608,7 @@ class DetectionService:
"temporal_compatibility": temporal_compatibility,
"box_to_footprint_diagnostics": box_to_footprint_diagnostics,
"precision_recall_curve": precision_recall_curve,
"calibration_sweep": calibration_sweep,
"match_evidence": evidence.match_evidence,
"false_positive_evidence": evidence.false_positive_evidence,
"false_negative_evidence": evidence.false_negative_evidence,
@@ -647,6 +658,7 @@ class DetectionService:
"temporal_compatibility": temporal_compatibility,
"box_to_footprint_diagnostics": box_to_footprint_diagnostics,
"precision_recall_curve": precision_recall_curve,
"calibration_sweep": calibration_sweep,
"match_evidence": evidence.match_evidence,
"false_positive_evidence": evidence.false_positive_evidence,
"false_negative_evidence": evidence.false_negative_evidence,