calibrate a confidence threshold from one inference pass
Threshold calibration ran the model over every tile once per threshold — three GPU passes to compare 0.50, 0.25 and 0.15 on a hundred-tile raster. The answer is already in a single run at the lowest value: detections above a higher cut are a subset of it, and duplicate suppression walks candidates in descending confidence, so a lower-confidence box can never displace a higher-confidence one. The kept set above any cut is identical whichever threshold the run used, which is what makes one pass sufficient rather than merely cheaper. QA now takes calibration_thresholds and reads each operating point off the same precision/recall walk it already performs, marking the F1-optimal cut. The lab runs inference once and fills its table from the sweep. The contract test asserted the per-threshold loop by name, pinning the waste it was meant to describe. It now states what calibration owes an operator: a row per requested threshold, from one run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -377,6 +377,7 @@ class DetectionService:
|
||||
iou_threshold: float = 0.5,
|
||||
class_name: str | None = None,
|
||||
min_confidence: float | None = None,
|
||||
calibration_thresholds: list[float] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
run = db.get(AnalysisRun, analysis_run_id)
|
||||
if not run or run.analysis_type != "detection":
|
||||
@@ -563,6 +564,15 @@ class DetectionService:
|
||||
reference_geometries,
|
||||
iou_threshold=iou_threshold,
|
||||
)
|
||||
# Every requested confidence cut, answered from that one matching pass.
|
||||
# Re-running inference per threshold spends N GPU passes to reproduce
|
||||
# numbers already present here: suppression walks candidates in
|
||||
# descending confidence, so the kept set above a cut does not depend on
|
||||
# the threshold the run itself used.
|
||||
calibration_sweep = DetectionMetricsService.calibration_sweep(
|
||||
precision_recall_curve,
|
||||
thresholds=list(calibration_thresholds or []),
|
||||
)
|
||||
mean_iou = None if not evidence.match_iou_values else sum(evidence.match_iou_values) / len(evidence.match_iou_values)
|
||||
precision = evidence.matches / (evidence.matches + evidence.false_positives) if evidence.matches + evidence.false_positives > 0 else None
|
||||
recall = evidence.matches / (evidence.matches + evidence.false_negatives) if evidence.matches + evidence.false_negatives > 0 else None
|
||||
@@ -598,6 +608,7 @@ class DetectionService:
|
||||
"temporal_compatibility": temporal_compatibility,
|
||||
"box_to_footprint_diagnostics": box_to_footprint_diagnostics,
|
||||
"precision_recall_curve": precision_recall_curve,
|
||||
"calibration_sweep": calibration_sweep,
|
||||
"match_evidence": evidence.match_evidence,
|
||||
"false_positive_evidence": evidence.false_positive_evidence,
|
||||
"false_negative_evidence": evidence.false_negative_evidence,
|
||||
@@ -647,6 +658,7 @@ class DetectionService:
|
||||
"temporal_compatibility": temporal_compatibility,
|
||||
"box_to_footprint_diagnostics": box_to_footprint_diagnostics,
|
||||
"precision_recall_curve": precision_recall_curve,
|
||||
"calibration_sweep": calibration_sweep,
|
||||
"match_evidence": evidence.match_evidence,
|
||||
"false_positive_evidence": evidence.false_positive_evidence,
|
||||
"false_negative_evidence": evidence.false_negative_evidence,
|
||||
|
||||
Reference in New Issue
Block a user