calibrate a confidence threshold from one inference pass

Threshold calibration ran the model over every tile once per threshold — three
GPU passes to compare 0.50, 0.25 and 0.15 on a hundred-tile raster. The answer
is already in a single run at the lowest value: detections above a higher cut
are a subset of it, and duplicate suppression walks candidates in descending
confidence, so a lower-confidence box can never displace a higher-confidence
one. The kept set above any cut is identical whichever threshold the run used,
which is what makes one pass sufficient rather than merely cheaper.

QA now takes calibration_thresholds and reads each operating point off the same
precision/recall walk it already performs, marking the F1-optimal cut. The lab
runs inference once and fills its table from the sweep.

The contract test asserted the per-threshold loop by name, pinning the waste it
was meant to describe. It now states what calibration owes an operator: a row
per requested threshold, from one run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Jens
2026-08-22 19:37:19 +02:00
co-authored by Claude Opus 5
parent 8a26007281
commit 2cd2c49389
11 changed files with 303 additions and 52 deletions
+3
View File
@@ -73,6 +73,9 @@ class DetectionQaRequest(BaseModel):
iou_threshold: float = Field(default=0.5, ge=0.0, le=1.0)
class_name: str | None = None
min_confidence: float | None = Field(default=None, ge=0.0, le=1.0)
# Confidence cuts to report alongside the run's own operating point. They
# are read off the one matching pass, so a sweep costs no extra inference.
calibration_thresholds: list[float] = Field(default_factory=list, max_length=32)
class DetectionRunResponse(BaseModel):
+2
View File
@@ -121,6 +121,8 @@ class AnalysisQaResponse(BaseModel):
coverage: dict[str, Any] | None = None
temporal_compatibility: dict[str, Any] | None = None
box_to_footprint_diagnostics: dict[str, Any] | None = None
precision_recall_curve: dict[str, Any] | None = None
calibration_sweep: list[dict[str, Any]] = Field(default_factory=list)
match_evidence: list[dict[str, Any]] = Field(default_factory=list)
false_positive_evidence: list[dict[str, Any]] = Field(default_factory=list)
false_negative_evidence: list[dict[str, Any]] = Field(default_factory=list)