Threshold calibration ran the model over every tile once per threshold — three
GPU passes to compare 0.50, 0.25 and 0.15 on a hundred-tile raster. The answer
is already in a single run at the lowest value: detections above a higher cut
are a subset of it, and duplicate suppression walks candidates in descending
confidence, so a lower-confidence box can never displace a higher-confidence
one. The kept set above any cut is identical whichever threshold the run used,
which is what makes one pass sufficient rather than merely cheaper.
QA now takes calibration_thresholds and reads each operating point off the same
precision/recall walk it already performs, marking the F1-optimal cut. The lab
runs inference once and fills its table from the sweep.
The contract test asserted the per-threshold loop by name, pinning the waste it
was meant to describe. It now states what calibration owes an operator: a row
per requested threshold, from one run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The greedy IoU matcher gave a reference to whichever candidate was offered
first. Row order decided that, and every detection in a run shares one
transaction timestamp, so ordering by created_at left the assignment
undefined: the same QA run over the same data produced different mean IoU,
and the geometry shown to a reviewer as a false positive could be the better
of two detections. Candidates are now ranked by confidence with feature
identity as tiebreaker, which is also the COCO/PASCAL rule.
A single precision/recall/F1 triple describes one operating point, so two
models cannot be compared from it: a conservatively calibrated model looks
worse at a low confidence cut and better at a high one without detecting
anything differently. DetectionMetricsService adds the full curve, average
precision and the threshold where F1 actually peaks.
Also:
- report the population the metrics were computed over, so
matches + false_positives equals candidate_feature_count even under an
area filter; raw dataset totals move to the _raw fields;
- state whether candidates are axis-aligned boxes or footprint polygons.
A box can never reach IoU 1 against a rotated building, so the strict
score has a ceiling that has nothing to do with detection quality.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
