Detection and segmentation run listings returned every run a project had ever
produced. Runs accumulate with every analysis while the panel only ever draws
the recent ones, so the response grew without bound for no benefit.
Both take limit and offset now and report total, limit, offset and truncated,
matching the result listings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Detection QA reports a precision/recall curve, average precision and a
calibration sweep; segmentation QA reported a single operating point. Both rank
their outputs by confidence, so the same view applies, and the asymmetry meant
the two panels answered different questions about comparable runs — an
inconsistency introduced when detection gained the curve.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The workbench ranks model variants by a stored F1, each measured at that
variant's own confidence threshold. A conservatively calibrated detector then
looks worse than a liberal one without detecting anything differently: the
number says as much about the threshold as about the model. POST
/detection/runs/compare ranks on average precision instead, which describes the
whole ranking a model produced, and keeps each run's own-threshold F1 visible
next to it so the difference between the two readings is auditable.
Comparability comes before the ranking. Runs over different source rasters,
scored against different references, without a proven inference footprint, or
covering a different evaluated population are not alternatives to one another,
and no metric makes them so. The report names which of those applies and still
returns the numbers — they are simply not a ranking.
Each run is scored through the same QA path the workbench uses, so a comparison
and the persisted quality checks cannot drift apart.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Threshold calibration ran the model over every tile once per threshold — three
GPU passes to compare 0.50, 0.25 and 0.15 on a hundred-tile raster. The answer
is already in a single run at the lowest value: detections above a higher cut
are a subset of it, and duplicate suppression walks candidates in descending
confidence, so a lower-confidence box can never displace a higher-confidence
one. The kept set above any cut is identical whichever threshold the run used,
which is what makes one pass sufficient rather than merely cheaper.
QA now takes calibration_thresholds and reads each operating point off the same
precision/recall walk it already performs, marking the F1-optimal cut. The lab
runs inference once and fills its table from the sweep.
The contract test asserted the per-threshold loop by name, pinning the waste it
was meant to describe. It now states what calibration owes an operator: a row
per requested threshold, from one run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
/detection/runs/{id}/detections and its GeoJSON sibling returned every
persisted detection, as did the segmentation equivalents. A regional run holds
tens of thousands, and these are the endpoints the results table and the map
overlay call after every run.
They now take limit and offset, default to 2.000, and report total, limit,
offset and truncated so the complete population stays visible while what is
transferred does not. The GeoJSON responses carry the same window in a
geointel_result_window foreign member.
Rows are ordered by confidence, so a capped overlay draws the strongest
detections rather than an arbitrary slice, and the lab says how many of how
many are being shown rather than silently presenting a page as the whole run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
evidence_geojson emitted one feature per false positive, one per false negative
and two per match, with no limit. A regional check of 40k detections against 45k
reference footprints produced well over a hundred thousand features in a single
response, plus one warning string per unresolvable identifier. The endpoint the
entire review workflow depends on therefore failed exactly where review matters
most.
What to draw is now decided before any geometry is fetched, so the query work is
proportional to the result rather than to the size of the check — previously
130k geometries were resolved through an IN clause holding every identifier in
the check, to then discard most of them.
The budget is split between misses and false positives in proportion to their
populations with at least one of each, rather than by strict priority, which
would mean a check with 50.000 misses and three false positives never showed
one. Confirmations fill what remains, and a match is kept or dropped as a pair
because half a match is not reviewable evidence.
limit_evidence and evidence_role_counts are removed: plan_evidence supersedes
them, and helpers kept alive only by their own tests read like a contract.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Change detection was the one analysis that ignored the selection entirely. It
compared two datasets in full, loaded every feature of both into Python with no
spatial predicate, and — with include_unchanged defaulting to true — returned a
FeatureCollection holding both datasets. For a regional building layer that is
the wrong answer to "what changed here" and a response no browser should be
asked to hold.
It now accepts bbox and area_id, resolved the way every other analysis resolves
them, and loads through an indexed ST_Intersects predicate.
Features are deliberately not clipped to the selection. A change class
describes a whole object: comparing a clipped earlier footprint against an
unclipped later one would report the selection edge itself as a change. Objects
the edge crosses are compared in full and counted in a warning.
The returned geometry is capped by preview_limit, spending that budget on
modified, added and removed before unchanged, while every count still describes
the whole selection.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Tile handling produced results that were wrong before any model quality
question arose:
- orthophoto tiles reached the model through PIL convert("RGB"), which
truncates the high byte of a 16-bit product and treats a 4-band RGB+NIR
tile's infrared channel as colour. Tiles are now read with rasterio, the
visible bands are chosen explicitly, and values are percentile-stretched
across all three bands together so hue is preserved;
- an object wider than the tile overlap was truncated by both tiles into two
boxes that barely intersect, so IoU suppression kept both: two false
positives and one missed footprint per seam building. Suppression now also
compares overlap against the smaller box, and boxes cut by an interior tile
edge are dropped in favour of the neighbouring tile's complete view;
- georeferencing fell back to an assumed EPSG:4326 when a manifest carried no
CRS, producing geometry that renders plausibly in the wrong place. QA
already refused such a tile; inference now fails closed too.
Segmentation QA scored candidates against every reference feature in the
dataset, so every building outside the inferred tiles counted as a false
negative. It now applies the same persisted tile coverage that detection QA
has always used, including the indexed ST_Intersects prefilter.
Duplicate suppression uses an STRtree instead of the O(n^2) scan, tiles are
predicted in batches of YOLO_BATCH_SIZE (a setting that existed but was never
read), and detection/segmentation runs can be queued through /run-async for a
polling background worker rather than holding an HTTP worker thread for
minutes of GPU work.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Jens