scripts/evaluate_belgium_building_candidate.py freezes its post-processing
before the protected test — NMS IoU and a containment threshold selected during
calibration, defaulting to 1.0. The runtime applied a hardcoded 0.85, so a
promoted candidate was served under suppression its evaluation never measured
and dropped detections the gate had counted. Neither report showed the
difference. That constant was mine, added without noticing the evaluation
pipeline already had a tuned value for the same concept.
Containment is now configuration, recorded on every run beside the duplicate
IoU threshold, so an operator can serve a candidate at the value it was gated
at and afterwards see which value produced a given score.
Two runs that post-processed differently produced different candidate sets from
the same model output, so the comparison endpoint refuses to rank them. Runs
recorded before those values were persisted carry none, and absence is not
treated as a difference.
The remaining gap is deliberate and documented rather than closed: the gate
scores the model on its protected test set, the workbench scores the whole
pipeline including coverage clipping and the tile-edge filter. A promoted
candidate will not reproduce its gate F1 exactly, and pretending otherwise
would be the worse answer.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two defects of the same kind: work that is supposed to be bounded is not.
The analysis worker selected queued jobs and then set them to running in a
second statement. A restarted process overlapping the previous one, or a second
replica, could both select the same row and both start tiled GPU inference on
it — duplicate analysis runs and double the GPU load. The AOI worker beside it
already claims with FOR UPDATE SKIP LOCKED; this uses a conditional update,
which is the same guarantee in one statement. run_once now reports jobs it
actually claimed rather than jobs it looked at.
urlopen follows redirects, so although every acquisition URL is built from
settings and cannot be steered by a request payload, a misconfigured or
compromised upstream could send the runtime to the loopback interface, to
another container on the compose network, or to a cloud metadata endpoint — and
the bytes would then be persisted under an official provenance. That is exactly
the substitution the product forbids. All eight fetch sites now open through a
guard that refuses private, loopback and link-local destinations (resolving the
host first, so a DNS name cannot hide one) and refuses a redirect that leaves
the configured origin or downgrades from HTTPS.
The guard is proven by calling the services' own fetch paths, not by grepping
for the call: every existing acquisition test injects an opener, which bypasses
it by design.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The README's architecture diagram and stack list promised Redis + RQ. There is
no such thing in this codebase: redis_url is a settings field nothing reads,
rq is not a dependency, and the only worker is an asyncio poller over a
Postgres job table. Describe what exists.
API_CONTRACTS and KNOWN_LIMITATIONS gain the rules a reader needs to interpret
a result: deterministic confidence-ranked matching, the precision/recall curve,
the three flood-hazard cell populations, the sub-cell selection fallback, the
whole-feature object count next to clipped area metrics, and the required
manifest CRS. The new limitations are stated as limits rather than buried:
raster analysis has no sub-cell precision, and the object count is not
fractional.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
