Two problems of the same shape: information about *why* something failed being
replaced by something vaguer.
get_dataset_geojson wrapped the JSON parse, the metadata read, the CRS
resolution and the canonicalisation in one try and reported all of it as
"Stored dataset is not valid JSON" with a 500. An operator whose dataset had an
unusable CRS was sent to inspect a file that parses perfectly well, and the
canonicaliser's own AppError — with its code and its status — never reached
them. Only the parse is now inside that handler; everything after it keeps the
error it raised, and a genuine bug becomes a distinct 500 rather than a
mislabelled client error. A guard finds the same shape elsewhere: catching
Exception around a call into another component and relabelling what it
reported. Wrapping one's own private helper stays legitimate and the guard
says so.
The redirect policy was split without anyone saying so. Two acquisition
services rejected every redirect through a hand-rolled opener, while eight
allowed a same-origin one through the shared guard — and only the latter
checked where the response came from. Both live in the guard now, and the
strict path uses the rejecting handler rather than the guard's after-the-fact
check: objecting to response.url means urllib already opened the connection and
read the body, which for a metadata endpoint is the whole attack. That was a
weakening I introduced in this same commit's first draft.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two defects of the same kind: work that is supposed to be bounded is not.
The analysis worker selected queued jobs and then set them to running in a
second statement. A restarted process overlapping the previous one, or a second
replica, could both select the same row and both start tiled GPU inference on
it — duplicate analysis runs and double the GPU load. The AOI worker beside it
already claims with FOR UPDATE SKIP LOCKED; this uses a conditional update,
which is the same guarantee in one statement. run_once now reports jobs it
actually claimed rather than jobs it looked at.
urlopen follows redirects, so although every acquisition URL is built from
settings and cannot be steered by a request payload, a misconfigured or
compromised upstream could send the runtime to the loopback interface, to
another container on the compose network, or to a cloud metadata endpoint — and
the bytes would then be persisted under an official provenance. That is exactly
the substitution the product forbids. All eight fetch sites now open through a
guard that refuses private, loopback and link-local destinations (resolving the
host first, so a DNS name cannot hide one) and refuses a redirect that leaves
the configured origin or downgrades from HTTPS.
The guard is proven by calling the services' own fetch paths, not by grepping
for the call: every existing acquisition test injects an opener, which bypasses
it by design.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Jens