claim analysis jobs atomically and keep acquisition on its official host

Two defects of the same kind: work that is supposed to be bounded is not.

The analysis worker selected queued jobs and then set them to running in a
second statement. A restarted process overlapping the previous one, or a second
replica, could both select the same row and both start tiled GPU inference on
it — duplicate analysis runs and double the GPU load. The AOI worker beside it
already claims with FOR UPDATE SKIP LOCKED; this uses a conditional update,
which is the same guarantee in one statement. run_once now reports jobs it
actually claimed rather than jobs it looked at.

urlopen follows redirects, so although every acquisition URL is built from
settings and cannot be steered by a request payload, a misconfigured or
compromised upstream could send the runtime to the loopback interface, to
another container on the compose network, or to a cloud metadata endpoint — and
the bytes would then be persisted under an official provenance. That is exactly
the substitution the product forbids. All eight fetch sites now open through a
guard that refuses private, loopback and link-local destinations (resolving the
host first, so a DNS name cannot hide one) and refuses a redirect that leaves
the configured origin or downgrades from HTTPS.

The guard is proven by calling the services' own fetch paths, not by grepping
for the call: every existing acquisition test injects an opener, which bypasses
it by design.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Jens
2026-08-22 15:59:01 +02:00
co-authored by Claude Opus 5
parent 5b3839dc89
commit 16dedeb670
14 changed files with 493 additions and 16 deletions
@@ -15,6 +15,7 @@ from shapely.geometry import Point, box, mapping
from app.core.config import Settings, get_settings
from app.core.errors import AppError
from app.services.outbound_request_guard import guarded_opener
from app.models import Area, Dataset, DatasetVersion, Project
from app.schemas.bathymetry import (
BathymetryPartitionFinalizeRequest,
@@ -224,7 +225,7 @@ class BathymetryProfileAcquisitionService:
},
)
try:
with (opener or urlopen)(request, timeout=settings.bathymetry_profiles_timeout_seconds) as response:
with (opener or guarded_opener(url))(request, timeout=settings.bathymetry_profiles_timeout_seconds) as response:
limit = settings.bathymetry_profiles_max_response_mb * 1024 * 1024
content = response.read(limit + 1)
except HTTPError as exc: