An ArcGIS layer without supportsPagination accepts resultOffset and ignores it,
answering every page with the first one. The VHA profile reader advanced its
offset by the page length and stopped at the announced count, so for a count
that is a multiple of the page size it collected N copies of page one — and its
completeness check, len(features) == candidate_count, passed. Four announced
records became four stored records, two of them duplicates, filed under an
official provenance. That is the substitution bounded acquisition exists to
prevent, arriving through the front door.
The reader now refuses a record it already collected. It fails rather than
silently dropping the duplicate: a provider that cannot page is a provider
whose count proves nothing, so a smaller-but-clean result would still be
unverifiable. Its watercourse-name loop was worse — a bare `while True` that
ended only when the provider stopped setting exceededTransferLimit, with names
deduplicated by code so a stuck provider produced no visible change while the
requests continued. It now refuses a repeated page body, and both loops have
the page budget the sibling readers already had.
Those siblings turned out to be fine. GRB and official vector already refuse a
repeated page URL, bound the page count, and deduplicate on feature identity —
but none of it had a test, so none of it was known to work. Exercised now,
including the case where distinct URLs defeat the loop check and the budget is
the only backstop. A duplicate across two genuinely different pages is kept
once rather than failing, because a cursor over a changing table produces that
legitimately.
Also: _bash_path fell back to the raw path whenever wslpath failed, except on
timeout, which propagated and reddened the suite when starting WSL took more
than ten seconds under load.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two defects of the same kind: work that is supposed to be bounded is not.
The analysis worker selected queued jobs and then set them to running in a
second statement. A restarted process overlapping the previous one, or a second
replica, could both select the same row and both start tiled GPU inference on
it — duplicate analysis runs and double the GPU load. The AOI worker beside it
already claims with FOR UPDATE SKIP LOCKED; this uses a conditional update,
which is the same guarantee in one statement. run_once now reports jobs it
actually claimed rather than jobs it looked at.
urlopen follows redirects, so although every acquisition URL is built from
settings and cannot be steered by a request payload, a misconfigured or
compromised upstream could send the runtime to the loopback interface, to
another container on the compose network, or to a cloud metadata endpoint — and
the bytes would then be persisted under an official provenance. That is exactly
the substitution the product forbids. All eight fetch sites now open through a
guard that refuses private, loopback and link-local destinations (resolving the
host first, so a DNS name cannot hide one) and refuses a redirect that leaves
the configured origin or downgrades from HTTPS.
The guard is proven by calling the services' own fetch paths, not by grepping
for the call: every existing acquisition test injects an opener, which bypasses
it by design.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Jens