Add governed ALZ edition probe
This commit is contained in:
+16
-5
@@ -414,11 +414,15 @@ not marked stale merely because their source date is old. Unknown sources are
|
||||
|
||||
### GET `/api/v1/projects/{project_id}/datasets/source-catalog-probes`
|
||||
|
||||
Performs an explicit, read-only release probe for the allowlisted GRB WFS and
|
||||
most-recent orthophoto WMS. The endpoint reads bounded `GetCapabilities`
|
||||
responses, confirms the expected layers and follows only HTTPS ISO 19139
|
||||
`GetRecordById` links on `metadata.vlaanderen.be`. Query parameter
|
||||
`refresh=true` bypasses the short in-memory response cache.
|
||||
Performs an explicit, read-only release probe for the allowlisted GRB WFS,
|
||||
most-recent orthophoto WMS and ALZ agricultural-use parcel publication page.
|
||||
For OGC services, the endpoint reads bounded `GetCapabilities` responses,
|
||||
confirms the expected layers and follows only HTTPS ISO 19139 `GetRecordById`
|
||||
links on `metadata.vlaanderen.be`. For ALZ, it reads only the bounded official
|
||||
HTML release page and accepts only exact
|
||||
`www.landbouwvlaanderen.be/bestanden/gis/agpa_<year>_<date>_public.zip` link
|
||||
identities. It does not request those archives. Query parameter `refresh=true`
|
||||
bypasses the short in-memory response cache.
|
||||
|
||||
Each provider item returns service reachability, matched/missing layers, the
|
||||
official metadata identifier, title, edition, publication/metadata dates,
|
||||
@@ -427,6 +431,13 @@ local `source_version` and one comparison status: `same`, `different`,
|
||||
`available`, `degraded`, `unavailable` or `disabled`. A difference means only
|
||||
that an operator should review provenance; it is not an update instruction.
|
||||
|
||||
ALZ comparisons use only the latest definitive third snapshot, normalized as
|
||||
`<campaign>-v3`. A newer first or second snapshot is reported in the item title
|
||||
and message as provisional but cannot mark a local definitive historical
|
||||
edition as outdated. `service_type=HTML` uses `definitive_archive` and
|
||||
`current_snapshot` as evidence markers in the existing expected/matched/missing
|
||||
arrays; no parallel response shape is introduced.
|
||||
|
||||
The endpoint accepts no arbitrary URL, feature query, area or layer. It does
|
||||
not fetch vector features, raster pixels or models, create jobs/datasets, write
|
||||
to PostGIS or trigger an import. The normal `source-freshness` endpoint remains
|
||||
|
||||
@@ -173,7 +173,7 @@ gebeurd.
|
||||
| --- | --- | --- |
|
||||
| GRB gebouwen/wegen/water/percelen | operationele, expliciete plan-stage-apply refresh met onveranderlijke snapshots | alleen een nieuw officieel gedateerd cataloguseditie na operatorbevestiging ophalen |
|
||||
| Statbel bevolking | jaarlijkse, expliciete edities in één tijdreeks | een nieuwe publicatie alleen na schema-, sectorgeometrie- en totalencontrole toevoegen |
|
||||
| ALZ landbouwgebruikspercelen | definitieve jaarlijkse edities 2008-2025; metricvergelijking zonder objectlineage | eerst een stabiele machineleesbare editiebron verifiëren, daarna dezelfde begrensde operatorflow toepassen |
|
||||
| ALZ landbouwgebruikspercelen | definitieve jaarlijkse edities 2008-2025; expliciete officiële publicatieprobe; metricvergelijking zonder objectlineage | een nieuwere definitieve v3-editie eerst handmatig beoordelen en daarna via de bestaande begrensde operatorflow toevoegen |
|
||||
| orthofoto | vaste lokale opname per expliciete analysezone; catalogusprobe is alleen een signaal | vluchtjaar, productvariant en dekking vergelijken voordat nieuwe pixels worden opgehaald |
|
||||
| landgebruik, thematische rasters, DHMV en VMM-scenario's | vaste product-/scenario-edities, geen rolling snapshot | alleen een nieuwe gedocumenteerde producteditie als afzonderlijke Dataset verwerven |
|
||||
| bodemkaart en historische kaarten | historische referentie-editie | niet als verouderde actuele bron labelen; alleen vervangen bij een officiële inhoudelijke heruitgave |
|
||||
@@ -183,6 +183,13 @@ Geen van deze regels activeert een browserfetch, scheduler of automatische
|
||||
vervanging. Een bron wordt pas `refreshable` wanneer versie, dekking, schema,
|
||||
provenance en een begrensde acquisitieroute afzonderlijk verifieerbaar zijn.
|
||||
|
||||
De ALZ-publicatiepagina levert drie campagnesnapshots: v1 en v2 zijn
|
||||
voorlopig, v3 is de definitieve historische editie. GeoIntel vergelijkt daarom
|
||||
de lokale `2025-definitive` uitsluitend met de officiële `2025-v3`, terwijl de
|
||||
actuele `2026-v1` zichtbaar maar niet updategerechtigd blijft. De probe leest
|
||||
alleen de allowlisted HTML-pagina en linkidentiteiten; ZIP-archieven worden pas
|
||||
door de afzonderlijke operator opgehaald na menselijke editiebevestiging.
|
||||
|
||||
### Mol population history
|
||||
|
||||
`scripts/provision_mol_population_history.py` imports official Statbel
|
||||
|
||||
@@ -29,6 +29,7 @@ GRB_WFS_URL=
|
||||
# Explicit read-only source-edition probes (no provider import).
|
||||
SOURCE_CATALOG_PROBE_ENABLED=true
|
||||
SOURCE_CATALOG_GRB_WFS_URL=https://geo.api.vlaanderen.be/GRB/wfs
|
||||
SOURCE_CATALOG_ALZ_RELEASE_URL=https://landbouwcijfers.vlaanderen.be/open-geodata-landbouwgebruikspercelen
|
||||
SOURCE_CATALOG_PROBE_TIMEOUT_SECONDS=10
|
||||
SOURCE_CATALOG_PROBE_MAX_RESPONSE_MB=2
|
||||
SOURCE_CATALOG_PROBE_CACHE_TTL_SECONDS=900
|
||||
|
||||
+4
-3
@@ -41,7 +41,8 @@
|
||||
- [x] Add a read-only source freshness/version audit with explicit publication policies, local integrity checks, operator CLI and compact Status workspace surface.
|
||||
- [x] Add explicit bounded GRB/orthophoto catalogue release probes using official capabilities and ISO metadata; keep every acquisition separate and operator-controlled.
|
||||
- [x] Add a governed regional GRB plan -> stage -> checksum-confirmed apply workflow that preserves every previous snapshot.
|
||||
- [ ] Extend catalogue probes only to additional sources that publish a stable machine-readable edition contract; do not add background polling or infer releases from HTTP dates alone.
|
||||
- [x] Add a fail-closed ALZ publication probe that distinguishes provisional v1/v2 snapshots from the definitive v3 historical edition and never downloads an archive.
|
||||
- [ ] Extend catalogue probes only to additional sources that publish a stable official edition contract; do not add background polling or infer releases from HTTP dates alone.
|
||||
|
||||
## Governed source expansion backlog
|
||||
|
||||
@@ -683,7 +684,7 @@ This file now starts with the current implementation status. Older preparation/b
|
||||
- [x] Show precise daily edition ranges and explain registration-date nuance in
|
||||
the Evolution workspace.
|
||||
- [x] Document refresh readiness across the complete official-source portfolio.
|
||||
- [ ] Add the next governed catalog probe only after an official stable version
|
||||
contract is verified; ALZ annual releases are the first candidate.
|
||||
- [x] Verify and add the governed ALZ catalog probe against the official
|
||||
campaign-snapshot and definitive-archive publication contract.
|
||||
- [ ] Keep orthophoto refresh manual until product variant, flight year and
|
||||
complete selected-area coverage can be compared deterministically.
|
||||
|
||||
Reference in New Issue
Block a user