feat: provision regional Kempen buildings
GeoIntel CI / docs-smoke (push) Canceled after 0s
GeoIntel CI / contract-smoke (push) Canceled after 0s

This commit is contained in:
Codex
2026-07-14 19:01:49 +02:00
parent 8586921ae6
commit 4b0c016df9
14 changed files with 1323 additions and 29 deletions
+17
View File
@@ -7884,3 +7884,20 @@ Live operational proof:
Next:
- Build bounded, idempotent regional theme ingestion in municipality-sized partitions, starting with current buildings and retaining per-member provenance before exposing any Kempen-wide metric in the explorer.
## Sprint 190 Regional Kempen GRB buildings (2026-07-14)
Implemented:
- Added `provision_regional_grb_buildings.py` for the approved 28-member scope, with complete OGC pagination, per-member safety caps, resumable checksummed artifacts and an explicit total cap.
- Added deterministic cross-boundary ownership by maximum member intersection and NIS-code tie breaking, preventing duplicate source identities without cutting the final geometry at internal municipality borders.
- Added streaming managed-artifact copy plus batch-wise partition indexing through DatasetService and VectorFeatureService. One regional Dataset remains compatible with existing viewport, selection, QA and export ownership boundaries.
- Added exact manifest/index count enforcement, duplicate source-feature rejection and rollback/storage cleanup on import failure.
- Kept fetching operator-triggered and local service import container-only. No startup fetch, browser provider fetch, direct vector-feature SQL or public API change was introduced.
Validation before deployment:
- Backend/application and operator compilation passed.
- New focused operator/storage/batch-index suite passed 5 tests.
- Dataset, vector persistence, Mol municipality and Kempen scope regression selection passed 37 tests.
Next:
- Run the complete readiness gate, deploy to Tower, execute the full 28-partition source job, verify exact PostGIS counts and perform full-region plus rectangle browser selections.
+10
View File
@@ -224,6 +224,16 @@ Provider integration is a contract layer only in Sprint 7B. Providers do not wri
GRB and OSM live imports are intentionally `not_configured` in Sprint 7B. Manual and fixture reference datasets use existing upload and fixture flows.
Large explicit operator imports may invoke DatasetService directly inside the
backend container when a single multipart upload would require loading the
entire regional artifact into memory. The regional GRB building operator still
creates a normal Dataset and DatasetVersion and delegates every queryable row
to VectorFeatureService. It copies the immutable combined artifact in a
stream, loads one retained municipality partition at a time, flushes bounded
feature batches and commits only when the indexed count matches the manifest.
It does not expose a direct SQL/provider write path and does not alter the
public provider endpoint's `not_configured` status.
## Geometry normalization
- User-drawn polygons arrive as EPSG:4326.
+18
View File
@@ -106,6 +106,24 @@ boundaries.
no interactive request or application startup may download the entire
region or silently substitute missing data.
### Regional building partition policy
The current GRB building snapshot is fetched in one resumable partition per
registered municipality. Partitions are source/provenance artifacts, not 28
independent user-facing layers. A building intersecting more than one member
boundary is owned by the member with the largest intersection area; the lower
NIS code resolves exact ties. The retained geometry is clipped to the regional
union, preserving complete in-scope coverage while guaranteeing one source ID
per regional dataset.
After all partitions pass validation, one combined immutable GeoJSON artifact
is copied into managed storage and indexed in bounded batches through the
normal DatasetService/VectorFeatureService ownership boundary. The resulting
reference dataset uses `source_name=grb`, `reference_layer_name=buildings`,
`coverage_scope=kempen-transport-region` and an exact `feature_count`
selection aggregation. Existing viewport and bbox selection APIs remain the
only interactive delivery path.
## User-uploaded raster strategy
V1 must support controlled local datasets because public raster access and model compatibility can be difficult.
+15
View File
@@ -52,6 +52,21 @@ De scope-operator haalt geen thematische gegevens op. GRB, bevolking en
landgebruik volgen als afzonderlijke begrensde imports; ontbrekende regionale
thema's blijven zichtbaar onbeschikbaar.
De actuele regionale gebouwlaag wordt afzonderlijk opgehaald door
`scripts/provision_regional_grb_buildings.py`. De operator bevraagt
`GRB/GBG` per officiële gemeentegrens, volgt alle pagina's en weigert elk
veiligheidslimiet als truncatie te verbergen. Grensoverschrijdende objecten
worden aan één gemeentepartitie toegewezen op basis van de grootste overlap;
de uiteindelijke geometrie wordt uitsluitend tegen de volledige regiogrens
gesneden. Daardoor bevat het regionale dataset elke GRB-bronidentiteit exact
één keer.
De 28 bronpartities, paginabronnen, aantallen en checksums blijven als
operatorbewijs bewaard. GeoIntel indexeert ze batchgewijs als één regionaal
referentiedataset via DatasetService en VectorFeatureService. De publieke GRB
provider blijft `not_configured`: dit is een bewuste operatorrun en geen live
download vanuit een browseractie of applicatiestart.
### Mol population history
`scripts/provision_mol_population_history.py` imports official Statbel