feat: provision regional Kempen buildings
This commit is contained in:
@@ -7884,3 +7884,20 @@ Live operational proof:
|
||||
|
||||
Next:
|
||||
- Build bounded, idempotent regional theme ingestion in municipality-sized partitions, starting with current buildings and retaining per-member provenance before exposing any Kempen-wide metric in the explorer.
|
||||
|
||||
## Sprint 190 Regional Kempen GRB buildings (2026-07-14)
|
||||
|
||||
Implemented:
|
||||
- Added `provision_regional_grb_buildings.py` for the approved 28-member scope, with complete OGC pagination, per-member safety caps, resumable checksummed artifacts and an explicit total cap.
|
||||
- Added deterministic cross-boundary ownership by maximum member intersection and NIS-code tie breaking, preventing duplicate source identities without cutting the final geometry at internal municipality borders.
|
||||
- Added streaming managed-artifact copy plus batch-wise partition indexing through DatasetService and VectorFeatureService. One regional Dataset remains compatible with existing viewport, selection, QA and export ownership boundaries.
|
||||
- Added exact manifest/index count enforcement, duplicate source-feature rejection and rollback/storage cleanup on import failure.
|
||||
- Kept fetching operator-triggered and local service import container-only. No startup fetch, browser provider fetch, direct vector-feature SQL or public API change was introduced.
|
||||
|
||||
Validation before deployment:
|
||||
- Backend/application and operator compilation passed.
|
||||
- New focused operator/storage/batch-index suite passed 5 tests.
|
||||
- Dataset, vector persistence, Mol municipality and Kempen scope regression selection passed 37 tests.
|
||||
|
||||
Next:
|
||||
- Run the complete readiness gate, deploy to Tower, execute the full 28-partition source job, verify exact PostGIS counts and perform full-region plus rectangle browser selections.
|
||||
|
||||
@@ -224,6 +224,16 @@ Provider integration is a contract layer only in Sprint 7B. Providers do not wri
|
||||
|
||||
GRB and OSM live imports are intentionally `not_configured` in Sprint 7B. Manual and fixture reference datasets use existing upload and fixture flows.
|
||||
|
||||
Large explicit operator imports may invoke DatasetService directly inside the
|
||||
backend container when a single multipart upload would require loading the
|
||||
entire regional artifact into memory. The regional GRB building operator still
|
||||
creates a normal Dataset and DatasetVersion and delegates every queryable row
|
||||
to VectorFeatureService. It copies the immutable combined artifact in a
|
||||
stream, loads one retained municipality partition at a time, flushes bounded
|
||||
feature batches and commits only when the indexed count matches the manifest.
|
||||
It does not expose a direct SQL/provider write path and does not alter the
|
||||
public provider endpoint's `not_configured` status.
|
||||
|
||||
## Geometry normalization
|
||||
|
||||
- User-drawn polygons arrive as EPSG:4326.
|
||||
|
||||
@@ -106,6 +106,24 @@ boundaries.
|
||||
no interactive request or application startup may download the entire
|
||||
region or silently substitute missing data.
|
||||
|
||||
### Regional building partition policy
|
||||
|
||||
The current GRB building snapshot is fetched in one resumable partition per
|
||||
registered municipality. Partitions are source/provenance artifacts, not 28
|
||||
independent user-facing layers. A building intersecting more than one member
|
||||
boundary is owned by the member with the largest intersection area; the lower
|
||||
NIS code resolves exact ties. The retained geometry is clipped to the regional
|
||||
union, preserving complete in-scope coverage while guaranteeing one source ID
|
||||
per regional dataset.
|
||||
|
||||
After all partitions pass validation, one combined immutable GeoJSON artifact
|
||||
is copied into managed storage and indexed in bounded batches through the
|
||||
normal DatasetService/VectorFeatureService ownership boundary. The resulting
|
||||
reference dataset uses `source_name=grb`, `reference_layer_name=buildings`,
|
||||
`coverage_scope=kempen-transport-region` and an exact `feature_count`
|
||||
selection aggregation. Existing viewport and bbox selection APIs remain the
|
||||
only interactive delivery path.
|
||||
|
||||
## User-uploaded raster strategy
|
||||
V1 must support controlled local datasets because public raster access and model compatibility can be difficult.
|
||||
|
||||
|
||||
@@ -52,6 +52,21 @@ De scope-operator haalt geen thematische gegevens op. GRB, bevolking en
|
||||
landgebruik volgen als afzonderlijke begrensde imports; ontbrekende regionale
|
||||
thema's blijven zichtbaar onbeschikbaar.
|
||||
|
||||
De actuele regionale gebouwlaag wordt afzonderlijk opgehaald door
|
||||
`scripts/provision_regional_grb_buildings.py`. De operator bevraagt
|
||||
`GRB/GBG` per officiële gemeentegrens, volgt alle pagina's en weigert elk
|
||||
veiligheidslimiet als truncatie te verbergen. Grensoverschrijdende objecten
|
||||
worden aan één gemeentepartitie toegewezen op basis van de grootste overlap;
|
||||
de uiteindelijke geometrie wordt uitsluitend tegen de volledige regiogrens
|
||||
gesneden. Daardoor bevat het regionale dataset elke GRB-bronidentiteit exact
|
||||
één keer.
|
||||
|
||||
De 28 bronpartities, paginabronnen, aantallen en checksums blijven als
|
||||
operatorbewijs bewaard. GeoIntel indexeert ze batchgewijs als één regionaal
|
||||
referentiedataset via DatasetService en VectorFeatureService. De publieke GRB
|
||||
provider blijft `not_configured`: dit is een bewuste operatorrun en geen live
|
||||
download vanuit een browseractie of applicatiestart.
|
||||
|
||||
### Mol population history
|
||||
|
||||
`scripts/provision_mol_population_history.py` imports official Statbel
|
||||
|
||||
Reference in New Issue
Block a user