Harden Statbel population import compatibility
GeoIntel CI / docs-smoke (push) Canceled after 0s
GeoIntel CI / contract-smoke (push) Canceled after 0s

This commit is contained in:
Codex
2026-07-16 22:54:21 +02:00
parent 5942712c7b
commit 4f4d467d11
11 changed files with 1533 additions and 18 deletions
+30
View File
@@ -1043,6 +1043,36 @@ third imports the Departement Omgeving 10 m forest class for 2013, 2016, 2019,
API/DatasetService flow and retain fetched artifacts in persistent operator
storage. They never run on app startup.
Every newly fetched or `--force` rebuilt Statbel population edition now passes
`statbel_population_preflight.py` before a derived GeoJSON can reach the
upload API. The operator retains both official ZIPs, writes an atomic
preflight manifest and verifies the source and derived SHA-256 values again at
upload time. A passed preflight does not replace an existing Dataset.
The preflight can also be run without downloads or database mutation against
already staged official artifacts:
```bash
docker exec geointel python /app/scripts/statbel_population_preflight.py \
--year 2025 \
--layout new \
--scope kempen-transport-region \
--population-archive /tmp/OPENDATA_SECTOREN_2025_NEW.zip \
--population-url https://statbel.fgov.be/sites/default/files/files/opendata/bevolking/sectoren/OPENDATA_SECTOREN_2025_NEW.zip \
--geometry-archive /tmp/sh_statbel_statistical_sectors_31370_20250101.geojson.zip \
--geometry-url https://statbel.fgov.be/sites/default/files/files/opendata/Statistische%20sectoren/sh_statbel_statistical_sectors_31370_20250101.geojson.zip \
--baseline-snapshot /app/storage/operator-data/regional-timeseries/kempen-transport-region/population/kempen_transport_region_statbel_population_2024.geojson \
--output /app/storage/operator-evidence/statbel-population/2025-kempen.preflight.json
```
The command exits non-zero and emits a stable `error_code` when source
identity, archive safety, schema, CRS, geometry, join, total reconciliation,
scope coverage or the default 5% annualized population-change review limit
fails. `ZZZZ` rows are reconciled as official unlocated population but remain
excluded from spatial metrics. The 2025 REDEGEO contract deliberately compares
explicit municipality fields; it does not assume that `CD_SECTOR` still starts
with the current `CD_REFNIS` after municipal mergers.
Historical land-use work can be bounded explicitly:
```bash