diff --git a/CHANGELOG.md b/CHANGELOG.md index 76879ef6..1fd173e1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,25 @@ # Changelog +## Sprint 217 Regional DOV soil coverage (2026-07-16) + +- Added a governed regional DOV soil operator for all 28 approved Kempen + municipalities. It validates the official scope artifact, clips in Lambert + 72, retains deterministic gzip WFS responses and imports through the existing + DatasetService/vector-feature flow. +- Added NIS-suffixed persisted feature ids for source polygons split by + municipality boundaries, while retaining the original DOV source id in + provenance properties. +- Persisted one complete regional snapshot with 27,200 valid soil polygons + assembled from 62 source responses and 46,134 bounded source features. +- Kept the 1949-1971 survey period, 1:20,000 scale and historical drainage + limitation visible in map results and assistant context. +- Disabled preclipped metric shortcuts for the regional snapshot. Exact + PostGIS intersections remain mandatory because CRS round trips can create + negligible coordinate-rounding slivers at administrative boundaries. +- No migration, new endpoint, direct database write or current-soil claim was + introduced. + ## Sprint 216 Regional thematic coverage (2026-07-16) - Provisioned all five governed Departement Omgeving thematic rasters for all diff --git a/backend/README.md b/backend/README.md index 7ea8f549..bfcc865e 100644 --- a/backend/README.md +++ b/backend/README.md @@ -1328,6 +1328,20 @@ upload path: docker exec geointel python /app/scripts/provision_mol_soil_map.py ``` +Provision all approved Kempen municipalities and the complete work area after +the official geographic-scope artifacts exist: + +```bash +docker exec geointel python /app/scripts/provision_regional_soil_map.py +``` + +The regional operator retains one checksummed WFS evidence chain per +municipality, gives boundary-split source features a NIS suffix and assembles +one Dataset linked to the complete Kempen Area. EPSG:31370 clipping followed by +EPSG:4326 persistence can create submeter coordinate-rounding slivers at the +stored boundary, so regional and municipal selection metrics deliberately run +the exact PostGIS intersection instead of using a preclipped fast path. + Use `--fetch-only` to retain and validate source evidence without importing. The operator never writes directly to PostGIS. Soil drainage and related map classes represent the 1949-1971 survey and are not current observations. diff --git a/docs/CODEX_EXECUTION_LOG.md b/docs/CODEX_EXECUTION_LOG.md index fbbcceb9..d7ea7aba 100644 --- a/docs/CODEX_EXECUTION_LOG.md +++ b/docs/CODEX_EXECUTION_LOG.md @@ -1,3 +1,37 @@ +## Sprint 217 Regional DOV soil coverage (2026-07-16) + +Changed: +- Generalized the explicit DOV `bodemkaart:bodemtypes` operator from Mol to the + approved 28-member Kempen scope without changing source or persistence + semantics. +- Added independently reusable municipality partitions, deterministic gzip raw + responses, boundary/output checksums, unique NIS-suffixed persisted ids and + one regional snapshot manifest. +- Packaged and release-checked the operator in the all-in-one image. +- Kept regional and municipality metric queries on exact PostGIS intersections + instead of asserting a preclipped fast path across CRS round trips. + +Validation evidence: +- The live operator completed 28/28 partitions, 62 WFS responses, 46,134 + bounded source features and 27,200 persisted valid EPSG:4326 polygons. +- PostGIS found zero invalid/empty/wrong-SRID geometries, zero duplicate source + ids, complete municipality/NIS identity and 139,871.44 ha mapped area. +- A strict storage-boundary audit measured only 355.15 m2 of coordinate-rounding + slivers across 579 boundary features (about 0.000025%); exact selection + intersections prevent those slivers from bypassing the selected Area. +- A repeat operator run reused every verified partition and the existing + Dataset. +- Browser validation showed Bodem available for the complete Kempen and for + Arendonk, returned 139,871.44 ha and 5,514.99 ha respectively, rendered the + MapLibre layer at detail zoom and produced no browser warnings or errors. +- The pre-deployment readiness gate passed 738 backend tests, compilation, + contract checks, one Alembic head, frontend typecheck and production build. + +Next: +- Fill the remaining region-wide current-state gaps shown in the Map workspace: + VMM flood-hazard coverage and DHMV terrain. Preserve their separate modeled + scenario and elevation semantics; do not infer current water volume. + ## Sprint 216 Regional thematic coverage (2026-07-16) Changed: diff --git a/docs/DATA_SOURCES.md b/docs/DATA_SOURCES.md index 58f85328..84b18815 100644 --- a/docs/DATA_SOURCES.md +++ b/docs/DATA_SOURCES.md @@ -56,12 +56,14 @@ object counts. - Scale: 1:20,000 - Survey evidence: field data collected between 1949 and 1971 -The Mol operator follows all WFS pages, keeps checksummed raw responses and -persists soil type, series, generalized legend, texture, drainage, profile and -substrate fields. Selection output reports intersected mapped hectares and -fixed sand/anthropogenic context classes. The drainage field remains historical -baseline evidence; GeoIntel does not claim it describes current parcel -drainage or replace a site investigation. +The Mol and regional operators follow all WFS pages, keep checksummed raw +responses and persist soil type, series, generalized legend, texture, drainage, +profile and substrate fields. The regional snapshot is assembled from 28 exact +municipality partitions with unique NIS-suffixed source ids. Selection output +reports intersected mapped hectares and fixed sand/anthropogenic context +classes. The drainage field remains historical baseline evidence; GeoIntel +does not claim it describes current parcel drainage or replace a site +investigation. ## GRB — Basiskaart Vlaanderen diff --git a/docs/STORAGE_ARCHITECTURE.md b/docs/STORAGE_ARCHITECTURE.md index 643bf25e..e00cff04 100644 --- a/docs/STORAGE_ARCHITECTURE.md +++ b/docs/STORAGE_ARCHITECTURE.md @@ -228,6 +228,25 @@ Only the normalized, exactly clipped GeoJSON enters DatasetService and source page URLs/checksums, feature completeness, class-area summaries and historical limitations. Raw source responses remain operator evidence. +The regional operator keeps the same source semantics but partitions its +evidence by approved municipality before assembling one snapshot: + +```text +storage/operator-evidence/dov-soil-map/regional/ + kempen-transport-region/partitions// + raw/dov_soil_map_page_*.json.gz + dov_soil_map.geojson + manifest.json + snapshots/kempen-transport-region/ + dov_soil_map.geojson + manifest.json +``` + +The snapshot manifest binds all 28 partition checksums, the official scope +artifact, feature completeness and area summaries by generalized legend, +texture and historical drainage class. Only the combined GeoJSON enters the +canonical DatasetService/vector-feature path. + ## Model storage Model artifacts live under: diff --git a/docs/TODO.md b/docs/TODO.md index 203545a1..44cd7836 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -34,7 +34,7 @@ - [x] Add the DOV digital soil map for Mol with exact clipping, source attributes and historical survey limitations. - [x] Execute the governed thematic-raster operator for every persisted Kempen municipality after the Mol live gate passes. - [x] Persist one bounded whole-region snapshot per thematic raster so drawn selections can cross municipality boundaries without changing source semantics. -- [ ] Generalize the DOV soil-map operator to all 28 approved Kempen municipality partitions with one regional snapshot manifest. +- [x] Generalize the DOV soil-map operator to all 28 approved Kempen municipality partitions with one regional snapshot manifest. ## Governed source expansion backlog @@ -662,4 +662,4 @@ This file now starts with the current implementation status. Older preparation/b - [x] Add official space, soil, mobility, population, service, climate and economy candidates. - [x] Replace the water-first recommendation with a cross-domain implementation wave. - [ ] Implement the governed Flemish thematic-raster registry for space occupation, open space, population density, node value and service level. -- [ ] Add the digital soil map through DatasetService and VectorFeatureService. +- [x] Add the digital soil map through DatasetService and VectorFeatureService. diff --git a/scripts/README.md b/scripts/README.md index 50534b6b..9c65cc12 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -1635,6 +1635,20 @@ Load the official DOV soil map for Mol: docker exec geointel python /app/scripts/provision_mol_soil_map.py ``` +Load the same historical source for all 28 approved Kempen municipalities as +one partition-audited regional snapshot: + +```bash +docker exec geointel python /app/scripts/provision_regional_soil_map.py +``` + +The regional command verifies the checksummed municipality-scope manifest, +queries and clips every municipality separately, retains deterministic gzip +source pages and imports one complete snapshot through DatasetService. Repeat +runs reuse all verified partitions and the existing Dataset. Use +`--fetch-only` for evidence preparation without persistence and `--force` only +for an explicit source refetch. + `--fetch-only` builds the soil artifact and manifest without API import; `--force` is the only way to bypass an existing ready soil Dataset. Both operators use canonical APIs and persistent operator-evidence storage. They do