feat: add temporal Mol explorer
GeoIntel CI / docs-smoke (push) Canceled after 0s
GeoIntel CI / contract-smoke (push) Canceled after 0s

This commit is contained in:
Codex
2026-07-14 15:19:42 +02:00
parent 0ec1ab4970
commit c0943fe7d4
39 changed files with 3065 additions and 163 deletions
+50
View File
@@ -1393,3 +1393,53 @@ GET /api/v1/exports/{export_id}/download
```
PDF/report-designer functionality can be added after core GeoAI workflows work.
## Temporal datasets and area evolution
Temporal metadata describes the source observation, not API run history. A
snapshot is a normal persisted dataset grouped by `temporal_series_key` and
ordered by `observed_at`. Optional validity uses `valid_from` and `valid_to`;
`temporal_granularity` is `snapshot`, `day`, `month`, `year` or `period`.
Dataset upload accepts those temporal fields plus `source_version`. When a
dataset declares `source_metadata.selection_aggregation`, the vector bbox
selection response also contains a `summary` with metric label/value/unit,
aggregation method, feature count, estimate status and an optional warning.
Supported PostGIS aggregations are feature count, intersection area,
intersection length, numeric sum and area-weighted numeric sum. Area and length
are measured after transformation to EPSG:31370.
### PATCH `/api/v1/projects/{project_id}/datasets/{dataset_id}/temporal`
Updates the temporal provenance of an existing dataset. Series key and
observation date are required together. It does not alter features or
manufacture a historical observation.
### GET `/api/v1/projects/{project_id}/datasets/{dataset_id}/versions`
Lists immutable storage/provenance versions. Uploads and derived datasets
create version 1 in the same persistence transaction.
### GET `/api/v1/projects/{project_id}/temporal/series`
Returns dated project series in the canonical envelope. Each item contains its
source/layer identity, first and last observations and ordered datasets.
### POST `/api/v1/projects/{project_id}/temporal/compare`
```json
{
"earlier_dataset_id": "uuid",
"later_dataset_id": "uuid",
"bbox": {"west": 5.0, "south": 51.0, "east": 5.2, "north": 51.2},
"preview_limit": 500
}
```
Both datasets must belong to the project and the same temporal series, with
the earlier observation preceding the later one. The response contains source
snapshot references, selection bbox, earlier/later metric values,
absolute/percentage change, estimate status, warnings and GeoJSON evidence.
Added/removed/modified object changes are calculated only when source
provenance declares stable feature identities; otherwise
`object_changes.available=false` and no object history is inferred.
+26
View File
@@ -7768,3 +7768,29 @@ Limitations:
Next:
- Define authoritative population and land-cover source adapters, then reuse the proven municipality provisioner and bbox analysis flow for the complete Kempen.
## Sprint 187 Temporal Mol explorer (2026-07-14)
Implemented:
- Added observation/validity/source-version fields to datasets and immutable provenance fields to dataset versions, with one Alembic head and indexed temporal/source identity lookups.
- Persisted dataset version 1 atomically for uploads, demo fixtures and derived vector/raster operations.
- Added source-governed PostGIS selection summaries for object count, area, length, numeric sum and area-weighted sum.
- Added project temporal-series discovery and same-series bbox comparison with honest metric deltas and stable-identity-only object changes.
- Added the map-first `Laatste toestand` / `Evolutie` workflow with period selection, automatic rectangle analysis and change overlays.
- Added explicit, idempotent Statbel population (2021-2025) and Digitaal Vlaanderen historical land-use (1778/1873/1969) provisioners for Mol.
- Kept all source fetching operator-triggered; application startup and user queries never fabricate or silently download source data.
Methodology:
- Population totals are source-published per statistical sector. Intersections with partial sectors are labelled area-weighted estimates.
- Historical land-use classes are clipped from official editions and measured in EPSG:31370. They do not claim stable cadastral object identity.
- Every snapshot carries its source URL, observation date, source version, checksum and processing limitations.
Validation before live deployment:
- Script compilation passed.
- New temporal/API/static regression suite passed 6 tests, including append-only/idempotent temporal provenance updates.
- Raster and temporal focused suite passed 27 tests after extending existing assertions to require dataset-version persistence.
- Frontend TypeScript typecheck and production build passed.
- Offline Alembic SQL generation passed with head `202607140001`.
Next:
- Run the complete readiness gate, deploy to Tower/PostGIS, provision the official snapshots and verify current/evolution selection end to end in the internal browser.
+19
View File
@@ -236,3 +236,22 @@ GRB and OSM live imports are intentionally `not_configured` in Sprint 7B. Manual
- Multi-tenant row-level security.
- User accounts.
- Full model registry tables.
## Temporal dataset foundation
Historical observations remain normal `datasets` and `vector_features`; there
is no parallel temporal feature store. Snapshots are grouped by
`datasets.temporal_series_key` and carry `observed_at`, `valid_from`,
`valid_to`, `temporal_granularity` and `source_version`. Observation time is
kept separate from ingestion time (`imported_at`).
`dataset_versions` records immutable storage provenance for every upload and
derived output: dataset-local `version`, storage path, source version,
observation/validity dates, checksum and source/provenance JSON. The
`(dataset_id, version)` pair is unique. Temporal series lookup is indexed by
`(project_id, temporal_series_key, observed_at)` and source-feature lookup by
`(dataset_id, source_feature_id)`.
Time-series comparison is read-only and aggregates persisted geometry inside a
requested bbox. Object-level added/removed/modified evidence is only valid for
sources that explicitly declare stable source feature identifiers.
+17
View File
@@ -138,3 +138,20 @@ V1 dataset strategy is complete when:
- an area can request/cache a reference building layer;
- detection outputs can be compared with that reference layer;
- exports include source metadata.
## Temporal snapshots and evolution
- A historical observation is one persisted dataset snapshot. Existing
datasets are not overwritten and features are not hidden in job JSON.
- Related observations share a stable `temporal_series_key`; `observed_at`
records when the source describes reality, while `imported_at` records
ingestion time.
- The latest-state map uses the latest available observation but must not call
an old source edition current reality.
- Evolution metrics use the same selection geometry, aggregation and units for
both snapshots.
- Partial-sector population is an area-weighted estimate. GeoIntel must not
imply address-level distribution when only sector totals are available.
- Historical cartographic classes can change meaning between editions. Source
classes and processing notes remain provenance, and object changes require
explicit stable source identity.
+26
View File
@@ -29,6 +29,32 @@ start geen verborgen providerfetch. De resulterende GeoJSON-artefacten,
checksums en bron-URL's worden onder persistent operator storage bewaard en
via de bestaande DatasetService/vectorfeature-flow geïmporteerd.
### Mol population history
`scripts/provision_mol_population_history.py` imports official Statbel
population-by-statistical-sector tables and matching sector geometries for
2021 through 2025. It keeps the published sector total as
`population_total`, clips the official geometry to Mol NIS `13025` and uploads
each year as a separate dataset in
`statbel:population-statistical-sector:mol`.
Complete sectors use their published population total. A rectangle that cuts
through a sector uses an explicitly labelled area-weighted estimate; the
source does not justify a more precise intra-sector distribution.
### Mol historical land use
`scripts/provision_mol_historical_landuse.py` uses the official Digitaal
Vlaanderen Historical Land Use WFS for the 1778, 1873 and 1969 collections.
Buildings, forest, water and roads are filtered server-side, clipped to the
official Mol boundary and uploaded through the existing dataset service.
Source classes, request URLs, observation year, simplification tolerance and
methodological limitations remain in provenance.
These historical map editions support exploratory area evolution, not
cadastral object lineage. Their feature identities are declared unstable and
GeoIntel does not fabricate added/removed object counts.
## OSM
- Naam: OpenStreetMap