feat: add temporal Mol explorer
This commit is contained in:
@@ -1393,3 +1393,53 @@ GET /api/v1/exports/{export_id}/download
|
||||
```
|
||||
|
||||
PDF/report-designer functionality can be added after core GeoAI workflows work.
|
||||
|
||||
## Temporal datasets and area evolution
|
||||
|
||||
Temporal metadata describes the source observation, not API run history. A
|
||||
snapshot is a normal persisted dataset grouped by `temporal_series_key` and
|
||||
ordered by `observed_at`. Optional validity uses `valid_from` and `valid_to`;
|
||||
`temporal_granularity` is `snapshot`, `day`, `month`, `year` or `period`.
|
||||
|
||||
Dataset upload accepts those temporal fields plus `source_version`. When a
|
||||
dataset declares `source_metadata.selection_aggregation`, the vector bbox
|
||||
selection response also contains a `summary` with metric label/value/unit,
|
||||
aggregation method, feature count, estimate status and an optional warning.
|
||||
Supported PostGIS aggregations are feature count, intersection area,
|
||||
intersection length, numeric sum and area-weighted numeric sum. Area and length
|
||||
are measured after transformation to EPSG:31370.
|
||||
|
||||
### PATCH `/api/v1/projects/{project_id}/datasets/{dataset_id}/temporal`
|
||||
|
||||
Updates the temporal provenance of an existing dataset. Series key and
|
||||
observation date are required together. It does not alter features or
|
||||
manufacture a historical observation.
|
||||
|
||||
### GET `/api/v1/projects/{project_id}/datasets/{dataset_id}/versions`
|
||||
|
||||
Lists immutable storage/provenance versions. Uploads and derived datasets
|
||||
create version 1 in the same persistence transaction.
|
||||
|
||||
### GET `/api/v1/projects/{project_id}/temporal/series`
|
||||
|
||||
Returns dated project series in the canonical envelope. Each item contains its
|
||||
source/layer identity, first and last observations and ordered datasets.
|
||||
|
||||
### POST `/api/v1/projects/{project_id}/temporal/compare`
|
||||
|
||||
```json
|
||||
{
|
||||
"earlier_dataset_id": "uuid",
|
||||
"later_dataset_id": "uuid",
|
||||
"bbox": {"west": 5.0, "south": 51.0, "east": 5.2, "north": 51.2},
|
||||
"preview_limit": 500
|
||||
}
|
||||
```
|
||||
|
||||
Both datasets must belong to the project and the same temporal series, with
|
||||
the earlier observation preceding the later one. The response contains source
|
||||
snapshot references, selection bbox, earlier/later metric values,
|
||||
absolute/percentage change, estimate status, warnings and GeoJSON evidence.
|
||||
Added/removed/modified object changes are calculated only when source
|
||||
provenance declares stable feature identities; otherwise
|
||||
`object_changes.available=false` and no object history is inferred.
|
||||
|
||||
@@ -7768,3 +7768,29 @@ Limitations:
|
||||
|
||||
Next:
|
||||
- Define authoritative population and land-cover source adapters, then reuse the proven municipality provisioner and bbox analysis flow for the complete Kempen.
|
||||
|
||||
## Sprint 187 Temporal Mol explorer (2026-07-14)
|
||||
|
||||
Implemented:
|
||||
- Added observation/validity/source-version fields to datasets and immutable provenance fields to dataset versions, with one Alembic head and indexed temporal/source identity lookups.
|
||||
- Persisted dataset version 1 atomically for uploads, demo fixtures and derived vector/raster operations.
|
||||
- Added source-governed PostGIS selection summaries for object count, area, length, numeric sum and area-weighted sum.
|
||||
- Added project temporal-series discovery and same-series bbox comparison with honest metric deltas and stable-identity-only object changes.
|
||||
- Added the map-first `Laatste toestand` / `Evolutie` workflow with period selection, automatic rectangle analysis and change overlays.
|
||||
- Added explicit, idempotent Statbel population (2021-2025) and Digitaal Vlaanderen historical land-use (1778/1873/1969) provisioners for Mol.
|
||||
- Kept all source fetching operator-triggered; application startup and user queries never fabricate or silently download source data.
|
||||
|
||||
Methodology:
|
||||
- Population totals are source-published per statistical sector. Intersections with partial sectors are labelled area-weighted estimates.
|
||||
- Historical land-use classes are clipped from official editions and measured in EPSG:31370. They do not claim stable cadastral object identity.
|
||||
- Every snapshot carries its source URL, observation date, source version, checksum and processing limitations.
|
||||
|
||||
Validation before live deployment:
|
||||
- Script compilation passed.
|
||||
- New temporal/API/static regression suite passed 6 tests, including append-only/idempotent temporal provenance updates.
|
||||
- Raster and temporal focused suite passed 27 tests after extending existing assertions to require dataset-version persistence.
|
||||
- Frontend TypeScript typecheck and production build passed.
|
||||
- Offline Alembic SQL generation passed with head `202607140001`.
|
||||
|
||||
Next:
|
||||
- Run the complete readiness gate, deploy to Tower/PostGIS, provision the official snapshots and verify current/evolution selection end to end in the internal browser.
|
||||
|
||||
@@ -236,3 +236,22 @@ GRB and OSM live imports are intentionally `not_configured` in Sprint 7B. Manual
|
||||
- Multi-tenant row-level security.
|
||||
- User accounts.
|
||||
- Full model registry tables.
|
||||
|
||||
## Temporal dataset foundation
|
||||
|
||||
Historical observations remain normal `datasets` and `vector_features`; there
|
||||
is no parallel temporal feature store. Snapshots are grouped by
|
||||
`datasets.temporal_series_key` and carry `observed_at`, `valid_from`,
|
||||
`valid_to`, `temporal_granularity` and `source_version`. Observation time is
|
||||
kept separate from ingestion time (`imported_at`).
|
||||
|
||||
`dataset_versions` records immutable storage provenance for every upload and
|
||||
derived output: dataset-local `version`, storage path, source version,
|
||||
observation/validity dates, checksum and source/provenance JSON. The
|
||||
`(dataset_id, version)` pair is unique. Temporal series lookup is indexed by
|
||||
`(project_id, temporal_series_key, observed_at)` and source-feature lookup by
|
||||
`(dataset_id, source_feature_id)`.
|
||||
|
||||
Time-series comparison is read-only and aggregates persisted geometry inside a
|
||||
requested bbox. Object-level added/removed/modified evidence is only valid for
|
||||
sources that explicitly declare stable source feature identifiers.
|
||||
|
||||
@@ -138,3 +138,20 @@ V1 dataset strategy is complete when:
|
||||
- an area can request/cache a reference building layer;
|
||||
- detection outputs can be compared with that reference layer;
|
||||
- exports include source metadata.
|
||||
|
||||
## Temporal snapshots and evolution
|
||||
|
||||
- A historical observation is one persisted dataset snapshot. Existing
|
||||
datasets are not overwritten and features are not hidden in job JSON.
|
||||
- Related observations share a stable `temporal_series_key`; `observed_at`
|
||||
records when the source describes reality, while `imported_at` records
|
||||
ingestion time.
|
||||
- The latest-state map uses the latest available observation but must not call
|
||||
an old source edition current reality.
|
||||
- Evolution metrics use the same selection geometry, aggregation and units for
|
||||
both snapshots.
|
||||
- Partial-sector population is an area-weighted estimate. GeoIntel must not
|
||||
imply address-level distribution when only sector totals are available.
|
||||
- Historical cartographic classes can change meaning between editions. Source
|
||||
classes and processing notes remain provenance, and object changes require
|
||||
explicit stable source identity.
|
||||
|
||||
@@ -29,6 +29,32 @@ start geen verborgen providerfetch. De resulterende GeoJSON-artefacten,
|
||||
checksums en bron-URL's worden onder persistent operator storage bewaard en
|
||||
via de bestaande DatasetService/vectorfeature-flow geïmporteerd.
|
||||
|
||||
### Mol population history
|
||||
|
||||
`scripts/provision_mol_population_history.py` imports official Statbel
|
||||
population-by-statistical-sector tables and matching sector geometries for
|
||||
2021 through 2025. It keeps the published sector total as
|
||||
`population_total`, clips the official geometry to Mol NIS `13025` and uploads
|
||||
each year as a separate dataset in
|
||||
`statbel:population-statistical-sector:mol`.
|
||||
|
||||
Complete sectors use their published population total. A rectangle that cuts
|
||||
through a sector uses an explicitly labelled area-weighted estimate; the
|
||||
source does not justify a more precise intra-sector distribution.
|
||||
|
||||
### Mol historical land use
|
||||
|
||||
`scripts/provision_mol_historical_landuse.py` uses the official Digitaal
|
||||
Vlaanderen Historical Land Use WFS for the 1778, 1873 and 1969 collections.
|
||||
Buildings, forest, water and roads are filtered server-side, clipped to the
|
||||
official Mol boundary and uploaded through the existing dataset service.
|
||||
Source classes, request URLs, observation year, simplification tolerance and
|
||||
methodological limitations remain in provenance.
|
||||
|
||||
These historical map editions support exploratory area evolution, not
|
||||
cadastral object lineage. Their feature identities are declared unstable and
|
||||
GeoIntel does not fabricate added/removed object counts.
|
||||
|
||||
## OSM
|
||||
|
||||
- Naam: OpenStreetMap
|
||||
|
||||
Reference in New Issue
Block a user