feat: add governed hydrology and historical imagery
This commit is contained in:
+25
-5
@@ -191,15 +191,24 @@ Response: `DatasetRead` with extracted metadata if supported.
|
||||
|
||||
Vector uploads remain stored as original files and are also persisted into `vector_features` as queryable PostGIS state.
|
||||
|
||||
### GET `/api/v1/projects/{project_id}/datasets/orthophoto/products`
|
||||
|
||||
Return the governed Digitaal Vlaanderen orthophoto product allowlist in the
|
||||
canonical envelope. Every product reports its key, display/observation label,
|
||||
temporal granularity, native resolution, colour mode, catalogue URL,
|
||||
limitations and whether current configured-YOLO detection is allowed.
|
||||
|
||||
### POST `/api/v1/projects/{project_id}/datasets/orthophoto/acquire`
|
||||
|
||||
Explicitly acquire a bounded most-recent winter orthophoto selection from the
|
||||
official Digitaal Vlaanderen `OMWRGBMRVL` WMS `Ortho` layer.
|
||||
Explicitly acquire a bounded orthophoto selection from a governed official
|
||||
Digitaal Vlaanderen WMS product. Arbitrary WMS URLs and layer names are not
|
||||
accepted.
|
||||
|
||||
```json
|
||||
{
|
||||
"bbox": {"min_x": 5.10, "min_y": 51.17, "max_x": 5.11, "max_y": 51.18, "crs": "EPSG:4326"},
|
||||
"area_id": "optional-project-area-uuid",
|
||||
"product_key": "most_recent",
|
||||
"force_refresh": false
|
||||
}
|
||||
```
|
||||
@@ -207,7 +216,8 @@ official Digitaal Vlaanderen `OMWRGBMRVL` WMS `Ortho` layer.
|
||||
The canonical envelope contains a synchronous Job. Its `output_dataset_id`
|
||||
identifies the raster Dataset; `result_json` contains provider, layer, pixel
|
||||
dimensions, EPSG:4326/EPSG:31370 bounds, sampling resolution, attribution,
|
||||
cache reuse and limitation text.
|
||||
cache reuse and limitation text. Historical products also persist their
|
||||
observation/validity period and a spatially scoped temporal-series key.
|
||||
|
||||
Safety contract:
|
||||
|
||||
@@ -218,8 +228,11 @@ Safety contract:
|
||||
reuse;
|
||||
- WMS bytes are georeferenced to EPSG:31370 and persisted only through
|
||||
`DatasetService`; no fetch runs on startup;
|
||||
- this is the latest mosaic available at request time, not a historical
|
||||
observation date for every pixel.
|
||||
- only `most_recent` can enter the current configured-YOLO plus GRB-QA path;
|
||||
historical products are visual evidence and are never validated against the
|
||||
current GRB state;
|
||||
- product periods such as `1979_1990` remain explicitly multi-year and are not
|
||||
presented as exact annual observations.
|
||||
|
||||
### GET `/api/v1/projects/{project_id}/datasets`
|
||||
|
||||
@@ -274,6 +287,13 @@ If preview dependencies are unavailable:
|
||||
- code: `RASTER_PROCESSING_UNAVAILABLE`
|
||||
- message: `Raster preview unavailable...`
|
||||
|
||||
### GET `/api/v1/projects/{project_id}/datasets/{dataset_id}/raster/image`
|
||||
|
||||
Return a persisted orthophoto Dataset as a bounded browser-safe PNG. This is an
|
||||
explicit binary non-envelope endpoint used by the MapLibre image source. It
|
||||
accepts only ready datasets from the governed orthophoto provider and never
|
||||
reads arbitrary filesystem paths.
|
||||
|
||||
### POST `/api/v1/projects/{project_id}/datasets/{dataset_id}/raster/clip`
|
||||
|
||||
Clip raster by selected area. Returns a `202`-style accepted job payload through the job wrapper (`jobs` create/read flow).
|
||||
|
||||
@@ -8438,3 +8438,33 @@ Next:
|
||||
- Integrate Waterinfo/VMM station observations as point time series and add
|
||||
historical orthophoto acquisition, while preserving their spatial and
|
||||
methodological limitations.
|
||||
|
||||
## Sprint 203 - Governed Waterinfo history and historical orthophotos (2026-07-15)
|
||||
|
||||
Implemented:
|
||||
- Added `provision_waterinfo_station_history.py` using the documented Waterinfo
|
||||
KiWIS annual water-level/discharge groups. The operator filters station
|
||||
points against the exact persisted Area, retains raw JSON plus SHA256
|
||||
manifests and writes only through the canonical upload API.
|
||||
- Persisted one Point Dataset per station/year and kept station identities in
|
||||
separate temporal series. Added backend `mean` aggregation without combining
|
||||
stations or inferring area-wide water level/volume.
|
||||
- Added a governed orthophoto allowlist for the most-recent product, annual
|
||||
winter mosaics 2012-2025, older winter periods, RGB 1979-1990 and
|
||||
panchromatic 1971. Arbitrary WMS URLs/layers remain impossible.
|
||||
- Added raster temporal metadata and a constrained PNG rendering endpoint for
|
||||
persisted orthophoto datasets. The map can select and display official
|
||||
historical imagery over the same bounded rectangle.
|
||||
- Historical products explicitly bypass configured-YOLO and current-GRB QA;
|
||||
only `most_recent` retains that path.
|
||||
- Updated source inventory behavior and wrote a governed source backlog with
|
||||
acceptance criteria for BWK/Natura 2000, agricultural parcels, Buildings
|
||||
Register, DHMV and bathymetry.
|
||||
|
||||
Validation evidence:
|
||||
- Focused Waterinfo/orthophoto/semantic metric tests and frontend typecheck/build
|
||||
passed before the full repository gate.
|
||||
|
||||
Remaining operational step:
|
||||
- Deploy the all-in-one image, provision the real Mol Waterinfo station series
|
||||
and verify one historical image acquisition/overlay through the LAN browser.
|
||||
|
||||
+29
-6
@@ -214,9 +214,6 @@ These sources are available from their public authorities but are not silently
|
||||
treated as loaded GeoIntel data. The Source inventory labels them separately
|
||||
until a governed operator import, provenance record and validation pass exist.
|
||||
|
||||
- Historical orthophotos (Digitaal Vlaanderen): 1971 and 1979-1990 through
|
||||
the `OKZ` WMS, with additional dated mosaics as separate products. Suitable
|
||||
for visual/image evolution after a bounded acquisition contract is added.
|
||||
- Biologische Waarderingskaart / Natura 2000 (INBO), state 2025: suitable for
|
||||
habitat, biotope and ecological-value analysis, not a continuous annual
|
||||
series.
|
||||
@@ -229,9 +226,35 @@ until a governed operator import, provenance record and validation pass exist.
|
||||
- DHMV II DTM/DSM (Digitaal Vlaanderen): 1 m/5 m elevation based on 2013-2015
|
||||
LiDAR, suitable for elevation, slope and drainage. It does not provide water
|
||||
depth.
|
||||
- Waterinfo/VMM: station time series for water level, flow and precipitation.
|
||||
These can describe hydrological state, but do not provide area-wide water
|
||||
volume without a compatible bottom profile/bathymetry model.
|
||||
|
||||
## Governed Waterinfo station history
|
||||
|
||||
`scripts/provision_waterinfo_station_history.py` uses the public Waterinfo
|
||||
KiWIS query service only after an explicit operator command. It discovers the
|
||||
documented annual water-level (`192784`) and discharge (`192895`) groups,
|
||||
filters station points against the exact persisted Area and retains the raw
|
||||
station/value JSON plus SHA256 manifests.
|
||||
|
||||
Each station and year becomes one immutable reference Dataset through the
|
||||
normal upload API. Series keys include the station identity; different stations
|
||||
are never averaged into one municipal value. Selection aggregation is a numeric
|
||||
mean over the selected station records and remains labelled as a point
|
||||
measurement. Water level or discharge does not establish area-wide water
|
||||
volume without compatible depth, profile and coverage data.
|
||||
|
||||
## Governed historical orthophotos
|
||||
|
||||
The bounded map acquisition registry includes the official annual winter
|
||||
mosaics for 2012-2025, period products for 2000-2003, 2005-2007 and 2008-2011,
|
||||
the RGB 1979-1990 summer mosaic and the panchromatic 1971 mosaic. Requests stay
|
||||
within the configured 128-1,024 m safety envelope and are persisted as
|
||||
EPSG:31370 raster Datasets with explicit product, layer, observation period,
|
||||
attribution and request hashes.
|
||||
|
||||
Historical mosaics are for visual comparison only in this phase. Current YOLO
|
||||
building detection and current GRB QA remain restricted to `most_recent`, since
|
||||
validating an old image against today's building state would produce dishonest
|
||||
quality metrics.
|
||||
|
||||
## OSM
|
||||
|
||||
|
||||
@@ -198,6 +198,21 @@ Metadata:
|
||||
- nodata
|
||||
- resolution
|
||||
|
||||
Temporal orthophoto rasters additionally require a governed product key, WMS
|
||||
layer, observation label, `observed_at`, optional `valid_from`/`valid_to`,
|
||||
temporal granularity, request/spatial hash, attribution and a limitation that
|
||||
states whether the product is annual, multi-year or merely most recent.
|
||||
|
||||
### Hydrological station observations
|
||||
|
||||
Waterinfo observations are persisted as EPSG:4326 Point features, one station
|
||||
and one observation year per immutable Dataset. Required properties are the
|
||||
station/timeseries identity, measurement type, numeric annual value, reported
|
||||
unit, observation year, provider/owner and attribution. Different station
|
||||
series may not be merged into one area-wide value. Point water level and
|
||||
discharge may not be converted to water volume without governed compatible
|
||||
depth/profile data.
|
||||
|
||||
### Vector
|
||||
|
||||
Ondersteund:
|
||||
|
||||
@@ -99,6 +99,16 @@ Mask files are provenance/debug artifacts. QA, map display and GeoJSON output mu
|
||||
|
||||
Do not delete originals automatically. Derived outputs may be cleaned through explicit cache management.
|
||||
|
||||
## Official temporal source artifacts
|
||||
|
||||
Waterinfo raw station layers, timeseries responses and checksum manifests live
|
||||
under `storage/operator-data/waterinfo/<scope>/`. These are immutable source
|
||||
evidence; queryable annual Point snapshots are normal Dataset/vector_feature
|
||||
records. Bounded orthophotos are normal raster Dataset files. Their WMS URL,
|
||||
product/layer, request/spatial hash, temporal validity and limitations are held
|
||||
in source/provenance metadata. Browser PNG rendering is derived on request and
|
||||
does not replace the stored GeoTIFF.
|
||||
|
||||
Offline demo export artifacts can be inspected and cleaned with:
|
||||
|
||||
```bash
|
||||
|
||||
+45
-2
@@ -22,12 +22,55 @@
|
||||
- [x] Add a source inventory that separates loaded data from audited official follow-up sources.
|
||||
- [x] Add a local Ollama question window grounded in persisted GeoIntel metrics and installed server models.
|
||||
- [ ] Add a governed depth/bathymetry source before exposing water volume; never infer volume from 2D GRB water geometry.
|
||||
- [ ] Integrate one governed hydrology source (Waterinfo/VMM station series) without presenting point measurements as area-wide water volume.
|
||||
- [ ] Add bounded historical orthophoto acquisition and visual change analysis after validating layer/year coverage.
|
||||
- [x] Integrate governed Waterinfo/VMM annual station series without presenting point measurements as area-wide water volume.
|
||||
- [x] Add bounded historical orthophoto acquisition for official 1971-2025 products with a map overlay and no current-GRB QA on old imagery.
|
||||
- [ ] Add BWK/Natura 2000 and annual agricultural-use parcels through explicit provider/operator contracts.
|
||||
- [ ] Extend the official 1778/1873/1969 historical buildings, water and roads series from Mol to the approved regional scope with partitioned source audits.
|
||||
- [x] Connect a drawn rectangle to bounded official orthophoto acquisition, local configured-YOLO detection and persisted GRB QA.
|
||||
|
||||
## Governed source expansion backlog
|
||||
|
||||
Implement these in order. Every source must use an explicit operator/provider
|
||||
contract, retain raw checksummed evidence, persist through DatasetService and
|
||||
pass live Mol validation before regional expansion.
|
||||
|
||||
### P1 - Biologische Waarderingskaart / Natura 2000
|
||||
|
||||
- [ ] Confirm the official downloadable service/layer/version and licence for the 2025 state.
|
||||
- [ ] Define stable habitat/biotope/value fields and keep BWK valuation separate from Natura 2000 habitat classification.
|
||||
- [ ] Clip in EPSG:31370 to Mol, transform to EPSG:4326 and persist valid polygonal `vector_features` with source feature ids.
|
||||
- [ ] Add hectare metrics per governed class plus unknown/unmapped-class diagnostics; do not invent an annual trend from one state.
|
||||
- [ ] Add source manifest, unit tests, live row/count/geometry audit and map/source-inventory presentation.
|
||||
|
||||
### P2 - Annual agricultural-use parcels
|
||||
|
||||
- [ ] Inventory official annual editions and document schema/code-list changes before selecting a comparable year range.
|
||||
- [ ] Retain annual source files and crop code lists; normalize only fields whose meaning is stable across editions.
|
||||
- [ ] Persist separate annual Datasets with hectares by crop/use class and an explicit unstable-parcel-identity limitation.
|
||||
- [ ] Compare area/category totals over time; do not claim parcel lineage where identifiers or boundaries changed.
|
||||
- [ ] Validate Mol first, then partition all 28 Kempen municipalities with completeness/checksum manifests.
|
||||
|
||||
### P3 - Buildings and Addresses Register
|
||||
|
||||
- [ ] Define a governed snapshot operator and relation mapping between building unit, building object, address and GRB geometry.
|
||||
- [ ] Keep register lifecycle/status semantics separate from GRB footprint geometry and document match confidence/unmatched rows.
|
||||
- [ ] Add current building-status/address metrics without exposing personal data or treating addresses as households/population.
|
||||
- [ ] Add Mol reconciliation tests and a live mismatch report before regional import.
|
||||
|
||||
### P4 - Digitaal Hoogtemodel Vlaanderen
|
||||
|
||||
- [ ] Select official DTM/DSM product, service and native resolution; retain acquisition date and vertical reference.
|
||||
- [ ] Add bounded raster acquisition/clip storage with nodata, CRS, resolution and checksum validation.
|
||||
- [ ] Implement governed elevation, relief and slope statistics in metres/degrees; keep drainage interpretation explicitly derived.
|
||||
- [ ] Do not label terrain/surface height as water depth and do not enable volume from DHMV alone.
|
||||
|
||||
### P5 - Water depth / bathymetry
|
||||
|
||||
- [ ] Identify an authoritative source with compatible spatial coverage, vertical datum, date and uncertainty; otherwise keep volume unavailable.
|
||||
- [ ] Define waterbody linkage, surface elevation, bottom elevation and uncertainty propagation before adding any volume metric.
|
||||
- [ ] Validate coverage gaps and prohibit extrapolation outside measured/profiled waterbodies.
|
||||
- [ ] Add independent GIS review and golden-volume fixtures before exposing the result to users or Ollama.
|
||||
|
||||
This file now starts with the current implementation status. Older preparation/backlog sections are preserved below as historical planning context and should not be treated as the live sprint board without checking `docs/CODEX_EXECUTION_LOG.md`.
|
||||
|
||||
## Release hardening status
|
||||
|
||||
Reference in New Issue
Block a user