Complete regional raster exploration
GeoIntel CI / docs-smoke (push) Canceled after 0s
GeoIntel CI / contract-smoke (push) Canceled after 0s

This commit is contained in:
Codex
2026-07-16 14:21:32 +02:00
parent 153cff06c0
commit 45dc730e76
24 changed files with 1162 additions and 123 deletions
+21
View File
@@ -7,6 +7,27 @@
# Changelog # Changelog
## Sprint 218 Regional terrain and flood completion (2026-07-16)
- Provisioned and audited the complete governed Kempen matrices: 56 DHMV
DTM/DSM rasters and 336 VMM flood-scenario rasters across all 28 approved
municipalities, each with one DatasetVersion and a non-empty checksum-bound
GeoTIFF.
- Bounded official WCS edge-grid rounding to at most 5%/0.25 m for DHMV and
VMM tile assembly, resampled accepted edge tiles to the exact 5 m analysis
grid and retained every source resolution and harmonized tile index in
provenance. Larger mismatches still fail closed.
- Added exact regional raster-selection endpoints. They open only intersecting
persisted municipality partitions, mosaic the selected windows in memory
and calculate global cell statistics under the existing 12-million-cell
guard; no monolithic or hidden authoritative raster is created.
- Updated the Map workspace to expose DHMV and VMM on the complete Kempen Area,
render all 28 matching MapLibre partitions, deduplicate VMM into twelve
scenario choices and analyse a drawn cross-boundary rectangle without first
selecting a municipality.
- Preserved the distinction between DHMV height, modeled VMM scenario depth,
permanent water, bathymetry and concurrent flood volume.
## Sprint 217 Regional DOV soil coverage (2026-07-16) ## Sprint 217 Regional DOV soil coverage (2026-07-16)
- Added a governed regional DOV soil operator for all 28 approved Kempen - Added a governed regional DOV soil operator for all 28 approved Kempen
+12
View File
@@ -1246,6 +1246,12 @@ and a complete failure summary. Persistence remains inside the canonical
DHMV acquisition service and Dataset/DatasetVersion/Job flow; the operator DHMV acquisition service and Dataset/DatasetVersion/Job flow; the operator
does not fetch WCS bytes or write raster metadata directly. does not fetch WCS bytes or write raster metadata directly.
The complete live matrix contains 56 ready Datasets and 56 DatasetVersions
across 28 Areas. On the complete Kempen Area the Map workspace presents those
partitions as one logical DTM/DSM layer. `POST .../datasets/raster/terrain/select`
opens only partitions intersecting the drawn rectangle and computes exact
global cell statistics. It does not create a hidden regional mosaic.
Settings: `DHMV_ENABLED`, `DHMV_WCS_URL`, `DHMV_RESOLUTION_M`, Settings: `DHMV_ENABLED`, `DHMV_WCS_URL`, `DHMV_RESOLUTION_M`,
`DHMV_MIN_SIDE_M`, `DHMV_MAX_SIDE_M`, `DHMV_MAX_PIXELS`, `DHMV_MIN_SIDE_M`, `DHMV_MAX_SIDE_M`, `DHMV_MAX_PIXELS`,
`DHMV_TIMEOUT_SECONDS` and `DHMV_MAX_RESPONSE_MB`. `DHMV_TIMEOUT_SECONDS` and `DHMV_MAX_RESPONSE_MB`.
@@ -1295,6 +1301,12 @@ The full Kempen scope with all products means 28 municipalities times 12
scenario rasters. This is intentionally explicit operator work, not startup scenario rasters. This is intentionally explicit operator work, not startup
work and not a browser-side provider fetch. work and not a browser-side provider fetch.
The complete live matrix contains 336 ready Datasets and 336 DatasetVersions.
The regional Map workspace deduplicates them into twelve scenario choices,
renders all municipality image partitions for the selected scenario and uses
`POST .../datasets/raster/flood-hazard/select` for exact bounded cross-boundary
analysis. The same 12-million-cell guard prevents unsafe full-region reads.
Use `--products pluviaal_current_t100`, `--resolution-m 5` or `--force` for an Use `--products pluviaal_current_t100`, `--resolution-m 5` or `--force` for an
explicit subset/refresh. `POST .../raster/flood-hazard/select` returns mapped explicit subset/refresh. `POST .../raster/flood-hazard/select` returns mapped
inundated hectares, selection share and local modeled maximum-depth statistics. inundated hectares, selection share and local modeled maximum-depth statistics.
+21 -4
View File
@@ -4,7 +4,6 @@ import json
from datetime import datetime from datetime import datetime
from typing import Any from typing import Any
from uuid import UUID from uuid import UUID
from uuid import UUID as _UUID
from fastapi import APIRouter, Depends, File, Form, HTTPException, Query, Response from fastapi import APIRouter, Depends, File, Form, HTTPException, Query, Response
from fastapi import UploadFile from fastapi import UploadFile
@@ -23,8 +22,10 @@ from app.schemas import (
RasterNdbiRequest, RasterNdbiRequest,
OrthophotoAcquireRequest, OrthophotoAcquireRequest,
DhmvAcquireRequest, DhmvAcquireRequest,
TerrainPartitionSelectionRequest,
TerrainSelectionRequest, TerrainSelectionRequest,
FloodHazardAcquireRequest, FloodHazardAcquireRequest,
FloodHazardPartitionSelectionRequest,
FloodHazardSelectionRequest, FloodHazardSelectionRequest,
ThematicRasterAcquireRequest, ThematicRasterAcquireRequest,
ThematicRasterSelectionRequest, ThematicRasterSelectionRequest,
@@ -32,14 +33,12 @@ from app.schemas import (
VectorBufferRequest, VectorBufferRequest,
VectorClipRequest, VectorClipRequest,
VectorIntersectRequest, VectorIntersectRequest,
VectorSelectionBBox, VectorSelectionBBox, # noqa: F401 - retained as a route-module compatibility export
VectorSelectionDeriveRequest, VectorSelectionDeriveRequest,
VectorSelectionRequest, VectorSelectionRequest,
VectorSelectionResponse, VectorSelectionResponse,
) )
from app.schemas.job import JobCreate
from app.schemas.dataset import DatasetCreateResponse, DatasetTemporalUpdate from app.schemas.dataset import DatasetCreateResponse, DatasetTemporalUpdate
from app.schemas.operations import VectorOperationResult
from app.services.job_service import JobService from app.services.job_service import JobService
from app.services.raster_operations_service import RasterOperationsService from app.services.raster_operations_service import RasterOperationsService
from app.services.vector_operations_service import VectorOperationsService from app.services.vector_operations_service import VectorOperationsService
@@ -550,6 +549,15 @@ def raster_terrain_selection(
return envelope(TerrainAnalysisService.analyze(db, project_id, dataset_id, payload)) return envelope(TerrainAnalysisService.analyze(db, project_id, dataset_id, payload))
@router.post("/datasets/raster/terrain/select", response_model=dict)
def partitioned_raster_terrain_selection(
project_id: UUID,
payload: TerrainPartitionSelectionRequest,
db: Session = Depends(get_db),
):
return envelope(TerrainAnalysisService.analyze_partitions(db, project_id, payload))
@router.get("/datasets/{dataset_id}/raster/terrain/image") @router.get("/datasets/{dataset_id}/raster/terrain/image")
def raster_terrain_image( def raster_terrain_image(
project_id: UUID, project_id: UUID,
@@ -574,6 +582,15 @@ def raster_flood_hazard_selection(
return envelope(FloodHazardAnalysisService.analyze(db, project_id, dataset_id, payload)) return envelope(FloodHazardAnalysisService.analyze(db, project_id, dataset_id, payload))
@router.post("/datasets/raster/flood-hazard/select", response_model=dict)
def partitioned_raster_flood_hazard_selection(
project_id: UUID,
payload: FloodHazardPartitionSelectionRequest,
db: Session = Depends(get_db),
):
return envelope(FloodHazardAnalysisService.analyze_partitions(db, project_id, payload))
@router.get("/datasets/{dataset_id}/raster/flood-hazard/image") @router.get("/datasets/{dataset_id}/raster/flood-hazard/image")
def raster_flood_hazard_image( def raster_flood_hazard_image(
project_id: UUID, project_id: UUID,
+4
View File
@@ -38,6 +38,7 @@ from .dhmv import (
DhmvAcquisitionResult, DhmvAcquisitionResult,
DhmvProductRead, DhmvProductRead,
TerrainMetric, TerrainMetric,
TerrainPartitionSelectionRequest,
TerrainSelectionRequest, TerrainSelectionRequest,
TerrainSelectionResponse, TerrainSelectionResponse,
TerrainSelectionSummary, TerrainSelectionSummary,
@@ -46,6 +47,7 @@ from .flood_hazard import (
FloodHazardAcquireRequest, FloodHazardAcquireRequest,
FloodHazardAcquisitionResult, FloodHazardAcquisitionResult,
FloodHazardMetric, FloodHazardMetric,
FloodHazardPartitionSelectionRequest,
FloodHazardProductRead, FloodHazardProductRead,
FloodHazardSelectionRequest, FloodHazardSelectionRequest,
FloodHazardSelectionResponse, FloodHazardSelectionResponse,
@@ -166,12 +168,14 @@ __all__ = [
"DhmvAcquisitionResult", "DhmvAcquisitionResult",
"DhmvProductRead", "DhmvProductRead",
"TerrainMetric", "TerrainMetric",
"TerrainPartitionSelectionRequest",
"TerrainSelectionRequest", "TerrainSelectionRequest",
"TerrainSelectionResponse", "TerrainSelectionResponse",
"TerrainSelectionSummary", "TerrainSelectionSummary",
"FloodHazardAcquireRequest", "FloodHazardAcquireRequest",
"FloodHazardAcquisitionResult", "FloodHazardAcquisitionResult",
"FloodHazardMetric", "FloodHazardMetric",
"FloodHazardPartitionSelectionRequest",
"FloodHazardProductRead", "FloodHazardProductRead",
"FloodHazardSelectionRequest", "FloodHazardSelectionRequest",
"FloodHazardSelectionResponse", "FloodHazardSelectionResponse",
+6
View File
@@ -56,6 +56,10 @@ class TerrainSelectionRequest(BaseModel):
area_id: UUID | None = None area_id: UUID | None = None
class TerrainPartitionSelectionRequest(TerrainSelectionRequest):
product_key: str = "dtm_1m"
class TerrainMetric(BaseModel): class TerrainMetric(BaseModel):
metric_key: str metric_key: str
metric_label: str metric_label: str
@@ -76,6 +80,8 @@ class TerrainSelectionSummary(BaseModel):
class TerrainSelectionResponse(BaseModel): class TerrainSelectionResponse(BaseModel):
dataset_id: UUID dataset_id: UUID
dataset_ids: list[UUID] = Field(default_factory=list)
partition_count: int = Field(default=1, ge=1)
product_key: str product_key: str
surface_model: str surface_model: str
selection_bbox: VectorSelectionBBox selection_bbox: VectorSelectionBBox
+6
View File
@@ -59,6 +59,10 @@ class FloodHazardSelectionRequest(BaseModel):
area_id: UUID | None = None area_id: UUID | None = None
class FloodHazardPartitionSelectionRequest(FloodHazardSelectionRequest):
product_key: str = "pluviaal_current_t100"
class FloodHazardMetric(BaseModel): class FloodHazardMetric(BaseModel):
metric_key: str metric_key: str
metric_label: str metric_label: str
@@ -79,6 +83,8 @@ class FloodHazardSelectionSummary(BaseModel):
class FloodHazardSelectionResponse(BaseModel): class FloodHazardSelectionResponse(BaseModel):
dataset_id: UUID dataset_id: UUID
dataset_ids: list[UUID] = Field(default_factory=list)
partition_count: int = Field(default=1, ge=1)
product_key: str product_key: str
mechanism: str mechanism: str
climate_context: str climate_context: str
@@ -16,11 +16,13 @@ from app.core.errors import AppError
from app.models import Area, Dataset from app.models import Area, Dataset
from app.schemas.flood_hazard import ( from app.schemas.flood_hazard import (
FloodHazardMetric, FloodHazardMetric,
FloodHazardPartitionSelectionRequest,
FloodHazardSelectionRequest, FloodHazardSelectionRequest,
FloodHazardSelectionResponse, FloodHazardSelectionResponse,
FloodHazardSelectionSummary, FloodHazardSelectionSummary,
) )
from app.services.flood_hazard_acquisition_service import FloodHazardAcquisitionService from app.services.flood_hazard_acquisition_service import FloodHazardAcquisitionService
from app.services.raster_partition_analysis_service import RasterPartitionAnalysisService
class FloodHazardAnalysisService: class FloodHazardAnalysisService:
@@ -170,6 +172,8 @@ class FloodHazardAnalysisService:
primary = metrics[0] primary = metrics[0]
response = FloodHazardSelectionResponse( response = FloodHazardSelectionResponse(
dataset_id=dataset.id, dataset_id=dataset.id,
dataset_ids=[dataset.id],
partition_count=1,
product_key=product.key, product_key=product.key,
mechanism=product.mechanism, mechanism=product.mechanism,
climate_context=product.climate_context, climate_context=product.climate_context,
@@ -195,6 +199,129 @@ class FloodHazardAnalysisService:
) )
return response.model_dump(mode="json") return response.model_dump(mode="json")
@staticmethod
def analyze_partitions(
db,
project_id: UUID,
payload: FloodHazardPartitionSelectionRequest,
*,
settings: Settings | None = None,
) -> dict:
resolved_settings = settings or get_settings()
product = FloodHazardAcquisitionService._products().get(payload.product_key.strip().lower())
if product is None:
raise AppError(
code="FLOOD_HAZARD_PRODUCT_NOT_SUPPORTED",
message="Select a governed VMM fluvial or pluvial flood-depth scenario",
details={"product_key": payload.product_key},
status_code=422,
)
selection_4326 = FloodHazardAnalysisService._selection_geometry(db, project_id, payload)
partition = RasterPartitionAnalysisService.select(
db,
project_id,
source_name=FloodHazardAcquisitionService.PROVIDER,
product_key=product.key,
selection_geometry_4326=selection_4326,
nodata=FloodHazardAcquisitionService.NODATA,
max_pixels=resolved_settings.flood_hazard_max_pixels,
)
try:
import numpy as np
except ImportError as exc:
raise AppError(
code="RASTER_PROCESSING_UNAVAILABLE",
message="Numpy is required for partitioned flood-hazard analysis",
status_code=503,
) from exc
raw = partition.values
valid = (
partition.selected_cells
& np.isfinite(raw)
& (raw != FloodHazardAcquisitionService.NODATA)
& (raw > 0.0)
)
values = raw[valid]
selected_cell_count = int(partition.selected_cells.sum())
inundated_cell_count = int(values.size)
cell_area_m2 = partition.resolution_x * partition.resolution_y
def metric(key: str, label: str, value: float, unit: str, method: str) -> FloodHazardMetric:
return FloodHazardMetric(
metric_key=key,
metric_label=label,
metric_value=round(float(value), 4),
metric_unit=unit,
aggregation_method=method,
)
inundated_area_ha = inundated_cell_count * cell_area_m2 / 10_000.0
metrics = [
metric(
"modelled_inundated_area_ha",
"Gemodelleerd overstroomd oppervlak",
inundated_area_ha,
"ha",
"positive_depth_cells_times_cell_area",
),
metric(
"modelled_inundated_share_pct",
"Aandeel selectie met gemodelleerde diepte",
inundated_cell_count / max(1, selected_cell_count) * 100.0,
"%",
"positive_depth_cells_divided_by_selected_cells",
),
]
if inundated_cell_count:
metrics.extend(
[
metric("modelled_depth_mean_m", "Gemiddelde gemodelleerde maximumdiepte", values.mean(), "m", "mean_positive_depth_cells"),
metric("modelled_depth_p90_m", "90e percentiel gemodelleerde maximumdiepte", np.percentile(values, 90), "m", "percentile_90_positive_depth_cells"),
metric("modelled_depth_max_m", "Hoogste gemodelleerde maximumdiepte", values.max(), "m", "maximum_positive_depth_cells"),
metric(
"modelled_max_depth_area_integral_m3",
"Diepte-oppervlakte-integraal (geen gelijktijdig volume)",
values.sum() * cell_area_m2,
"m3",
"sum_local_max_depth_times_cell_area",
),
]
)
primary = metrics[0]
first_dataset = partition.datasets[0]
response = FloodHazardSelectionResponse(
dataset_id=first_dataset.id,
dataset_ids=[dataset.id for dataset in partition.datasets],
partition_count=len(partition.datasets),
product_key=product.key,
mechanism=product.mechanism,
climate_context=product.climate_context,
probability_class=product.probability_class,
return_period_years=product.return_period_years,
selection_bbox=payload.bbox,
selection_area_id=payload.area_id,
selected_cell_count=selected_cell_count,
inundated_cell_count=inundated_cell_count,
inundated_fraction=round(inundated_cell_count / max(1, selected_cell_count), 6),
resolution_m=round(max(partition.resolution_x, partition.resolution_y), 4),
summary=FloodHazardSelectionSummary(
metric_label=primary.metric_label,
metric_value=primary.metric_value,
metric_unit=primary.metric_unit,
aggregation_method=primary.aggregation_method,
primary_metric_key=primary.metric_key,
metrics=metrics,
),
unsupported_metrics=FloodHazardAnalysisService.UNSUPPORTED_METRICS,
limitation_message=(
f"{FloodHazardAnalysisService.LIMITATION} De selectie werd exact berekend over "
f"{len(partition.datasets)} persistente gemeentelijke rasterpartities."
),
generated_at=datetime.now(UTC).isoformat(),
)
return response.model_dump(mode="json")
@staticmethod @staticmethod
def render_png(db, project_id: UUID, dataset_id: UUID, *, max_dimension: int = 1800) -> bytes: def render_png(db, project_id: UUID, dataset_id: UUID, *, max_dimension: int = 1800) -> bytes:
dataset = FloodHazardAnalysisService._load_dataset(db, project_id, dataset_id) dataset = FloodHazardAnalysisService._load_dataset(db, project_id, dataset_id)
@@ -0,0 +1,200 @@
from __future__ import annotations
import math
from contextlib import ExitStack
from dataclasses import dataclass
from pathlib import Path
from typing import Any
from uuid import UUID
from pyproj import Transformer
from shapely.geometry import mapping
from shapely.ops import transform as shapely_transform
from app.core.errors import AppError
from app.models import Dataset
@dataclass(frozen=True)
class RasterPartitionSelection:
datasets: list[Dataset]
values: Any
selected_cells: Any
resolution_x: float
resolution_y: float
class RasterPartitionAnalysisService:
MAX_PARTITIONS = 64
@staticmethod
def _bbox_intersects(dataset: Dataset, bbox: tuple[float, float, float, float]) -> bool:
source_bbox = (dataset.source_metadata or {}).get("bbox_epsg4326")
if not isinstance(source_bbox, list) or len(source_bbox) != 4:
return True
try:
min_x, min_y, max_x, max_y = (float(value) for value in source_bbox)
except (TypeError, ValueError):
return True
return not (
max_x <= bbox[0]
or min_x >= bbox[2]
or max_y <= bbox[1]
or min_y >= bbox[3]
)
@staticmethod
def _candidate_datasets(
db,
project_id: UUID,
*,
source_name: str,
product_key: str,
bbox: tuple[float, float, float, float],
) -> list[Dataset]:
rows = (
db.query(Dataset)
.filter(
Dataset.project_id == project_id,
Dataset.source_name == source_name,
Dataset.dataset_type == "raster",
Dataset.status == "ready",
)
.all()
)
candidates = [
dataset
for dataset in rows
if str((dataset.source_metadata or {}).get("product_key") or "") == product_key
and dataset.storage_path
and Path(dataset.storage_path).is_file()
and RasterPartitionAnalysisService._bbox_intersects(dataset, bbox)
]
candidates.sort(key=lambda dataset: (str(dataset.area_id or ""), str(dataset.id)))
if not candidates:
raise AppError(
code="RASTER_PARTITIONS_NOT_FOUND",
message="No persisted raster partitions cover this selection",
details={"source_name": source_name, "product_key": product_key},
status_code=404,
)
if len(candidates) > RasterPartitionAnalysisService.MAX_PARTITIONS:
raise AppError(
code="RASTER_PARTITION_LIMIT_EXCEEDED",
message="The selection intersects too many raster partitions",
details={
"partition_count": len(candidates),
"max_partitions": RasterPartitionAnalysisService.MAX_PARTITIONS,
},
status_code=422,
)
return candidates
@staticmethod
def select(
db,
project_id: UUID,
*,
source_name: str,
product_key: str,
selection_geometry_4326,
nodata: float,
max_pixels: int,
) -> RasterPartitionSelection:
try:
import numpy as np
import rasterio
from rasterio.features import geometry_mask
from rasterio.merge import merge
except ImportError as exc:
raise AppError(
code="RASTER_PROCESSING_UNAVAILABLE",
message="Rasterio and numpy are required for partitioned raster analysis",
status_code=503,
) from exc
bbox = tuple(float(value) for value in selection_geometry_4326.bounds)
datasets = RasterPartitionAnalysisService._candidate_datasets(
db,
project_id,
source_name=source_name,
product_key=product_key,
bbox=bbox,
)
transformer = Transformer.from_crs("EPSG:4326", "EPSG:31370", always_xy=True)
selection_metric = shapely_transform(transformer.transform, selection_geometry_4326)
min_x, min_y, max_x, max_y = selection_metric.bounds
try:
with ExitStack() as stack:
sources = [stack.enter_context(rasterio.open(dataset.storage_path)) for dataset in datasets]
invalid_sources = [
index
for index, source in enumerate(sources)
if source.crs is None or source.crs.to_epsg() != 31370 or source.count != 1
]
if invalid_sources:
raise AppError(
code="RASTER_PARTITION_MISMATCH",
message="Raster partitions do not share the governed CRS and band layout",
details={"invalid_partition_indexes": invalid_sources},
status_code=409,
)
target_resolution = max(abs(float(sources[0].res[0])), abs(float(sources[0].res[1])))
invalid_resolutions = [
{
"partition_index": index,
"resolution": [abs(float(source.res[0])), abs(float(source.res[1]))],
}
for index, source in enumerate(sources)
if not all(
math.isclose(abs(float(value)), target_resolution, rel_tol=0.001, abs_tol=0.01)
for value in source.res
)
]
if invalid_resolutions:
raise AppError(
code="RASTER_PARTITION_MISMATCH",
message="Raster partitions do not share one analysis resolution",
details={"invalid_resolutions": invalid_resolutions},
status_code=409,
)
width = max(1, math.ceil((max_x - min_x) / target_resolution))
height = max(1, math.ceil((max_y - min_y) / target_resolution))
if width * height > max_pixels:
raise AppError(
code="RASTER_PARTITION_SELECTION_TOO_LARGE",
message="Select a smaller rectangle for regional raster analysis",
details={"pixel_count": width * height, "max_pixels": max_pixels},
status_code=422,
)
mosaic, transform = merge(
sources,
bounds=(min_x, min_y, max_x, max_y),
res=(target_resolution, target_resolution),
nodata=nodata,
dtype="float32",
)
values = np.asarray(mosaic[0], dtype="float64")
selected_cells = geometry_mask(
[mapping(selection_metric)],
out_shape=values.shape,
transform=transform,
invert=True,
)
return RasterPartitionSelection(
datasets=datasets,
values=values,
selected_cells=selected_cells,
resolution_x=target_resolution,
resolution_y=target_resolution,
)
except AppError:
raise
except Exception as exc:
raise AppError(
code="RASTER_PARTITION_ANALYSIS_FAILED",
message="Persisted raster partitions could not be assembled for this selection",
details={"reason": str(exc)},
status_code=500,
) from exc
@@ -14,8 +14,15 @@ from shapely.ops import transform as shapely_transform
from app.core.config import Settings, get_settings from app.core.config import Settings, get_settings
from app.core.errors import AppError from app.core.errors import AppError
from app.models import Area, Dataset from app.models import Area, Dataset
from app.schemas.dhmv import TerrainMetric, TerrainSelectionRequest, TerrainSelectionResponse, TerrainSelectionSummary from app.schemas.dhmv import (
TerrainMetric,
TerrainPartitionSelectionRequest,
TerrainSelectionRequest,
TerrainSelectionResponse,
TerrainSelectionSummary,
)
from app.services.dhmv_acquisition_service import DhmvAcquisitionService from app.services.dhmv_acquisition_service import DhmvAcquisitionService
from app.services.raster_partition_analysis_service import RasterPartitionAnalysisService
class TerrainAnalysisService: class TerrainAnalysisService:
@@ -177,6 +184,8 @@ class TerrainAnalysisService:
selected_cell_count = int(selected_cells.sum()) selected_cell_count = int(selected_cells.sum())
response = TerrainSelectionResponse( response = TerrainSelectionResponse(
dataset_id=dataset.id, dataset_id=dataset.id,
dataset_ids=[dataset.id],
partition_count=1,
product_key=product_key, product_key=product_key,
surface_model=surface_model, surface_model=surface_model,
selection_bbox=payload.bbox, selection_bbox=payload.bbox,
@@ -200,6 +209,139 @@ class TerrainAnalysisService:
) )
return response.model_dump(mode="json") return response.model_dump(mode="json")
@staticmethod
def analyze_partitions(
db,
project_id: UUID,
payload: TerrainPartitionSelectionRequest,
*,
settings: Settings | None = None,
) -> dict:
resolved_settings = settings or get_settings()
product = DhmvAcquisitionService._products().get(payload.product_key.strip().lower())
if product is None:
raise AppError(
code="DHMV_PRODUCT_NOT_SUPPORTED",
message="Select a governed DHMV terrain or surface product",
details={"product_key": payload.product_key},
status_code=422,
)
selection_4326 = TerrainAnalysisService._selection_geometry(db, project_id, payload)
partition = RasterPartitionAnalysisService.select(
db,
project_id,
source_name=DhmvAcquisitionService.PROVIDER,
product_key=product.key,
selection_geometry_4326=selection_4326,
nodata=DhmvAcquisitionService.NODATA,
max_pixels=resolved_settings.dhmv_max_pixels,
)
surface_models = {
str((dataset.source_metadata or {}).get("surface_model") or "")
for dataset in partition.datasets
}
if surface_models != {product.surface_model}:
raise AppError(
code="INVALID_TERRAIN_METADATA",
message="DHMV partition provenance is incomplete",
details={"surface_models": sorted(surface_models)},
status_code=409,
)
try:
import numpy as np
except ImportError as exc:
raise AppError(
code="RASTER_PROCESSING_UNAVAILABLE",
message="Numpy is required for partitioned terrain analysis",
status_code=503,
) from exc
raw = partition.values
invalid = ~np.isfinite(raw) | (raw == DhmvAcquisitionService.NODATA)
valid_mask = partition.selected_cells & ~invalid
values = raw[valid_mask]
if values.size == 0:
raise AppError(
code="TERRAIN_NO_VALID_DATA",
message="No valid DHMV height cells occur in this selection",
status_code=422,
)
slope_values = np.asarray([], dtype="float64")
if raw.shape[0] >= 2 and raw.shape[1] >= 2:
surface = np.where(valid_mask, raw, np.nan)
gradient_y, gradient_x = np.gradient(
surface,
partition.resolution_y,
partition.resolution_x,
)
slope = np.degrees(np.arctan(np.hypot(gradient_x, gradient_y)))
slope_values = slope[np.isfinite(slope) & valid_mask]
def metric(key: str, label: str, value: float, unit: str, method: str) -> TerrainMetric:
return TerrainMetric(
metric_key=key,
metric_label=label,
metric_value=round(float(value), 4),
metric_unit=unit,
aggregation_method=method,
)
prefix = "terrain" if product.surface_model == "terrain" else "surface"
elevation_label = (
"Gemiddelde maaiveldhoogte"
if product.surface_model == "terrain"
else "Gemiddelde oppervlaktehoogte"
)
metrics = [
metric(f"{prefix}_elevation_mean_m", elevation_label, values.mean(), "m TAW", "mean_valid_cells"),
metric(f"{prefix}_elevation_min_m", "Laagste hoogte", values.min(), "m TAW", "minimum_valid_cells"),
metric(f"{prefix}_elevation_max_m", "Hoogste hoogte", values.max(), "m TAW", "maximum_valid_cells"),
metric(f"{prefix}_elevation_p10_m", "10e percentiel hoogte", np.percentile(values, 10), "m TAW", "percentile_10_valid_cells"),
metric(f"{prefix}_elevation_p90_m", "90e percentiel hoogte", np.percentile(values, 90), "m TAW", "percentile_90_valid_cells"),
metric("relief_m", "Reliëfverschil", values.max() - values.min(), "m", "maximum_minus_minimum"),
]
if slope_values.size:
metrics.extend(
[
metric("slope_mean_deg", "Gemiddelde helling", slope_values.mean(), "°", "mean_finite_gradient"),
metric("slope_p90_deg", "90e percentiel helling", np.percentile(slope_values, 90), "°", "percentile_90_finite_gradient"),
metric("slope_max_deg", "Steilste helling", slope_values.max(), "°", "maximum_finite_gradient"),
]
)
primary = metrics[0]
selected_cell_count = int(partition.selected_cells.sum())
first_dataset = partition.datasets[0]
response = TerrainSelectionResponse(
dataset_id=first_dataset.id,
dataset_ids=[dataset.id for dataset in partition.datasets],
partition_count=len(partition.datasets),
product_key=product.key,
surface_model=product.surface_model,
selection_bbox=payload.bbox,
selection_area_id=payload.area_id,
sample_count=int(values.size),
slope_sample_count=int(slope_values.size),
coverage_ratio=round(float(values.size / max(1, selected_cell_count)), 6),
resolution_m=round(max(partition.resolution_x, partition.resolution_y), 4),
vertical_reference=DhmvAcquisitionService.VERTICAL_REFERENCE,
summary=TerrainSelectionSummary(
metric_label=primary.metric_label,
metric_value=primary.metric_value,
metric_unit=primary.metric_unit,
aggregation_method=primary.aggregation_method,
primary_metric_key=primary.metric_key,
metrics=metrics,
),
unsupported_metrics=TerrainAnalysisService.UNSUPPORTED_METRICS,
limitation_message=(
f"{TerrainAnalysisService.LIMITATION} De selectie werd exact berekend over "
f"{len(partition.datasets)} persistente gemeentelijke rasterpartities."
),
generated_at=datetime.now(UTC).isoformat(),
)
return response.model_dump(mode="json")
@staticmethod @staticmethod
def render_png(db, project_id: UUID, dataset_id: UUID, *, max_dimension: int = 1800) -> bytes: def render_png(db, project_id: UUID, dataset_id: UUID, *, max_dimension: int = 1800) -> bytes:
dataset = TerrainAnalysisService._load_dataset(db, project_id, dataset_id) dataset = TerrainAnalysisService._load_dataset(db, project_id, dataset_id)
+93 -1
View File
@@ -18,7 +18,7 @@ from app.core.errors import AppError
from app.db.session import get_db from app.db.session import get_db
from app.main import app from app.main import app
from app.models import Area, Dataset, DatasetVersion, Job, Project from app.models import Area, Dataset, DatasetVersion, Job, Project
from app.schemas.dhmv import DhmvAcquireRequest, TerrainSelectionRequest from app.schemas.dhmv import DhmvAcquireRequest, TerrainPartitionSelectionRequest, TerrainSelectionRequest
from app.services.dhmv_acquisition_service import DhmvAcquisitionService from app.services.dhmv_acquisition_service import DhmvAcquisitionService
from app.services.terrain_analysis_service import TerrainAnalysisService from app.services.terrain_analysis_service import TerrainAnalysisService
@@ -39,6 +39,9 @@ class FakeQuery:
def first(self): def first(self):
return self.result return self.result
def all(self):
return self.result if isinstance(self.result, list) else []
class FakeSession: class FakeSession:
def __init__(self, rows=None, query_result=None): def __init__(self, rows=None, query_result=None):
@@ -115,6 +118,23 @@ def elevation_tiff(*, left: float, top: float, width: int, height: int, resoluti
return memory.read() return memory.read()
def constant_elevation_tiff(*, left: float, top: float, value: float) -> bytes:
values = np.full((20, 20), value, dtype="float32")
with MemoryFile() as memory:
with memory.open(
driver="GTiff",
width=20,
height=20,
count=1,
dtype="float32",
crs="EPSG:31370",
transform=from_origin(left, top, 5.0, 5.0),
nodata=-9999.0,
) as output:
output.write(values, 1)
return memory.read()
def edge_elevation_tiff(*, left: float, top: float, x_resolution: float, y_resolution: float = 5.0) -> bytes: def edge_elevation_tiff(*, left: float, top: float, x_resolution: float, y_resolution: float = 5.0) -> bytes:
rows, columns = np.indices((20, 20)) rows, columns = np.indices((20, 20))
values = (20.0 + columns * 0.5 + rows).astype("float32") values = (20.0 + columns * 0.5 + rows).astype("float32")
@@ -359,6 +379,61 @@ def test_terrain_analysis_returns_governed_elevation_relief_and_slope(tmp_path)
assert "Waterdiepte" in result["limitation_message"] assert "Waterdiepte" in result["limitation_message"]
def test_partitioned_terrain_analysis_is_exact_across_municipality_boundaries(tmp_path) -> None:
project_id = uuid4()
transformer = Transformer.from_crs("EPSG:31370", "EPSG:4326", always_xy=True)
min_x, min_y = transformer.transform(200_000, 210_000)
middle_x, _ = transformer.transform(200_100, 210_000)
max_x, max_y = transformer.transform(200_200, 210_100)
paths = [tmp_path / "left-terrain.tif", tmp_path / "right-terrain.tif"]
paths[0].write_bytes(constant_elevation_tiff(left=200_000, top=210_100, value=10.0))
paths[1].write_bytes(constant_elevation_tiff(left=200_100, top=210_100, value=20.0))
datasets = [
Dataset(
id=uuid4(),
project_id=project_id,
area_id=uuid4(),
name=path.name,
dataset_type="raster",
source="official WCS",
source_name="digitaal_vlaanderen_dhmv",
source_metadata={
"product_key": "dtm_1m",
"surface_model": "terrain",
"bbox_epsg4326": [left, min_y, right, max_y],
},
status="ready",
storage_path=str(path),
)
for path, left, right in (
(paths[0], min_x, middle_x),
(paths[1], middle_x, max_x),
)
]
db = FakeSession(query_result=datasets)
payload = TerrainPartitionSelectionRequest(
bbox={"min_x": min_x, "min_y": min_y, "max_x": max_x, "max_y": max_y, "crs": "EPSG:4326"},
product_key="dtm_1m",
)
result = TerrainAnalysisService.analyze_partitions(
db,
project_id,
payload,
settings=Settings(_env_file=None),
)
metrics = {item["metric_key"]: item["metric_value"] for item in result["summary"]["metrics"]}
assert result["partition_count"] == 2
assert set(result["dataset_ids"]) == {str(dataset.id) for dataset in datasets}
assert result["sample_count"] >= 790
assert metrics["terrain_elevation_mean_m"] == pytest.approx(15.0, abs=0.1)
assert metrics["terrain_elevation_min_m"] == 10.0
assert metrics["terrain_elevation_max_m"] == 20.0
assert metrics["terrain_elevation_p90_m"] == 20.0
assert "2 persistente gemeentelijke rasterpartities" in result["limitation_message"]
def test_terrain_analysis_rejects_non_dhmv_raster(tmp_path) -> None: def test_terrain_analysis_rejects_non_dhmv_raster(tmp_path) -> None:
project_id = uuid4() project_id = uuid4()
dataset_id = uuid4() dataset_id = uuid4()
@@ -425,6 +500,16 @@ def test_dhmv_endpoints_use_canonical_envelopes(monkeypatch) -> None:
"unsupported_metrics": ["water_depth_m", "water_volume_m3"], "unsupported_metrics": ["water_depth_m", "water_volume_m3"],
}, },
) )
monkeypatch.setattr(
TerrainAnalysisService,
"analyze_partitions",
lambda *_args, **_kwargs: {
"dataset_id": str(output_dataset_id),
"dataset_ids": [str(output_dataset_id)],
"partition_count": 1,
"sample_count": 100,
},
)
app.dependency_overrides[get_db] = lambda: db app.dependency_overrides[get_db] = lambda: db
try: try:
products = TestClient(app).get(f"/api/v1/projects/{project_id}/datasets/dhmv/products") products = TestClient(app).get(f"/api/v1/projects/{project_id}/datasets/dhmv/products")
@@ -436,6 +521,10 @@ def test_dhmv_endpoints_use_canonical_envelopes(monkeypatch) -> None:
f"/api/v1/projects/{project_id}/datasets/{output_dataset_id}/raster/terrain/select", f"/api/v1/projects/{project_id}/datasets/{output_dataset_id}/raster/terrain/select",
json={"bbox": lambert_bbox_payload().bbox.model_dump()}, json={"bbox": lambert_bbox_payload().bbox.model_dump()},
) )
regional_terrain = TestClient(app).post(
f"/api/v1/projects/{project_id}/datasets/raster/terrain/select",
json={"bbox": lambert_bbox_payload().bbox.model_dump(), "product_key": "dtm_1m"},
)
finally: finally:
app.dependency_overrides.clear() app.dependency_overrides.clear()
@@ -450,6 +539,9 @@ def test_dhmv_endpoints_use_canonical_envelopes(monkeypatch) -> None:
assert set(terrain.json()) == {"data"} assert set(terrain.json()) == {"data"}
assert terrain.json()["data"]["sample_count"] == 100 assert terrain.json()["data"]["sample_count"] == 100
assert terrain.json()["data"]["unsupported_metrics"] == ["water_depth_m", "water_volume_m3"] assert terrain.json()["data"]["unsupported_metrics"] == ["water_depth_m", "water_volume_m3"]
assert regional_terrain.status_code == 200
assert set(regional_terrain.json()) == {"data"}
assert regional_terrain.json()["data"]["partition_count"] == 1
assert any(isinstance(item, Job) for item in db.added) assert any(isinstance(item, Job) for item in db.added)
@@ -16,7 +16,11 @@ from app.core.errors import AppError
from app.db.session import get_db from app.db.session import get_db
from app.main import app from app.main import app
from app.models import Dataset, Job, Project from app.models import Dataset, Job, Project
from app.schemas.flood_hazard import FloodHazardAcquireRequest, FloodHazardSelectionRequest from app.schemas.flood_hazard import (
FloodHazardAcquireRequest,
FloodHazardPartitionSelectionRequest,
FloodHazardSelectionRequest,
)
from app.schemas.assistant import AssistantQueryRequest from app.schemas.assistant import AssistantQueryRequest
from app.services.geo_assistant_service import GeoAssistantService from app.services.geo_assistant_service import GeoAssistantService
from app.services.flood_hazard_acquisition_service import FloodHazardAcquisitionService from app.services.flood_hazard_acquisition_service import FloodHazardAcquisitionService
@@ -120,6 +124,23 @@ def edge_depth_tiff(*, left: float, top: float, x_resolution: float, y_resolutio
return memory.read() return memory.read()
def normalized_depth_tiff(*, left: float, top: float, value: float) -> bytes:
values = np.full((20, 20), value, dtype="float32")
with MemoryFile() as memory:
with memory.open(
driver="GTiff",
width=20,
height=20,
count=1,
dtype="float32",
crs="EPSG:31370",
transform=from_origin(left, top, 5.0, 5.0),
nodata=-9999.0,
) as output:
output.write(values, 1)
return memory.read()
def test_flood_hazard_registry_is_complete_and_semantically_honest() -> None: def test_flood_hazard_registry_is_complete_and_semantically_honest() -> None:
products = FloodHazardAcquisitionService.list_products() products = FloodHazardAcquisitionService.list_products()
@@ -274,6 +295,61 @@ def test_flood_hazard_analysis_reports_scenario_metrics_without_claiming_waterbo
assert "geen gelijktijdig" in result["limitation_message"] assert "geen gelijktijdig" in result["limitation_message"]
def test_partitioned_flood_analysis_is_exact_across_municipality_boundaries(tmp_path) -> None:
project_id = uuid4()
transformer = Transformer.from_crs("EPSG:31370", "EPSG:4326", always_xy=True)
min_x, min_y = transformer.transform(200_000, 210_000)
middle_x, _ = transformer.transform(200_100, 210_000)
max_x, max_y = transformer.transform(200_200, 210_100)
paths = [tmp_path / "left-flood.tif", tmp_path / "right-flood.tif"]
paths[0].write_bytes(normalized_depth_tiff(left=200_000, top=210_100, value=1.0))
paths[1].write_bytes(normalized_depth_tiff(left=200_100, top=210_100, value=2.0))
datasets = [
Dataset(
id=uuid4(),
project_id=project_id,
area_id=uuid4(),
name=path.name,
dataset_type="raster",
source="VMM",
source_name=FloodHazardAcquisitionService.PROVIDER,
source_metadata={
"product_key": "pluviaal_current_t100",
"normalized_value_unit": "m",
"bbox_epsg4326": [left, min_y, right, max_y],
},
status="ready",
storage_path=str(path),
)
for path, left, right in (
(paths[0], min_x, middle_x),
(paths[1], middle_x, max_x),
)
]
db = FakeSession(query_result=datasets)
payload = FloodHazardPartitionSelectionRequest(
bbox={"min_x": min_x, "min_y": min_y, "max_x": max_x, "max_y": max_y, "crs": "EPSG:4326"},
product_key="pluviaal_current_t100",
)
result = FloodHazardAnalysisService.analyze_partitions(
db,
project_id,
payload,
settings=Settings(_env_file=None),
)
metrics = {item["metric_key"]: item["metric_value"] for item in result["summary"]["metrics"]}
assert result["partition_count"] == 2
assert set(result["dataset_ids"]) == {str(dataset.id) for dataset in datasets}
assert result["inundated_cell_count"] >= 790
assert result["inundated_fraction"] == pytest.approx(1.0)
assert metrics["modelled_depth_mean_m"] == pytest.approx(1.5, abs=0.01)
assert metrics["modelled_depth_p90_m"] == 2.0
assert metrics["modelled_inundated_area_ha"] == pytest.approx(2.0, abs=0.03)
assert "2 persistente gemeentelijke rasterpartities" in result["limitation_message"]
def test_flood_hazard_renderer_returns_transparent_png(tmp_path) -> None: def test_flood_hazard_renderer_returns_transparent_png(tmp_path) -> None:
project_id = uuid4() project_id = uuid4()
dataset_id = uuid4() dataset_id = uuid4()
@@ -314,6 +390,16 @@ def test_flood_hazard_api_uses_canonical_envelopes(monkeypatch) -> None:
"unsupported_metrics": ["permanent_water_volume_m3"], "unsupported_metrics": ["permanent_water_volume_m3"],
}, },
) )
monkeypatch.setattr(
FloodHazardAnalysisService,
"analyze_partitions",
lambda *_args, **_kwargs: {
"dataset_id": str(output_dataset_id),
"dataset_ids": [str(output_dataset_id)],
"partition_count": 1,
"inundated_cell_count": 4,
},
)
app.dependency_overrides[get_db] = lambda: db app.dependency_overrides[get_db] = lambda: db
try: try:
client = TestClient(app) client = TestClient(app)
@@ -326,6 +412,10 @@ def test_flood_hazard_api_uses_canonical_envelopes(monkeypatch) -> None:
f"/api/v1/projects/{project_id}/datasets/{output_dataset_id}/raster/flood-hazard/select", f"/api/v1/projects/{project_id}/datasets/{output_dataset_id}/raster/flood-hazard/select",
json={"bbox": flood_payload().bbox.model_dump()}, json={"bbox": flood_payload().bbox.model_dump()},
) )
regional_selection = client.post(
f"/api/v1/projects/{project_id}/datasets/raster/flood-hazard/select",
json={"bbox": flood_payload().bbox.model_dump(), "product_key": "pluviaal_current_t100"},
)
finally: finally:
app.dependency_overrides.clear() app.dependency_overrides.clear()
@@ -334,6 +424,8 @@ def test_flood_hazard_api_uses_canonical_envelopes(monkeypatch) -> None:
assert acquisition.status_code == 200 and set(acquisition.json()) == {"data"} assert acquisition.status_code == 200 and set(acquisition.json()) == {"data"}
assert acquisition.json()["data"]["job_type"] == "raster.flood_hazard.acquire" assert acquisition.json()["data"]["job_type"] == "raster.flood_hazard.acquire"
assert selection.status_code == 200 and set(selection.json()) == {"data"} assert selection.status_code == 200 and set(selection.json()) == {"data"}
assert regional_selection.status_code == 200 and set(regional_selection.json()) == {"data"}
assert regional_selection.json()["data"]["partition_count"] == 1
assert any(isinstance(item, Job) for item in db.added) assert any(isinstance(item, Job) for item in db.added)
@@ -0,0 +1,52 @@
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
def test_partitioned_raster_routes_are_canonical_and_documented() -> None:
routes = (ROOT / "backend/app/api/routes/datasets.py").read_text(encoding="utf-8")
contracts = (ROOT / "docs/API_CONTRACTS.md").read_text(encoding="utf-8")
for path in (
"/datasets/raster/terrain/select",
"/datasets/raster/flood-hazard/select",
):
assert f'@router.post("{path}", response_model=dict)' in routes
assert path in contracts
assert "envelope(TerrainAnalysisService.analyze_partitions" in routes
assert "envelope(FloodHazardAnalysisService.analyze_partitions" in routes
def test_regional_map_uses_logical_partition_groups_and_exact_analysis() -> None:
workspace = (ROOT / "frontend/src/components/map/MapWorkspace.tsx").read_text(encoding="utf-8")
hook = (ROOT / "frontend/src/hooks/useMapThemeSelectionInsights.ts").read_text(encoding="utf-8")
api = (ROOT / "frontend/src/services/api/datasets.ts").read_text(encoding="utf-8")
assert "regionalScopeSelected" in workspace
assert "rasterPartitionsForDataset" in workspace
assert "imageOverlays={activeImageOverlays}" in workspace
assert "de juiste gemeentelijke rasters worden automatisch gecombineerd" in workspace
assert "selectTerrainPartitions" in hook
assert "selectFloodHazardPartitions" in hook
assert "/datasets/raster/terrain/select" in api
assert "/datasets/raster/flood-hazard/select" in api
def test_maplibre_supports_multiple_persisted_raster_overlays() -> None:
map_source = (ROOT / "frontend/src/components/GeoMap.tsx").read_text(encoding="utf-8")
assert "imageOverlays?: MapImageOverlay[]" in map_source
assert "imageOverlayIdsRef" in map_source
assert "imageOverlays.forEach" in map_source
assert "bounded-raster-" in map_source
def test_regional_analysis_does_not_create_an_authoritative_mosaic() -> None:
service = (ROOT / "backend/app/services/raster_partition_analysis_service.py").read_text(encoding="utf-8")
storage = (ROOT / "docs/STORAGE_ARCHITECTURE.md").read_text(encoding="utf-8")
assert "from rasterio.merge import merge" in service
assert "DatasetService" not in service
assert "12-million-cell limit" in storage
assert "does not create another authoritative raster" in storage
+21
View File
@@ -257,6 +257,17 @@ in degrees. Area geometry is an exact mask, not only a bounding box.
The response always lists `water_depth_m` and `water_volume_m3` under The response always lists `water_depth_m` and `water_volume_m3` under
`unsupported_metrics`. Drainage is not calculated by this endpoint. `unsupported_metrics`. Drainage is not calculated by this endpoint.
### POST `/api/v1/projects/{project_id}/datasets/raster/terrain/select`
Runs the same exact terrain calculation over every persisted municipal DHMV
partition intersecting one bounded EPSG:4326 rectangle. The request adds the
governed `product_key` (`dtm_1m` or `dsm_1m`) to the ordinary selection bbox
and optional Area id. The backend mosaics only the intersecting windows in
EPSG:31370, enforces the existing 12-million-cell limit and calculates global
cell statistics. The canonical response includes `dataset_ids` and
`partition_count`; percentiles are calculated from the combined cells and are
not averages of municipal summaries.
### GET `/api/v1/projects/{project_id}/datasets/{dataset_id}/raster/terrain/image` ### GET `/api/v1/projects/{project_id}/datasets/{dataset_id}/raster/terrain/image`
Returns a browser-safe PNG colour relief for the persisted governed DHMV Returns a browser-safe PNG colour relief for the persisted governed DHMV
@@ -316,6 +327,16 @@ it is explicitly not concurrent flood storage, permanent waterbody content,
current water level or bathymetry. These unsupported metrics remain listed in current water level or bathymetry. These unsupported metrics remain listed in
the response. the response.
### POST `/api/v1/projects/{project_id}/datasets/raster/flood-hazard/select`
Runs exact bounded analysis over the persisted municipal VMM partitions for
one governed `product_key`. Only partitions intersecting the selection are
opened, the normalized metre grids are combined at their common 5 m analysis
resolution and the global area/depth metrics are calculated from the combined
cells. The canonical response includes every contributing Dataset id in
`dataset_ids` plus `partition_count`. The existing flood-volume and bathymetry
prohibitions are unchanged.
### GET `/api/v1/projects/{project_id}/datasets/{dataset_id}/raster/flood-hazard/image` ### GET `/api/v1/projects/{project_id}/datasets/{dataset_id}/raster/flood-hazard/image`
Returns a constrained transparent PNG for a persisted governed VMM flood-depth Returns a constrained transparent PNG for a persisted governed VMM flood-depth
+49
View File
@@ -1,3 +1,52 @@
## Sprint 218 Regional terrain and flood completion (2026-07-16)
Changed:
- Added the resumable 28-municipality DHMV DTM/DSM operator and executed all
56 governed acquisitions through the canonical API.
- Executed all 336 governed VMM mechanism/climate/return-period combinations
across the same 28 persisted municipality Areas.
- Diagnosed Retie's official WCS integer-grid edge rounding in both providers.
Accepted only bounded 5%/0.25 m edge drift, harmonized accepted tiles to the
exact requested grid and retained source resolutions, tile indexes and
method in provenance. A 4.5 m regression fixture remains rejected.
- Added exact partitioned terrain and flood selection routes for bounded
rectangles that cross municipality boundaries. They use only persisted
GeoTIFFs, calculate global statistics from combined cells and retain the
contributing Dataset ids in the response.
- Made the complete Kempen Area expose DHMV/VMM as logical regional MapLibre
layers while retaining municipality-linked storage and twelve distinct VMM
scenario identities.
Live evidence:
- VMM: 336 ready Datasets, 28 Areas, 12 products, 336 DatasetVersions,
165,277,992 bytes, zero duplicate Area/product pairs and zero missing or
size-mismatched files.
- DHMV: 56 ready Datasets, 28 Areas, two products, 56 DatasetVersions,
282,645,991 bytes, zero duplicate Area/product pairs and zero missing or
size-mismatched files.
- Every VMM Dataset has null `observed_at` and explicit false flags for
bathymetry, permanent depth/volume and concurrent volume. A repeated Retie
run reused all twelve immutable Dataset ids.
- Tower commit `153cff0` passed container health, PostGIS 3.6, Alembic head
`202607160001` and frontend/API/icon proxy checks before the regional UI
follow-up.
Validation evidence:
- Focused DHMV/VMM and regional explorer backend suites passed, including
exact adjacent-partition percentile and area calculations.
- The pre-UI-fix release gate passed 746 tests, backend compilation, API
contract checks, one Alembic head, frontend typecheck and production build.
- The final local release gate passed all 752 backend tests, backend
compilation, 107 documented API route checks, Alembic head `202607160001`,
frontend typecheck and the production build. Live deployment and browser
evidence are recorded after the Tower rollout.
Next:
- Use the now-complete regional current-state layers as the baseline for a
governed refresh/change scheduler. Keep source-specific publication dates
and scenario semantics; do not turn VMM scenarios into a historical water
level series.
## Sprint 217 Regional DOV soil coverage (2026-07-16) ## Sprint 217 Regional DOV soil coverage (2026-07-16)
Changed: Changed:
+17 -11
View File
@@ -387,7 +387,7 @@ quality metrics.
- Cache: canonical raster Dataset plus WCS request/response/output checksums - Cache: canonical raster Dataset plus WCS request/response/output checksums
- Operators: `scripts/provision_mol_dhmv.py`, - Operators: `scripts/provision_mol_dhmv.py`,
`scripts/provision_regional_dhmv.py` `scripts/provision_regional_dhmv.py`
- Prioriteit: P4 uitgevoerd voor Mol en operationeel regionaal uitbreidbaar - Prioriteit: P4 uitgevoerd voor alle 28 Kempen-gemeenten
The operator requests a bounded 5 m analysis copy by default so a complete The operator requests a bounded 5 m analysis copy by default so a complete
municipality remains operationally manageable while retaining the official municipality remains operationally manageable while retaining the official
@@ -401,7 +401,10 @@ De regionale operator gebruikt exact de 28 persistente gemeente-Areas van de
goedgekeurde Kempen-scope en plant twee outputs per gemeente. Die 56 goedgekeurde Kempen-scope en plant twee outputs per gemeente. Die 56
gemeentepartities vermijden een onnodig monolithisch hoogtebestand, blijven gemeentepartities vermijden een onnodig monolithisch hoogtebestand, blijven
binnen WCS/pixelgrenzen en sluiten aan op de gebiedsgebonden datasetselectie in binnen WCS/pixelgrenzen en sluiten aan op de gebiedsgebonden datasetselectie in
de kaart. Herhaalruns gebruiken de bestaande checksummed requestcache. de kaart. De live matrix bevat 56 geverifieerde Datasets en DatasetVersions.
De regionale kaart combineert alleen de partities die een getekende selectie
raken; globale statistieken worden uit de samengevoegde cellen berekend.
Herhaalruns gebruiken de bestaande checksummed requestcache.
## VMM overstromingsgevaarkaarten ## VMM overstromingsgevaarkaarten
@@ -415,8 +418,9 @@ de kaart. Herhaalruns gebruiken de bestaande checksummed requestcache.
- Publicatie: 2019/2021 afhankelijk van product; scenario-identiteit is - Publicatie: 2019/2021 afhankelijk van product; scenario-identiteit is
leidend en wordt niet als observatiedatum opgeslagen leidend en wordt niet als observatiedatum opgeslagen
- Cache: canonical raster Dataset plus request/response/output checksums - Cache: canonical raster Dataset plus request/response/output checksums
- Operator: `scripts/provision_mol_flood_hazards.py` - Operators: `scripts/provision_mol_flood_hazards.py`,
- Prioriteit: P5 scenariofundament uitgevoerd voor Mol `scripts/provision_regional_flood_hazards.py`
- Prioriteit: P5 scenariofundament uitgevoerd voor alle 28 Kempen-gemeenten
VMM beschrijft deze lagen als maximale lokale waterdiepte tussen wateroppervlak VMM beschrijft deze lagen als maximale lokale waterdiepte tussen wateroppervlak
en maaiveld voor een gekozen kans- en klimaatscenario. GeoIntel converteert en maaiveld voor een gekozen kans- en klimaatscenario. GeoIntel converteert
@@ -441,15 +445,17 @@ scenario uitsluitend de bestaande canonical API aan:
- `POST /api/v1/projects/{project_id}/datasets/{dataset_id}/raster/flood-hazard/select` - `POST /api/v1/projects/{project_id}/datasets/{dataset_id}/raster/flood-hazard/select`
Deze gemeentepartities zijn bewust. Een volledig regionaal raster in een Deze gemeentepartities zijn bewust. Een volledig regionaal raster in een
aanvraag zou de publieke WCS- en pixelgrenzen onnodig belasten. In de UI wordt aanvraag zou de publieke WCS- en pixelgrenzen onnodig belasten. In de UI blijft
een VMM-dataset alleen als overstromingslaag getoond voor het actieve een gemeente gekoppeld aan haar eigen bestand. Op het volledige Kempen-gebied
werkgebied waaraan die dataset gekoppeld is. Zo blijft Mol bij Mol, Geel bij worden de 28 partities als één logische scenario-laag getoond. Een getekende
Geel, enzovoort. rechthoek opent alleen de rakende partities en berekent globale statistieken
uit de werkelijk samengevoegde cellen.
De regionale operator ondersteunt `--dry-run`, `--members` en `--products`. De regionale operator ondersteunt `--dry-run`, `--members` en `--products`.
Een volledige scope met alle twaalf scenario's plant 336 gecontroleerde De uitgevoerde volledige scope bevat 336 gecontroleerde Datasets en 336
acquisities. Herhaalruns gebruiken bestaande checksummed Datasets via de DatasetVersions zonder ontbrekende gemeente/scenario-combinaties. Herhaalruns
backend-cache zolang de requestidentiteit niet verandert. gebruiken bestaande checksummed Datasets via de backend-cache zolang de
requestidentiteit niet verandert.
Ook regionaal blijft de semantiek onveranderd: VMM-waterdiepte is een Ook regionaal blijft de semantiek onveranderd: VMM-waterdiepte is een
gemodelleerde maximale lokale diepte per kans- en klimaatscenario. GeoIntel kan gemodelleerde maximale lokale diepte per kans- en klimaatscenario. GeoIntel kan
+8
View File
@@ -134,6 +134,14 @@ URLs, response/coverage/normalized checksums, EPSG:31370 bounds, scenario
metadata and explicit unsupported-volume flags. Repeat runs reuse matching metadata and explicit unsupported-volume flags. Repeat runs reuse matching
ready Datasets through the acquisition service cache. ready Datasets through the acquisition service cache.
The regional Map workspace does not create another authoritative raster or
copy pixels into PostgreSQL. Its partition-selection endpoints read only the
municipal GeoTIFF windows intersecting a bounded selection, mosaic those
windows in memory at the governed analysis resolution and return metrics plus
the complete contributing `dataset_ids`. The 12-million-cell limit applies to
the combined window. Persisted files, checksums and DatasetVersions remain the
only authoritative artifacts.
BWK/Natura 2000 evidence lives under BWK/Natura 2000 evidence lives under
`storage/operator-evidence/bwk-natura2000-2025/mol/`. The `raw/` directory `storage/operator-evidence/bwk-natura2000-2025/mol/`. The `raw/` directory
contains immutable WFS pages; the adjacent manifest records their URLs, contains immutable WFS pages; the adjacent manifest records their URLs,
+2 -1
View File
@@ -36,7 +36,8 @@
- [x] Persist one bounded whole-region snapshot per thematic raster so drawn selections can cross municipality boundaries without changing source semantics. - [x] Persist one bounded whole-region snapshot per thematic raster so drawn selections can cross municipality boundaries without changing source semantics.
- [x] Generalize the DOV soil-map operator to all 28 approved Kempen municipality partitions with one regional snapshot manifest. - [x] Generalize the DOV soil-map operator to all 28 approved Kempen municipality partitions with one regional snapshot manifest.
- [x] Add a resumable regional DHMV DTM/DSM operator for all 28 approved Kempen municipality Areas. - [x] Add a resumable regional DHMV DTM/DSM operator for all 28 approved Kempen municipality Areas.
- [ ] Execute and audit the complete 336-product VMM and 56-product DHMV regional runtime matrices. - [x] Execute and audit the complete 336-product VMM and 56-product DHMV regional runtime matrices.
- [x] Present municipal DHMV/VMM partitions as logical regional layers and analyse cross-boundary rectangles without a municipality prerequisite.
## Governed source expansion backlog ## Governed source expansion backlog
+8 -6
View File
@@ -519,12 +519,14 @@ GeoJSON export stays disabled. The UI never labels the maximum-depth area
integral as current, permanent or concurrent water volume. integral as current, permanent or concurrent water volume.
Regional VMM provisioning creates one scenario raster per municipality Area. Regional VMM provisioning creates one scenario raster per municipality Area.
The explorer therefore shows only the flood scenarios whose `area_id` matches For a municipality the explorer still uses only that exact Area-linked file.
the active work area. This avoids presenting a Mol scenario while the map is For the complete Kempen Area it presents the 28 VMM and DHMV partitions as one
focused on another municipality. The region-wide Area remains the navigation logical map layer, deduplicates VMM into twelve scenario choices and renders
context; municipality Areas are the analysis scope for flood rasters because every matching MapLibre image partition. A drawn rectangle is sent to the
the public WCS and raster cell limits make one monolithic Kempen raster partition endpoint, which opens only intersecting files and calculates exact
operationally unsafe. combined cell statistics. A monolithic full-region 5 m calculation remains
disabled because it exceeds the governed raster-cell limit; users draw a
bounded rectangle without first choosing a municipality.
## Useful repository scripts ## Useful repository scripts
+20 -18
View File
@@ -13,7 +13,7 @@ interface GeoMapProps {
selectedFeature?: GeoJSON.Feature | null selectedFeature?: GeoJSON.Feature | null
selectionData?: GeoJSON.FeatureCollection | null selectionData?: GeoJSON.FeatureCollection | null
qaEvidenceData?: GeoJSON.FeatureCollection | null qaEvidenceData?: GeoJSON.FeatureCollection | null
imageOverlay?: MapImageOverlay | null imageOverlays?: MapImageOverlay[]
selectionBbox?: { min_x: number; min_y: number; max_x: number; max_y: number } | null selectionBbox?: { min_x: number; min_y: number; max_x: number; max_y: number } | null
bboxSelectionMode?: boolean bboxSelectionMode?: boolean
visible?: boolean visible?: boolean
@@ -154,7 +154,7 @@ function GeoMap({
selectedFeature = null, selectedFeature = null,
selectionData = null, selectionData = null,
qaEvidenceData = null, qaEvidenceData = null,
imageOverlay = null, imageOverlays = [],
selectionBbox = null, selectionBbox = null,
bboxSelectionMode = false, bboxSelectionMode = false,
visible = true, visible = true,
@@ -180,6 +180,7 @@ function GeoMap({
const dataRef = useRef<GeoJSON.FeatureCollection | null>(data) const dataRef = useRef<GeoJSON.FeatureCollection | null>(data)
const fitDataOnChangeRef = useRef(fitDataOnChange) const fitDataOnChangeRef = useRef(fitDataOnChange)
const lastFittedAreaRef = useRef<GeoJSON.FeatureCollection | null>(null) const lastFittedAreaRef = useRef<GeoJSON.FeatureCollection | null>(null)
const imageOverlayIdsRef = useRef<string[]>([])
const [mapStyleReady, setMapStyleReady] = useState(false) const [mapStyleReady, setMapStyleReady] = useState(false)
areaDataRef.current = areaData areaDataRef.current = areaData
@@ -353,17 +354,20 @@ function GeoMap({
if (!map || !mapStyleReady || !map.isStyleLoaded()) { if (!map || !mapStyleReady || !map.isStyleLoaded()) {
return return
} }
if (map.getLayer('bounded-orthophoto')) { for (const overlayId of [...imageOverlayIdsRef.current].reverse()) {
map.removeLayer('bounded-orthophoto') if (map.getLayer(overlayId)) {
map.removeLayer(overlayId)
} }
if (map.getSource('bounded-orthophoto')) { if (map.getSource(overlayId)) {
map.removeSource('bounded-orthophoto') map.removeSource(overlayId)
} }
if (!imageOverlay) {
return
} }
imageOverlayIdsRef.current = []
const beforeLayer = ['area-fill', 'dataset-fill', 'selection-bbox-fill'].find((layerId) => map.getLayer(layerId))
imageOverlays.forEach((imageOverlay, index) => {
const overlayId = `bounded-raster-${index}`
const [minX, minY, maxX, maxY] = imageOverlay.bbox const [minX, minY, maxX, maxY] = imageOverlay.bbox
map.addSource('bounded-orthophoto', { map.addSource(overlayId, {
type: 'image', type: 'image',
url: imageOverlay.url, url: imageOverlay.url,
coordinates: [ coordinates: [
@@ -373,17 +377,15 @@ function GeoMap({
[minX, minY], [minX, minY],
], ],
}) })
const beforeLayer = ['area-fill', 'dataset-fill', 'selection-bbox-fill'].find((layerId) => map.getLayer(layerId)) map.addLayer({
map.addLayer( id: overlayId,
{
id: 'bounded-orthophoto',
type: 'raster', type: 'raster',
source: 'bounded-orthophoto', source: overlayId,
paint: { 'raster-opacity': imageOverlay.opacity ?? 0.88 }, paint: { 'raster-opacity': imageOverlay.opacity ?? 0.88 },
}, }, beforeLayer)
beforeLayer, imageOverlayIdsRef.current.push(overlayId)
) })
}, [imageOverlay, mapStyleReady]) }, [imageOverlays, mapStyleReady])
useEffect(() => { useEffect(() => {
const map = mapRef.current const map = mapRef.current
+188 -48
View File
@@ -158,14 +158,15 @@ const DATA_THEME_MAP_STYLES: Record<DataThemeId, { fill: string; line: string }>
parcels: { fill: '#a7792f', line: '#7d571f' }, parcels: { fill: '#a7792f', line: '#7d571f' },
} }
function datasetAvailabilityLabel(dataset: DatasetCreateResponse): string { function datasetAvailabilityLabel(dataset: DatasetCreateResponse, partitionCount = 1): string {
const regionalSuffix = partitionCount > 1 ? ` · ${partitionCount} gemeenten` : ''
if (dataset.dataset_type === 'raster' && dataset.source_name === 'digitaal_vlaanderen_dhmv') { if (dataset.dataset_type === 'raster' && dataset.source_name === 'digitaal_vlaanderen_dhmv') {
const resolution = Number(dataset.source_metadata?.['analysis_resolution_m']) const resolution = Number(dataset.source_metadata?.['analysis_resolution_m'])
return `${Number.isFinite(resolution) ? `${resolution.toLocaleString('nl-BE')} m` : 'Raster'} hoogtegrid beschikbaar` return `${Number.isFinite(resolution) ? `${resolution.toLocaleString('nl-BE')} m` : 'Raster'} hoogtegrid${regionalSuffix}`
} }
if (dataset.dataset_type === 'raster' && dataset.source_name === 'vmm_flood_hazard') { if (dataset.dataset_type === 'raster' && dataset.source_name === 'vmm_flood_hazard') {
const resolution = Number(dataset.source_metadata?.['analysis_resolution_m']) const resolution = Number(dataset.source_metadata?.['analysis_resolution_m'])
return `${Number.isFinite(resolution) ? `${resolution.toLocaleString('nl-BE')} m` : 'Raster'} overstromingsscenario` return `${Number.isFinite(resolution) ? `${resolution.toLocaleString('nl-BE')} m` : 'Raster'} overstromingsscenario${regionalSuffix}`
} }
if (dataset.dataset_type === 'raster' && dataset.source_name === 'department_omgeving_thematic_raster') { if (dataset.dataset_type === 'raster' && dataset.source_name === 'department_omgeving_thematic_raster') {
const resolution = Number(dataset.source_metadata?.['analysis_resolution_m']) const resolution = Number(dataset.source_metadata?.['analysis_resolution_m'])
@@ -213,21 +214,69 @@ function datasetMatchesTheme(dataset: DatasetCreateResponse, theme: DataTheme):
return theme.tokens.some((token) => searchText.includes(token)) return theme.tokens.some((token) => searchText.includes(token))
} }
function datasetCoversSelectedArea(dataset: DatasetCreateResponse, selectedAreaId: string | null): boolean { function isMunicipalityAreaName(name: string | null | undefined): boolean {
return /^Gemeente\s/i.test(name ?? '')
}
function isPartitionedRaster(dataset: DatasetCreateResponse | null | undefined): boolean {
return Boolean(
dataset?.dataset_type === 'raster'
&& ['digitaal_vlaanderen_dhmv', 'vmm_flood_hazard'].includes(dataset.source_name ?? ''),
)
}
function datasetProductKey(dataset: DatasetCreateResponse): string {
return String(dataset.source_metadata?.['product_key'] ?? '')
}
function datasetCoversSelectedArea(
dataset: DatasetCreateResponse,
selectedAreaId: string | null,
regionalScope = false,
): boolean {
const coverageScope = String(dataset.source_metadata?.['coverage_scope'] ?? '') const coverageScope = String(dataset.source_metadata?.['coverage_scope'] ?? '')
if (coverageScope !== 'municipality' || !dataset.area_id) { if (coverageScope !== 'municipality' || !dataset.area_id) {
return true return true
} }
if (regionalScope) {
return true
}
return Boolean(selectedAreaId) && dataset.area_id === selectedAreaId return Boolean(selectedAreaId) && dataset.area_id === selectedAreaId
} }
function rasterPartitionsForDataset(
datasets: DatasetCreateResponse[],
representative: DatasetCreateResponse | null,
selectedAreaId: string | null,
regionalScope: boolean,
): DatasetCreateResponse[] {
if (!representative) {
return []
}
if (!regionalScope || !isPartitionedRaster(representative)) {
return [representative]
}
const productKey = datasetProductKey(representative)
return datasets
.filter(
(dataset) =>
dataset.source_name === representative.source_name
&& datasetProductKey(dataset) === productKey
&& datasetCoversSelectedArea(dataset, selectedAreaId, true),
)
.sort((left, right) => String(left.area_id ?? '').localeCompare(String(right.area_id ?? '')))
}
function pickThemeDataset( function pickThemeDataset(
datasets: DatasetCreateResponse[], datasets: DatasetCreateResponse[],
theme: DataTheme, theme: DataTheme,
selectedAreaId: string | null, selectedAreaId: string | null,
regionalScope = false,
): DatasetCreateResponse | null { ): DatasetCreateResponse | null {
const candidates = datasets.filter( const candidates = datasets.filter(
(dataset) => datasetMatchesTheme(dataset, theme) && datasetCoversSelectedArea(dataset, selectedAreaId), (dataset) =>
datasetMatchesTheme(dataset, theme)
&& datasetCoversSelectedArea(dataset, selectedAreaId, regionalScope),
) )
candidates.sort((left, right) => { candidates.sort((left, right) => {
const score = (dataset: DatasetCreateResponse) => const score = (dataset: DatasetCreateResponse) =>
@@ -745,6 +794,7 @@ export function MapWorkspace({
const [fullWorkflowError, setFullWorkflowError] = useState<string | null>(null) const [fullWorkflowError, setFullWorkflowError] = useState<string | null>(null)
const [fullWorkflowMode, setFullWorkflowMode] = useState<'new' | 'reuse'>('new') const [fullWorkflowMode, setFullWorkflowMode] = useState<'new' | 'reuse'>('new')
const selectedMapArea = areas.find((area) => area.id === selectedMapAreaId) const selectedMapArea = areas.find((area) => area.id === selectedMapAreaId)
const regionalScopeSelected = Boolean(selectedMapArea && !isMunicipalityAreaName(selectedMapArea.name))
const featureProperties = selectedMapFeature?.properties ?? null const featureProperties = selectedMapFeature?.properties ?? null
const featureSummaryEntries = featureProperties const featureSummaryEntries = featureProperties
? Object.entries(featureProperties) ? Object.entries(featureProperties)
@@ -767,70 +817,138 @@ export function MapWorkspace({
const selectedMapDataset = availableMapDatasets.find((dataset) => dataset.id === selectedMapDatasetId) ?? null const selectedMapDataset = availableMapDatasets.find((dataset) => dataset.id === selectedMapDatasetId) ?? null
const usesDefaultOsmBasemap = !import.meta.env.VITE_MAP_STYLE_URL const usesDefaultOsmBasemap = !import.meta.env.VITE_MAP_STYLE_URL
const floodHazardDatasets = useMemo( const floodHazardDatasets = useMemo(
() => availableMapDatasets () => {
.filter((dataset) => dataset.source_name === 'vmm_flood_hazard' && datasetCoversSelectedArea(dataset, selectedMapAreaId)) const scoped = availableMapDatasets
.sort((left, right) => floodScenarioLabel(left).localeCompare(floodScenarioLabel(right), 'nl')), .filter(
[availableMapDatasets, selectedMapAreaId], (dataset) =>
dataset.source_name === 'vmm_flood_hazard'
&& datasetCoversSelectedArea(dataset, selectedMapAreaId, regionalScopeSelected),
)
.sort((left, right) => floodScenarioLabel(left).localeCompare(floodScenarioLabel(right), 'nl'))
if (!regionalScopeSelected) {
return scoped
}
const products = new Map<string, DatasetCreateResponse>()
for (const dataset of scoped) {
const key = datasetProductKey(dataset)
if (key && !products.has(key)) {
products.set(key, dataset)
}
}
return Array.from(products.values())
},
[availableMapDatasets, regionalScopeSelected, selectedMapAreaId],
) )
const themeDatasetMap = useMemo(() => { const themeDatasetMap = useMemo(() => {
const result = Object.fromEntries( const result = Object.fromEntries(
DATA_THEMES.map((theme) => [theme.id, pickThemeDataset(availableMapDatasets, theme, selectedMapAreaId)]), DATA_THEMES.map((theme) => [
theme.id,
pickThemeDataset(availableMapDatasets, theme, selectedMapAreaId, regionalScopeSelected),
]),
) as Record<DataThemeId, DatasetCreateResponse | null> ) as Record<DataThemeId, DatasetCreateResponse | null>
const selectedFloodHazard = floodHazardDatasets.find((dataset) => dataset.id === selectedFloodHazardDatasetId) const selectedFloodHazard = floodHazardDatasets.find((dataset) => dataset.id === selectedFloodHazardDatasetId)
if (selectedFloodHazard) { if (selectedFloodHazard) {
result.flood_hazard = selectedFloodHazard result.flood_hazard = selectedFloodHazard
} }
return result return result
}, [availableMapDatasets, floodHazardDatasets, selectedFloodHazardDatasetId, selectedMapAreaId]) }, [availableMapDatasets, floodHazardDatasets, regionalScopeSelected, selectedFloodHazardDatasetId, selectedMapAreaId])
const themePartitionMap = useMemo(
() =>
Object.fromEntries(
DATA_THEMES.map((theme) => [
theme.id,
rasterPartitionsForDataset(
availableMapDatasets,
themeDatasetMap[theme.id],
selectedMapAreaId,
regionalScopeSelected,
),
]),
) as Record<DataThemeId, DatasetCreateResponse[]>,
[availableMapDatasets, regionalScopeSelected, selectedMapAreaId, themeDatasetMap],
)
const activeTheme = DATA_THEMES.find((theme) => theme.id === activeThemeId) ?? DATA_THEMES[0] const activeTheme = DATA_THEMES.find((theme) => theme.id === activeThemeId) ?? DATA_THEMES[0]
const activeThemeMapStyle = DATA_THEME_MAP_STYLES[activeTheme.id] const activeThemeMapStyle = DATA_THEME_MAP_STYLES[activeTheme.id]
const analysisOverlayActive = mapContentMode === 'analysis' && analysisLayerAvailable && Boolean(mapFeatureCollection) const analysisOverlayActive = mapContentMode === 'analysis' && analysisLayerAvailable && Boolean(mapFeatureCollection)
const selectedOrthophotoProduct = orthophotoProducts.find((item) => item.key === selectedOrthophotoProductKey) ?? null const selectedOrthophotoProduct = orthophotoProducts.find((item) => item.key === selectedOrthophotoProductKey) ?? null
const orthophotoImageOverlay = orthophotoResult && orthophotoImageUrl && orthophotoResult.bbox_epsg4326.length === 4 const orthophotoImageOverlay = useMemo(
() => orthophotoResult && orthophotoImageUrl && orthophotoResult.bbox_epsg4326.length === 4
? { ? {
url: orthophotoImageUrl, url: orthophotoImageUrl,
bbox: orthophotoResult.bbox_epsg4326 as [number, number, number, number], bbox: orthophotoResult.bbox_epsg4326 as [number, number, number, number],
label: orthophotoResult.display_name, label: orthophotoResult.display_name,
opacity: 0.9, opacity: 0.9,
} }
: null : null,
[orthophotoImageUrl, orthophotoResult],
)
const activeThemeDataset = themeDatasetMap[activeTheme.id] const activeThemeDataset = themeDatasetMap[activeTheme.id]
const terrainBounds = activeThemeDataset?.source_name === 'digitaal_vlaanderen_dhmv' const activeThemePartitions = themePartitionMap[activeTheme.id]
? activeThemeDataset.source_metadata?.['bbox_epsg4326'] const regionalRasterThemeActive = regionalScopeSelected && isPartitionedRaster(activeThemeDataset)
: null const terrainImageOverlays = useMemo(
const terrainImageOverlay = activeTheme.id === 'elevation' && activeThemeDataset && selectedProjectId && Array.isArray(terrainBounds) && terrainBounds.length === 4 () =>
? { activeTheme.id === 'elevation' && selectedProjectId
url: terrainImageUrl(selectedProjectId, activeThemeDataset.id), ? activeThemePartitions.flatMap((dataset) => {
bbox: terrainBounds.map(Number) as [number, number, number, number], const bounds = dataset.source_metadata?.['bbox_epsg4326']
label: getDatasetDisplayName(activeThemeDataset), return dataset.source_name === 'digitaal_vlaanderen_dhmv'
&& Array.isArray(bounds)
&& bounds.length === 4
? [{
url: terrainImageUrl(selectedProjectId, dataset.id),
bbox: bounds.map(Number) as [number, number, number, number],
label: getDatasetDisplayName(dataset),
opacity: 0.82, opacity: 0.82,
} }]
: null : []
const floodHazardBounds = activeThemeDataset?.source_name === 'vmm_flood_hazard' })
? activeThemeDataset.source_metadata?.['bbox_epsg4326'] : [],
: null [activeTheme.id, activeThemePartitions, selectedProjectId],
const floodHazardImageOverlay = activeTheme.id === 'flood_hazard' && activeThemeDataset && selectedProjectId && Array.isArray(floodHazardBounds) && floodHazardBounds.length === 4 )
? { const floodHazardImageOverlays = useMemo(
url: floodHazardImageUrl(selectedProjectId, activeThemeDataset.id), () =>
bbox: floodHazardBounds.map(Number) as [number, number, number, number], activeTheme.id === 'flood_hazard' && selectedProjectId
label: floodScenarioLabel(activeThemeDataset), ? activeThemePartitions.flatMap((dataset) => {
const bounds = dataset.source_metadata?.['bbox_epsg4326']
return dataset.source_name === 'vmm_flood_hazard'
&& Array.isArray(bounds)
&& bounds.length === 4
? [{
url: floodHazardImageUrl(selectedProjectId, dataset.id),
bbox: bounds.map(Number) as [number, number, number, number],
label: floodScenarioLabel(dataset),
opacity: 0.82, opacity: 0.82,
} }]
: null : []
})
: [],
[activeTheme.id, activeThemePartitions, selectedProjectId],
)
const thematicRasterBounds = activeThemeDataset?.source_name === 'department_omgeving_thematic_raster' const thematicRasterBounds = activeThemeDataset?.source_name === 'department_omgeving_thematic_raster'
? activeThemeDataset.source_metadata?.['bbox_epsg4326'] ? activeThemeDataset.source_metadata?.['bbox_epsg4326']
: null : null
const thematicRasterImageOverlay = activeThemeDataset?.source_name === 'department_omgeving_thematic_raster' && selectedProjectId && Array.isArray(thematicRasterBounds) && thematicRasterBounds.length === 4 const thematicRasterImageOverlays = useMemo(
? { () => activeThemeDataset?.source_name === 'department_omgeving_thematic_raster' && selectedProjectId && Array.isArray(thematicRasterBounds) && thematicRasterBounds.length === 4
? [{
url: thematicRasterImageUrl(selectedProjectId, activeThemeDataset.id), url: thematicRasterImageUrl(selectedProjectId, activeThemeDataset.id),
bbox: thematicRasterBounds.map(Number) as [number, number, number, number], bbox: thematicRasterBounds.map(Number) as [number, number, number, number],
label: getDatasetDisplayName(activeThemeDataset), label: getDatasetDisplayName(activeThemeDataset),
opacity: 0.78, opacity: 0.78,
} }]
: null : [],
[activeThemeDataset, selectedProjectId, thematicRasterBounds],
)
const thematicLegendMin = String(activeThemeDataset?.source_metadata?.['legend_min_label'] ?? 'Lagere waarde') const thematicLegendMin = String(activeThemeDataset?.source_metadata?.['legend_min_label'] ?? 'Lagere waarde')
const thematicLegendMax = String(activeThemeDataset?.source_metadata?.['legend_max_label'] ?? 'Hogere waarde') const thematicLegendMax = String(activeThemeDataset?.source_metadata?.['legend_max_label'] ?? 'Hogere waarde')
const activeImageOverlay = thematicRasterImageOverlay ?? floodHazardImageOverlay ?? terrainImageOverlay ?? orthophotoImageOverlay const activeImageOverlays = useMemo(
() => thematicRasterImageOverlays.length > 0
? thematicRasterImageOverlays
: floodHazardImageOverlays.length > 0
? floodHazardImageOverlays
: terrainImageOverlays.length > 0
? terrainImageOverlays
: orthophotoImageOverlay ? [orthophotoImageOverlay] : [],
[floodHazardImageOverlays, orthophotoImageOverlay, terrainImageOverlays, thematicRasterImageOverlays],
)
const activeScopeProject = projects.find((project) => project.id === selectedProjectId) ?? null const activeScopeProject = projects.find((project) => project.id === selectedProjectId) ?? null
const activeScopeLabel = activeScopeProject ? operationalScopeProjectLabel(activeScopeProject) : 'Werkgebied' const activeScopeLabel = activeScopeProject ? operationalScopeProjectLabel(activeScopeProject) : 'Werkgebied'
const municipalityAreaCount = areas.filter((area) => /^Gemeente\s/i.test(area.name)).length const municipalityAreaCount = areas.filter((area) => /^Gemeente\s/i.test(area.name)).length
@@ -865,7 +983,7 @@ export function MapWorkspace({
[themeInsights], [themeInsights],
) )
const activeSelectionResult = themeResults.find((item) => item.theme.id === activeThemeId)?.result const activeSelectionResult = themeResults.find((item) => item.theme.id === activeThemeId)?.result
?? (selectedMapDataset?.id === activeThemeDataset?.id ? mapSelectionResult : null) ?? (!regionalRasterThemeActive && selectedMapDataset?.id === activeThemeDataset?.id ? mapSelectionResult : null)
const selectedAreaSquareMetres = useMemo( const selectedAreaSquareMetres = useMemo(
() => () =>
bboxesEqual(mapSelectionBbox, selectedAreaBbox) && selectedMapArea?.area_m2 bboxesEqual(mapSelectionBbox, selectedAreaBbox) && selectedMapArea?.area_m2
@@ -1106,14 +1224,23 @@ export function MapWorkspace({
const loadAllThemeResults = async (bbox: VectorSelectionBBox, areaId?: string) => { const loadAllThemeResults = async (bbox: VectorSelectionBBox, areaId?: string) => {
const availableThemes = DATA_THEMES.flatMap((theme) => { const availableThemes = DATA_THEMES.flatMap((theme) => {
const dataset = themeDatasetMap[theme.id] const dataset = themeDatasetMap[theme.id]
return dataset ? [{ themeId: theme.id, dataset }] : [] return dataset
? [{
themeId: theme.id,
dataset,
partitioned: regionalScopeSelected && isPartitionedRaster(dataset),
}]
: []
}) })
await loadThemeInsights(bbox, availableThemes, areaId) await loadThemeInsights(bbox, availableThemes, areaId)
} }
const analyzeSelection = async (bbox: VectorSelectionBBox, areaId?: string) => { const analyzeSelection = async (bbox: VectorSelectionBBox, areaId?: string) => {
setSelectionBbox(bbox) setSelectionBbox(bbox)
const tasks: Array<Promise<unknown>> = [onRunMapSelectionExtract(bbox, areaId), loadAllThemeResults(bbox, areaId)] const tasks: Array<Promise<unknown>> = [loadAllThemeResults(bbox, areaId)]
if (!regionalRasterThemeActive) {
tasks.push(onRunMapSelectionExtract(bbox, areaId))
}
if (analysisMode === 'evolution' && earlierDatasetId && laterDatasetId) { if (analysisMode === 'evolution' && earlierDatasetId && laterDatasetId) {
tasks.push(compareTemporalSnapshots(earlierDatasetId, laterDatasetId, bbox, areaId)) tasks.push(compareTemporalSnapshots(earlierDatasetId, laterDatasetId, bbox, areaId))
} }
@@ -1267,6 +1394,7 @@ export function MapWorkspace({
<div className="geo-theme-list"> <div className="geo-theme-list">
{DATA_THEMES.map((theme) => { {DATA_THEMES.map((theme) => {
const dataset = themeDatasetMap[theme.id] const dataset = themeDatasetMap[theme.id]
const partitionCount = themePartitionMap[theme.id].length
const temporalGroups = themeTemporalSeriesMap[theme.id] const temporalGroups = themeTemporalSeriesMap[theme.id]
const temporalGroup = temporalGroups[0] const temporalGroup = temporalGroups[0]
const evolutionAvailable = temporalGroups.some((group) => group.items.length >= 2) const evolutionAvailable = temporalGroups.some((group) => group.items.length >= 2)
@@ -1296,7 +1424,7 @@ export function MapWorkspace({
? 'Alleen huidige toestand' ? 'Alleen huidige toestand'
: 'Bron nog niet ingeladen' : 'Bron nog niet ingeladen'
: dataset : dataset
? datasetAvailabilityLabel(dataset) ? datasetAvailabilityLabel(dataset, partitionCount)
: 'Bron nog niet ingeladen'} : 'Bron nog niet ingeladen'}
</small> </small>
</span> </span>
@@ -1416,7 +1544,13 @@ export function MapWorkspace({
<span>2</span> <span>2</span>
<div> <div>
<h3>Selecteer een gebied</h3> <h3>Selecteer een gebied</h3>
<p>{bboxSelectionMode ? 'Sleep nu een rechthoek op de kaart.' : 'Sleep een rechthoek of analyseer het volledige werkgebied.'}</p> <p>
{bboxSelectionMode
? 'Sleep nu een rechthoek op de kaart.'
: regionalRasterThemeActive
? 'Teken een rechthoek; de juiste gemeentelijke rasters worden automatisch gecombineerd.'
: 'Sleep een rechthoek of analyseer het volledige werkgebied.'}
</p>
</div> </div>
</div> </div>
<div className="geo-map-actions"> <div className="geo-map-actions">
@@ -1430,11 +1564,12 @@ export function MapWorkspace({
</button> </button>
<button <button
className="secondary-action" className="secondary-action"
disabled={!activeThemeDataset || (analysisMode === 'evolution' && activeTemporalSeries.length < 2) || !selectedAreaBbox || mapSelectionLoading || themeResultsLoading} disabled={!activeThemeDataset || regionalRasterThemeActive || (analysisMode === 'evolution' && activeTemporalSeries.length < 2) || !selectedAreaBbox || mapSelectionLoading || themeResultsLoading}
type="button" type="button"
title={regionalRasterThemeActive ? 'Teken een begrensde rechthoek voor een regionale rasteranalyse.' : undefined}
onClick={() => selectedAreaBbox && void analyzeSelection(selectedAreaBbox, selectedMapArea?.id)} onClick={() => selectedAreaBbox && void analyzeSelection(selectedAreaBbox, selectedMapArea?.id)}
> >
Volledig werkgebied {regionalRasterThemeActive ? 'Selecteer een deelgebied' : 'Volledig werkgebied'}
</button> </button>
<button className="secondary-action" disabled={!mapSelectionBbox} type="button" onClick={clearAreaSelection}> <button className="secondary-action" disabled={!mapSelectionBbox} type="button" onClick={clearAreaSelection}>
Wis selectie Wis selectie
@@ -1450,7 +1585,7 @@ export function MapWorkspace({
areaData={areaFeatureCollection} areaData={areaFeatureCollection}
selectedFeature={selectedFeature} selectedFeature={selectedFeature}
selectionData={analysisMode === 'current' ? mapSelectionResult?.geojson ?? null : null} selectionData={analysisMode === 'current' ? mapSelectionResult?.geojson ?? null : null}
imageOverlay={activeImageOverlay} imageOverlays={activeImageOverlays}
selectionBbox={mapSelectionBbox} selectionBbox={mapSelectionBbox}
bboxSelectionMode={bboxSelectionMode} bboxSelectionMode={bboxSelectionMode}
visible={mapLayerVisible} visible={mapLayerVisible}
@@ -1466,12 +1601,17 @@ export function MapWorkspace({
/> />
<div className="geo-map-legend" aria-label="Kaartlegende"> <div className="geo-map-legend" aria-label="Kaartlegende">
<span><i className="geo-legend-area" /> Werkgebied</span> <span><i className="geo-legend-area" /> Werkgebied</span>
{thematicRasterImageOverlay ? ( {thematicRasterImageOverlays.length > 0 ? (
<span className="geo-legend-thematic"> <span className="geo-legend-thematic">
<i className={`geo-legend-ramp geo-legend-ramp-${activeTheme.id}`} /> <i className={`geo-legend-ramp geo-legend-ramp-${activeTheme.id}`} />
<small>{thematicLegendMin} {thematicLegendMax}</small> <small>{thematicLegendMin} {thematicLegendMax}</small>
</span> </span>
) : activeImageOverlay ? <span><i className="geo-legend-imagery" /> {activeImageOverlay.label}</span> : null} ) : activeImageOverlays.length > 0 ? (
<span>
<i className="geo-legend-imagery" /> {activeImageOverlays[0].label}
{activeImageOverlays.length > 1 ? ` · ${activeImageOverlays.length} gemeenten` : ''}
</span>
) : null}
{analysisOverlayActive ? ( {analysisOverlayActive ? (
<> <>
<span><i className="geo-legend-layer geo-legend-layer-buildings" /> AI-kandidaten</span> <span><i className="geo-legend-layer geo-legend-layer-buildings" /> AI-kandidaten</span>
@@ -9,6 +9,7 @@ import { thematicRasterSelectionToMapSelection } from '../lib/thematicRaster'
export interface MapThemeQuery<TThemeId extends string> { export interface MapThemeQuery<TThemeId extends string> {
themeId: TThemeId themeId: TThemeId
dataset: DatasetCreateResponse dataset: DatasetCreateResponse
partitioned?: boolean
} }
export interface MapThemeInsight<TThemeId extends string> extends MapThemeQuery<TThemeId> { export interface MapThemeInsight<TThemeId extends string> extends MapThemeQuery<TThemeId> {
@@ -54,19 +55,35 @@ export function useMapThemeSelectionInsights<TThemeId extends string>(
setThemeInsightsError(null) setThemeInsightsError(null)
try { try {
const settled = await Promise.allSettled( const settled = await Promise.allSettled(
queries.map(async ({ themeId, dataset }) => ({ queries.map(async ({ themeId, dataset, partitioned }) => ({
themeId, themeId,
dataset, dataset,
result: dataset.dataset_type === 'raster' && dataset.source_name === 'digitaal_vlaanderen_dhmv' result: dataset.dataset_type === 'raster' && dataset.source_name === 'digitaal_vlaanderen_dhmv'
? terrainSelectionToMapSelection(await datasetsApi.selectTerrain(selectedProjectId, dataset.id, { ? terrainSelectionToMapSelection(
partitioned
? await datasetsApi.selectTerrainPartitions(selectedProjectId, {
bbox, bbox,
area_id: areaId, area_id: areaId,
})) product_key: String(dataset.source_metadata?.['product_key'] ?? 'dtm_1m'),
})
: await datasetsApi.selectTerrain(selectedProjectId, dataset.id, {
bbox,
area_id: areaId,
}),
)
: dataset.dataset_type === 'raster' && dataset.source_name === 'vmm_flood_hazard' : dataset.dataset_type === 'raster' && dataset.source_name === 'vmm_flood_hazard'
? floodHazardSelectionToMapSelection(await datasetsApi.selectFloodHazard(selectedProjectId, dataset.id, { ? floodHazardSelectionToMapSelection(
partitioned
? await datasetsApi.selectFloodHazardPartitions(selectedProjectId, {
bbox, bbox,
area_id: areaId, area_id: areaId,
})) product_key: String(dataset.source_metadata?.['product_key'] ?? 'pluviaal_current_t100'),
})
: await datasetsApi.selectFloodHazard(selectedProjectId, dataset.id, {
bbox,
area_id: areaId,
}),
)
: dataset.dataset_type === 'raster' && dataset.source_name === 'department_omgeving_thematic_raster' : dataset.dataset_type === 'raster' && dataset.source_name === 'department_omgeving_thematic_raster'
? thematicRasterSelectionToMapSelection(await datasetsApi.selectThematicRaster(selectedProjectId, dataset.id, { ? thematicRasterSelectionToMapSelection(await datasetsApi.selectThematicRaster(selectedProjectId, dataset.id, {
bbox, bbox,
+10
View File
@@ -134,6 +134,11 @@ export const datasetsApi = {
payload: { bbox: VectorSelectionRequest['bbox']; area_id?: string }, payload: { bbox: VectorSelectionRequest['bbox']; area_id?: string },
): Promise<TerrainSelectionResponse> => ): Promise<TerrainSelectionResponse> =>
apiPost<TerrainSelectionResponse>(`/api/v1/projects/${projectId}/datasets/${datasetId}/raster/terrain/select`, payload), apiPost<TerrainSelectionResponse>(`/api/v1/projects/${projectId}/datasets/${datasetId}/raster/terrain/select`, payload),
selectTerrainPartitions: (
projectId: string,
payload: { bbox: VectorSelectionRequest['bbox']; area_id?: string; product_key: string },
): Promise<TerrainSelectionResponse> =>
apiPost<TerrainSelectionResponse>(`/api/v1/projects/${projectId}/datasets/raster/terrain/select`, payload),
acquireFloodHazard: (projectId: string, payload: FloodHazardAcquireRequest): Promise<JobRead> => acquireFloodHazard: (projectId: string, payload: FloodHazardAcquireRequest): Promise<JobRead> =>
apiPost<JobRead>(`/api/v1/projects/${projectId}/datasets/flood-hazard/acquire`, payload), apiPost<JobRead>(`/api/v1/projects/${projectId}/datasets/flood-hazard/acquire`, payload),
listFloodHazardProducts: (projectId: string): Promise<{ items: FloodHazardProductRead[]; total: number }> => listFloodHazardProducts: (projectId: string): Promise<{ items: FloodHazardProductRead[]; total: number }> =>
@@ -144,6 +149,11 @@ export const datasetsApi = {
payload: { bbox: VectorSelectionRequest['bbox']; area_id?: string }, payload: { bbox: VectorSelectionRequest['bbox']; area_id?: string },
): Promise<FloodHazardSelectionResponse> => ): Promise<FloodHazardSelectionResponse> =>
apiPost<FloodHazardSelectionResponse>(`/api/v1/projects/${projectId}/datasets/${datasetId}/raster/flood-hazard/select`, payload), apiPost<FloodHazardSelectionResponse>(`/api/v1/projects/${projectId}/datasets/${datasetId}/raster/flood-hazard/select`, payload),
selectFloodHazardPartitions: (
projectId: string,
payload: { bbox: VectorSelectionRequest['bbox']; area_id?: string; product_key: string },
): Promise<FloodHazardSelectionResponse> =>
apiPost<FloodHazardSelectionResponse>(`/api/v1/projects/${projectId}/datasets/raster/flood-hazard/select`, payload),
acquireThematicRaster: (projectId: string, payload: ThematicRasterAcquireRequest): Promise<JobRead> => acquireThematicRaster: (projectId: string, payload: ThematicRasterAcquireRequest): Promise<JobRead> =>
apiPost<JobRead>(`/api/v1/projects/${projectId}/datasets/thematic-raster/acquire`, payload), apiPost<JobRead>(`/api/v1/projects/${projectId}/datasets/thematic-raster/acquire`, payload),
listThematicRasterProducts: (projectId: string): Promise<{ items: ThematicRasterProductRead[]; total: number }> => listThematicRasterProducts: (projectId: string): Promise<{ items: ThematicRasterProductRead[]; total: number }> =>
+4
View File
@@ -360,6 +360,8 @@ export interface DhmvProductRead {
export interface TerrainSelectionResponse { export interface TerrainSelectionResponse {
dataset_id: string dataset_id: string
dataset_ids: string[]
partition_count: number
product_key: string product_key: string
surface_model: 'terrain' | 'surface' surface_model: 'terrain' | 'surface'
selection_bbox: VectorSelectionBBox selection_bbox: VectorSelectionBBox
@@ -410,6 +412,8 @@ export interface FloodHazardProductRead {
export interface FloodHazardSelectionResponse { export interface FloodHazardSelectionResponse {
dataset_id: string dataset_id: string
dataset_ids: string[]
partition_count: number
product_key: string product_key: string
mechanism: 'pluviaal' | 'fluviaal' mechanism: 'pluviaal' | 'fluviaal'
climate_context: string climate_context: string
+8
View File
@@ -1582,6 +1582,10 @@ not assemble a monolithic Kempen height raster, does not claim annual terrain
change and rejects any terrain-analysis response that stops listing water change and rejects any terrain-analysis response that stops listing water
depth and water volume as unsupported. depth and water volume as unsupported.
The live completed matrix contains 56/56 ready Dataset/DatasetVersion pairs.
The regional Map workspace reads intersecting partitions through the bounded
partition-selection endpoint; operator storage remains unchanged.
## Mol VMM flood-hazard scenarios ## Mol VMM flood-hazard scenarios
Acquire and validate all twelve official VMM fluvial/pluvial flood-depth Acquire and validate all twelve official VMM fluvial/pluvial flood-depth
@@ -1639,6 +1643,10 @@ take a long time because every VMM WCS tile is bounded, rate-limited and
validated. This is expected operator work; the app never fetches these rasters validated. This is expected operator work; the app never fetches these rasters
on page load or map click. on page load or map click.
The live completed matrix contains 336/336 ready Dataset/DatasetVersion pairs.
A repeat run reuses the existing request identities. Regional map selections
analyse the persisted files and never trigger the public WCS.
## Cross-domain Mol profile ## Cross-domain Mol profile
Load the five official policy rasters for the exact Mol municipality Area and Load the five official policy rasters for the exact Mol municipality Area and