geointel/docs/DATA_SOURCES.md

# Data Sources

## Orthofotomozaiek Vlaanderen - meest recent

- Naam: Orthofotomozaiek middenschalig, winteropnamen, kleur, meest recent
- Beheerder: Digitaal Vlaanderen
- Type: RGB-raster via WMS
- Operationele laag: `OMWRGBMRVL` / `Ortho`
- CRS bij opslag: `EPSG:31370`
- Gebruik: expliciet begrensde luchtbeeldanalyse en lokale gebouwdetectie
- Autoriteit: officiele beeldbron; AI-detecties zelf zijn niet-autoritatief

De hoofdkaart kan na een getekende rechthoek een expliciete, begrensde WMS
GetMap-aanvraag uitvoeren. De backend aanvaardt alleen rechthoeken van 128 tot
1.024 meter per zijde, samplet standaard op 1 m/pixel voor het actieve lokale
modelprofiel en bewaart het gegeorefereerde GeoTIFF via DatasetService met
aanvraag, checksum, bron, attributie en beperking. Een identieke selectie mag
24 uur worden hergebruikt. Er is geen startupfetch en de browser bevraagt de
externe WMS nooit rechtstreeks.

De bron is de meest recente samengestelde wintermozaiek op het moment van de
aanvraag. GeoIntel verzint geen historische pixelopnamedatum. Bronnen:

- https://www.vlaanderen.be/datavindplaats/catalogus/orthofotomozaiek-middenschalig-winteropnamen-kleur-meest-recent-vlaanderen
- https://www.vlaanderen.be/digitaal-vlaanderen/onze-diensten-en-platformen/luchtopnamen/gebruik-orthofotomozaieken

Dit document verzamelt concrete databronnen voor GeoIntel Kempen.

## GRB — Basiskaart Vlaanderen

- Naam: Basiskaart Vlaanderen / GRB
- Beheerder: Digitaal Vlaanderen
- Type: vector
- Gebruik: referentiegebouwen, wegen, terreinobjecten, QA/QC
- Toegang: WFS of download
- Cache: PostGIS per analysegebied
- Prioriteit: V1
- Opmerking: belangrijkste officiële referentiebron voor QA/QC.

### Volledige gemeente Mol

De operationele Mol-workspace gebruikt twee expliciete Digitaal Vlaanderen
OGC API-bronnen:

- `VRBG/Refgem`, gefilterd op `NAAM='Mol'` en gecontroleerd op NIS-code
  `13025`, als officiële gemeentegrens.
- `GRB/GBG`, volledig gepagineerd en exact tegen die grens gesneden, als
  gemeentebrede gebouwreferentie.

Deze bronnen worden uitsluitend via het expliciete operatorcommando
`scripts/provision_mol_municipality_workspace.py` opgehaald. De applicatie
start geen verborgen providerfetch. De resulterende GeoJSON-artefacten,
checksums en bron-URL's worden onder persistent operator storage bewaard en
via de bestaande DatasetService/vectorfeature-flow geïmporteerd.

### Operationele regio Kempen

Voor regionale analyse gebruikt GeoIntel de officiële `Vervoerregio Kempen`
van de Vlaamse overheid. Deze beleidsregio bestaat uit 28 gemeenten. Ze is
expliciet gekozen omdat ze actueel, reproduceerbaar en bestuurlijk
gedocumenteerd is. GeoIntel beweert niet dat deze grens samenvalt met de
ruimere culturele, landschappelijke of historische Kempen.

`scripts/geographic_scopes.py` registreert de 28 actuele namen en NIS-codes.
`scripts/provision_geographic_scope.py` controleert die tegen `VRBG/Refgem`,
maakt de regionale union en bewaart zowel de union als alle afzonderlijke
gemeentegrenzen met bron-URL, checksum, autoriteit en beperking. De gemeten
operationele grens omvat circa `1.399,25 km²`.

Bronnen:

- https://www.vlaanderen.be/mobiliteitsprofessionals/personenvervoer/basisbereikbaarheid/mobiliteitsuitdagingen-regionaal-aanpakken/vervoerregios/over-de-vervoerregio-kempen
- https://geo.api.vlaanderen.be/VRBG/ogc/features/v1/collections/Refgem/items

De scope-operator haalt geen thematische gegevens op. GRB, bevolking en
landgebruik volgen als afzonderlijke begrensde imports; ontbrekende regionale
thema's blijven zichtbaar onbeschikbaar.

De actuele regionale gebouwlaag wordt afzonderlijk opgehaald door
`scripts/provision_regional_grb_buildings.py`. De operator bevraagt
`GRB/GBG` per officiële gemeentegrens, volgt alle pagina's en weigert elk
veiligheidslimiet als truncatie te verbergen. Grensoverschrijdende objecten
worden aan één gemeentepartitie toegewezen op basis van de grootste overlap;
de uiteindelijke geometrie wordt uitsluitend tegen de volledige regiogrens
gesneden. Daardoor bevat het regionale dataset elke GRB-bronidentiteit exact
één keer.

De 28 bronpartities, paginabronnen, aantallen en checksums blijven als
operatorbewijs bewaard. GeoIntel indexeert ze batchgewijs als één regionaal
referentiedataset via DatasetService en VectorFeatureService. De publieke GRB
provider blijft `not_configured`: dit is een bewuste operatorrun en geen live
download vanuit een browseractie of applicatiestart.

De overige actuele GRB-contextlagen volgen dezelfde begrensde aanpak via
`scripts/provision_regional_grb_context.py`:

- `roads`: collectie `Wegsegment` (lijngeometrie). Dit zijn wegsegmenten uit
  de basiskaart, geen verkeersmetingen en geen routeringsnetwerk.
- `water`: collecties `WTZ` (vlakken), `WLAS` en `WGR` (lijnen). De gemengde
  objecttelling is geen maat voor wateroppervlakte, debiet of volume.
- `parcels`: collectie `ADP` (vlakken). Dit is de grafische ligging van het
  vermoedelijke kadastrale perceel en geen juridisch landmeetkundig bewijs.

Lijnobjecten worden bij gemeentegrenzen toegewezen op basis van de grootste
intersectielengte, vlakobjecten op basis van de grootste intersectieoppervlakte
en exacte ties op de laagste NIS-code. De brongeometrie wordt alleen tegen de
volledige regiogrens gesneden. Bron-ID's worden met hun collectienaam
gekwalificeerd zodat de drie watercollecties nooit stilzwijgend botsen.

Collectiecatalogus en contract:

- https://geo.api.vlaanderen.be/GRB/ogc/features/v1/collections?f=text%2Fhtml
- https://geo.api.vlaanderen.be/GRB/ogc/features/v1/openapi?f=text%2Fhtml

### Mol population history

`scripts/provision_mol_population_history.py` imports official Statbel
population-by-statistical-sector tables and matching sector geometries for
2021 through 2025. It keeps the published sector total as
`population_total`, clips the official geometry to Mol NIS `13025` and uploads
each year as a separate dataset in
`statbel:population-statistical-sector:mol`.

Complete sectors use their published population total. A rectangle that cuts
through a sector uses an explicitly labelled area-weighted estimate; the
source does not justify a more precise intra-sector distribution.

Statistical-sector codes and boundaries can change between editions. GeoIntel
therefore compares population metrics but does not present sector additions,
removals or boundary changes as population-object lineage.

### Historical land use for Mol and the approved region

`scripts/provision_mol_historical_landuse.py` uses the official Digitaal
Vlaanderen Historical Land Use WFS for the 1778, 1873 and 1969 collections.
Buildings, forest, water and roads are filtered server-side, clipped to the
official Mol boundary and uploaded through the existing dataset service.
Source classes, request URLs, observation year, simplification tolerance and
methodological limitations remain in provenance.

These historical map editions support exploratory area evolution, not
cadastral object lineage. Their feature identities are declared unstable and
GeoIntel does not fabricate added/removed object counts.

The operator uses standards-compliant WFS 2.0 XML POST requests. This keeps
the spatial/class filters server-side without exposing a long XML filter in a
GET query, which the public gateway rejects.

`scripts/provision_regional_historical_landuse.py` extends the building,
water and road series to the approved 28-municipality transport-region scope.
Broad regional WFS counts are capped at 10,000 by the public service, so the
operator uses the retained official VRBG municipality boundaries as 28
explicit source partitions. Each exact WFS JSON response is retained as a
deterministically compressed artifact with response and artifact SHA256,
then clipped to its municipality before one regional snapshot is assembled.
Partition-suffixed feature ids prevent cross-boundary source polygons from
colliding. Those ids remain processing evidence and are not historical object
lineage.

### Mol modern land use

`scripts/provision_official_landuse_timeseries.py` uses the public Departement
Omgeving/MercatorNet WCS to retrieve the harmonized version 3 land-use maps for
2013, 2016, 2019, 2022 and 2025. The source is a categorical 10 m GeoTIFF in
Belgian Lambert 72 (`EPSG:31370`) with 19 documented classes. GeoIntel derives
four explicitly labelled, methodologically comparable surfaces from one
retained source raster per year:

- `forest`: class 12, bosoppervlakte;
- `water`: class 17, wateroppervlakte (not water volume);
- `built`: classes 1-4 and 6-8, surface used by built functions, not GRB
  building footprints;
- `transport`: class 5, transport-infrastructure surface, not GRB road length.

Every source raster is clipped against the explicit official boundary,
validated for integer classes, CRS and resolution, checksummed and retained in
persistent operator storage. Forest cells are polygonized and clipped in
`EPSG:31370`, then transformed to canonical `EPSG:4326` and uploaded through
DatasetService/VectorFeatureService. The raw raster remains the provenance
artifact; the persisted polygons provide rectangle selection, PostGIS area
aggregation and temporal comparison without a parallel query path.
All cells touching the requested boundary are considered before the exact
metric geometry clip; NoData cells are excluded and counted in the manifest.

The modern series key is
`department-omgeving:land-use:forest:mol`. It remains separate from
`digitaal-vlaanderen:historical-landuse:forest:mol`: the 1778-1969
cartographic editions and the harmonized 2013-2025 10 m land-use maps are not
presented as one continuous measurement method. Raster-derived polygon
identities are unstable, so GeoIntel compares hectares and never invents
added/removed forest objects.

### Regional official time series

`scripts/provision_regional_timeseries.py` applies the population, modern
land-use and partitioned historical land-use operators to the approved 28-municipality
`kempen-transport-region` scope. It resolves the checksummed VRBG union
boundary and writes to `Kempen Regional Workbench` through the normal dataset
API. Regional keys are
`statbel:population-statistical-sector:kempen-transport-region` and
`department-omgeving:land-use:{theme}:kempen-transport-region`, so Mol datasets
remain independent observations rather than aliases.
Historical buildings, water and roads use
`digitaal-vlaanderen:historical-landuse:{theme}:kempen-transport-region` and
remain separate from both current GRB geometry and the harmonized 2013-2025
land-use rasters.

The command is explicit and operator-triggered. No source fetch happens during
startup or map interaction. Partial statistical sectors remain area-weighted
population estimates; forest hectares remain measurements in the harmonized
10 m source representation and are not cadastral forest boundaries.
The WCS source limits large responses, so regional forest retrieval is
partitioned by the same 28 official municipality boundaries. GeoIntel retains
those source rasters, merges them on their native 10 m grid and applies the
exact regional union clip before polygon persistence.

Official catalogues:

- https://www.vlaanderen.be/datavindplaats/catalogus/wfs-historisch-landgebruik
- https://www.vlaanderen.be/datavindplaats/catalogus/digitalisatie-historisch-landgebruik-en-landgebruiksveranderingen-in-vlaanderen-1778-2022
- https://www.vlaanderen.be/datavindplaats/catalogus/landgebruik-vlaanderen-toestand-2013
- https://www.vlaanderen.be/datavindplaats/catalogus/landgebruik-vlaanderen-toestand-2016
- https://www.vlaanderen.be/datavindplaats/catalogus/landgebruik-vlaanderen-toestand-2019
- https://www.vlaanderen.be/datavindplaats/catalogus/landgebruik-vlaanderen-toestand-2022
- https://www.vlaanderen.be/datavindplaats/catalogus/landgebruik-vlaanderen-toestand-2025
- https://www.vlaanderen.be/statistiek-vlaanderen/ruimtegebruik/landgebruik/metadata-landgebruik

## Audited official follow-up sources

These sources are available from their public authorities but are not silently
treated as loaded GeoIntel data. The Source inventory labels them separately
until a governed operator import, provenance record and validation pass exist.

- Biologische Waarderingskaart / Natura 2000 (INBO), state 2025: suitable for
  habitat, biotope and ecological-value analysis, not a continuous annual
  series.
- Buildings and Addresses Register (Digitaal Vlaanderen): continuously updated
  building status, life cycle and address linkage; complementary to GRB
  geometry and not yet imported.
- DHMV II DTM/DSM (Digitaal Vlaanderen): governed bounded WCS acquisition is
  implemented below. The official 1 m source and 5 m analysis copy are suitable
  for measured elevation, relief and slope. Drainage remains an interpretation;
  the products do not provide water depth.

## Governed BWK and Natura 2000 state 2025

`scripts/provision_mol_bwk_natura2000.py` uses the official production WFS
`https://geo.api.vlaanderen.be/BWK/wfs`, feature type `BWK:Bwkhab`. It follows
server `next` links and uses WFS `startIndex` only when a full page omits that
link. A short final page is required for completeness. Raw response pages,
request URLs and SHA256 checksums are retained.

The source bbox is reduced to the exact persisted Mol Area in Belgian Lambert
72. Only valid polygonal intersections are emitted, transformed back to
EPSG:4326 and imported through DatasetService with `source_name` equal to
`inbo_bwk_natura2000` and `reference_layer_name` equal to `nature_value`.
Required attribution is `Bron: INBO`.

`EVAL` classes are reported separately. Natura 2000 and regionally important
biotope hectares use official `PHAB` shares; uncertain `ohab` polygons remain
knowledge gaps. The official report warns that parts of the 2025 edition,
especially outside Habitats Directive areas, can still be based on older
mapping. GeoIntel therefore preserves origin fields and never presents this
single edition as a historical trend.

## Governed annual agricultural-use parcels

`scripts/provision_agricultural_parcel_history.py` uses the official annual
download archives published by Agentschap Landbouw en Zeevisserij. Only the
definitive editions 2008-2025 are allowed. The provisional current-campaign
snapshot is deliberately excluded, and the retiring Digitaal Vlaanderen
`Landbgebrperc` WFS is not used as the long-term source path.

Each official ZIP, SHA256 checksum, full-edition crop code list and manifest is
retained. The temporary GeoPackage is validated as EPSG:31370 with the stable
annual field set, read through the GIS optional dependencies and deleted after
the normalized GeoJSON has been built. Geometry is clipped exactly against the
persisted scope Area in Lambert 72 and imported only through DatasetService.

The regional default creates the series
`alz:agricultural-use-parcels:kempen-transport-region`; `--scope mol` creates an
independent Mol series. Selection and evolution expose exact intersected
hectares for total declared use and official main-crop groups. Individual
parcel additions/removals are unavailable because parcel identity is not
stable across campaign years.

Official catalogues:

- https://landbouwcijfers.vlaanderen.be/open-geodata-landbouwgebruikspercelen
- https://www.vlaanderen.be/datavindplaats/catalogus/open-geodata-landbouwgebruikspercelen

## Governed Waterinfo station history

`scripts/provision_waterinfo_station_history.py` uses the public Waterinfo
KiWIS query service only after an explicit operator command. It discovers the
documented annual water-level (`192784`) and discharge (`192895`) groups,
filters station points against the exact persisted Area and retains the raw
station/value JSON plus SHA256 manifests.

Each station and year becomes one immutable reference Dataset through the
normal upload API. Series keys include the station identity; different stations
are never averaged into one municipal value. Selection aggregation is a numeric
mean over the selected station records and remains labelled as a point
measurement. Water level or discharge does not establish area-wide water
volume without compatible depth, profile and coverage data.

## Governed historical orthophotos

The bounded map acquisition registry includes the official annual winter
mosaics for 2012-2025, period products for 2000-2003, 2005-2007 and 2008-2011,
the RGB 1979-1990 summer mosaic and the panchromatic 1971 mosaic. Requests stay
within the configured 128-1,024 m safety envelope and are persisted as
EPSG:31370 raster Datasets with explicit product, layer, observation period,
attribution and request hashes.

Historical mosaics are for visual comparison only in this phase. Current YOLO
building detection and current GRB QA remain restricted to `most_recent`, since
validating an old image against today's building state would produce dishonest
quality metrics.

## OSM

- Naam: OpenStreetMap
- Type: vector
- Gebruik: fallback referentie, wegen, POI's, landuse
- Toegang: Overpass API of lokale extracten
- Cache: PostGIS per analysegebied
- Prioriteit: V1 fallback

## Sentinel-2

- Naam: Copernicus Sentinel-2
- Type: multispectraal raster
- Gebruik: NDVI, NDWI, NDBI, remote sensing change detection
- Toegang: Copernicus/STAC later te bepalen
- Cache: raster storage + metadata in PostGIS
- Prioriteit: V2

## DHMV

- Naam: Digitaal Hoogtemodel Vlaanderen
- Type: raster/hoogte
- Gebruik: DTM/DSM hoogte, reliëf, helling en laagste punten
- Toegang: productie-WCS `https://geo.api.vlaanderen.be/DHMV/wcs`
- Coverages: `DHMVII_DTM_1m`, `DHMVII_DSM_1m`
- Native raster: 1 m Float32, EPSG:31370, nodata `-9999`, hoogte in TAW
- Opnameperiode: 2013-2015; geen uniforme recente peildatum
- Cache: canonical raster Dataset plus WCS request/response/output checksums
- Operator: `scripts/provision_mol_dhmv.py`
- Prioriteit: P4 uitgevoerd voor Mol

The operator requests a bounded 5 m analysis copy by default so a complete
municipality remains operationally manageable while retaining the official
1 m native resolution in source metadata. The response is clipped to the exact
persisted Area before `DatasetService` stores it. DTM is bare-earth terrain;
DSM includes buildings and vegetation. Neither is exposed as water depth,
water volume or a directly measured building-height product. A drainage model
would require a separately governed hydrological processing pass.

## VMM overstromingsgevaarkaarten

- Naam: Overstromingsgevaarkaart Waterdiepte
- Uitgever: Vlaamse Milieumaatschappij
- Type: raster/modelscenario
- Toegang: publieke OGRK WCS 1.1 `https://geoservice.waterinfo.be/OGRK/wcs`
- Coverages: fluviaal/pluviaal, huidig klimaat/klimaatprojectie 2050,
  T10/T100/T1000
- Native raster: 2 m Float32, EPSG:31370, waterdiepte in centimeter
- Publicatie: 2019/2021 afhankelijk van product; scenario-identiteit is
  leidend en wordt niet als observatiedatum opgeslagen
- Cache: canonical raster Dataset plus request/response/output checksums
- Operator: `scripts/provision_mol_flood_hazards.py`
- Prioriteit: P5 scenariofundament uitgevoerd voor Mol

VMM beschrijft deze lagen als maximale lokale waterdiepte tussen wateroppervlak
en maaiveld voor een gekozen kans- en klimaatscenario. GeoIntel converteert
positieve waarden naar meter en kan een diepte-oppervlakte-integraal berekenen.
Omdat lokale maxima niet noodzakelijk op hetzelfde tijdstip optreden, is dat
geen gelijktijdig overstromingsvolume. De bron bevat evenmin bodemprofielen van
meren, kanalen of waterlopen en kan dus geen permanente waterinhoud leveren.

De officiële audit vond geen publieke, gebiedsdekkende inland-bathymetrie voor
Mol. Kust- en Noordzeeproducten vallen buiten de ruimtelijke scope. Waterinfo
stations blijven puntmetingen en GRB-watergeometrie blijft tweedimensionaal.

## Gebouwenregister

The governed operator `scripts/provision_buildings_addresses_register.py`
reads the official Digitaal Vlaanderen OGC API Features collections
`Gebouw`, `Gebouweenheid` and `Adres`. The source is continuously updated;
GeoIntel therefore creates a dated snapshot rather than claiming an annual
historical series. Raw response pages, request URLs and SHA256 checksums are
retained as operator evidence.
The operator follows advertised `next` links and, when a full page omits that
link, continues with an explicit `startIndex`. This is required because the
production address collection currently stops advertising `next` at a
service-side result window even though later offsets remain available. A short
final page is required before the snapshot is considered complete.

Building polygons are clipped against the exact persisted Area in
EPSG:31370, transformed to EPSG:4326 and persisted through the normal Dataset
upload route with `source_name=digitaal_vlaanderen_buildings_addresses_register`
and `reference_layer_name=building_registry`. Register lifecycle state remains
separate from GRB geometry. Each register building receives an explicit
`matched`, `review`, `ambiguous` or `unmatched` GRB reconciliation result; a
low-confidence or duplicate match is never silently promoted.

Building units use the official `GebouwObjectId` relation. The public address
collection does not expose that relation directly, so addresses are linked
only through an exact, unambiguous unit position or unambiguous polygon
containment. Ambiguous and unmatched rows are counted in the manifest and are
never forced to the nearest building.

The queryable layer contains building polygons, lifecycle status and aggregate
unit/address counts only. Street names, full addresses, house numbers and box
numbers are excluded. Address counts are not households, dwellings, residents
or population. Raw source pages remain restricted operator evidence and are
not exposed by the API or map.

Official endpoints and catalogues:

- https://geo.api.vlaanderen.be/Gebouwenregister/ogc/features/v1/collections/Gebouw/items
- https://geo.api.vlaanderen.be/Gebouwenregister/ogc/features/v1/collections/Gebouweenheid/items
- https://geo.api.vlaanderen.be/Adressenregister/ogc/features/v1/collections/Adres/items
- https://www.vlaanderen.be/datavindplaats/catalogus/gebouwen-en-adressenregister
- https://www.vlaanderen.be/datavindplaats/catalogus/gebouwenregister

Digitaal Vlaanderen is transitioning the download products in summer 2026.
This operator uses the current production OGC API and does not depend on the
retiring `/v2/extract` path.

## Lokale demo datasets

De repo moet ruimte voorzien voor kleine samplebestanden in `tests/fixtures/` of `datasets/raw/demo/`. Grote datasets mogen niet standaard in git.

## Sprint 7B provider registry

Sprint 7B exposes provider metadata only. It does not perform GRB WFS calls, OSM Overpass calls, downloads, or fake data generation.

| provider | authority_level | configured | layers | dataset mapping |
| --- | --- | --- | --- | --- |
| `grb` | authoritative | false | buildings, roads, parcels | `dataset_role=reference`, `source_name=grb` |
| `osm` | contextual | false | buildings, roads, water, landuse | default `dataset_role=source`, explicit reference `dataset_role=reference`, `source_name=osm` |
| `manual` | manual | true | uploaded user layers | `dataset_role=reference`, `source_name=manual` |
| `fixture` | fixture | true | demo/test fixture layers | `dataset_role=reference`, `source_name=fixture` |

Future provider output must flow through `DatasetService` and `VectorFeatureService`; providers must not write directly to `vector_features`.