geointel/docs/DATA_CATALOG.md

GeoIntel Kempen — Data Catalog v1.0

This catalog is the implementation reference for every dataset that GeoIntel may ingest, cache, analyse, validate against, or export. Codex must treat this file as source-of-truth when implementing data ingestion and analysis modules.

Data strategy

GeoIntel is built around a professional reference-data-first strategy:

1. Use official Flemish geodata where available.
2. Use OpenStreetMap as a supplemental and fallback source.
3. Use raster/satellite/aerial imagery for AI and remote-sensing pipelines.
4. Cache all expensive or external requests in PostGIS and/or filesystem storage.
5. Store provenance metadata for every derived result.

Priority sources

Priority	Source	Role	Data type	V1 status
P0	GRB / Basiskaart Vlaanderen	official reference geometry	vector	required as primary reference target
P0	User uploaded GeoTIFF / imagery	model input	raster	required
P0	User uploaded GeoJSON/Shapefile/GPKG	vector input/reference	vector	required
P1	OpenStreetMap	fallback and additional context	vector	required in V1 if GRB connector is not ready
P1	Gebouwenregister	building identifiers and metadata	vector/API	prepare architecture
P2	Sentinel-2	NDVI/NDWI/NDBI and temporal analysis	raster	roadmap
P2	DHMV / height products	DEM/DSM/slope/height analysis	raster/point cloud	roadmap
P3	LAS/LAZ point clouds	LiDAR workbench	point cloud	roadmap

GRB — Basiskaart Vlaanderen

Purpose

GRB is the main official reference layer for GeoIntel Kempen. It is used for QA/QC, validation, feature comparison, and map context. It contains accurately measured reference objects such as buildings, parcels, roads and road inrichting, watercourses, railway beds and road networks. It is a cost-free authentic source managed by Digitaal Vlaanderen.

Implementation role

GRB is not just a background map. It must become a validation and reference backbone:

• Compare AI building detections with official building footprints.
• Identify false positives and false negatives.
• Create QA/QC dashboards.
• Provide high-quality vector context for map overlays.
• Support change-detection workflows where official snapshots are available.

Required layer groups

The exact service layer names must be discovered during implementation through the GRB WFS/WMS capabilities endpoint or downloaded package metadata. The application-level canonical layer groups are:

Canonical group	Expected geometry	Usage
grb_buildings	polygon	building reference, QA/QC, footprint analysis
grb_roads	line/polygon	road/infrastructure context, pressure metrics
grb_water	line/polygon	water context, water proximity, hydro overlays
grb_railways	line/polygon	infrastructure barrier/context
grb_parcels	polygon	optional parcel context, not required for V1

Required normalized fields

For every imported GRB feature, normalize at least:

id: internal UUID
source: "GRB"
source_layer: original layer name
source_feature_id: original feature identifier when available
canonical_group: grb_buildings | grb_roads | grb_water | grb_railways | grb_parcels
geometry: PostGIS geometry
geometry_type: Polygon | MultiPolygon | LineString | MultiLineString
crs_original: original CRS
crs_storage: EPSG:31370 or EPSG:4326 depending DB policy
attributes_json: raw attributes
fetched_at: timestamp
source_updated_at: timestamp when available
bbox: calculated bounds
area_m2: calculated for polygon features
length_m: calculated for linear features

Access strategy

V1 may implement either:

1. WFS area-of-interest fetch, preferred for targeted analysis.
2. Downloaded package import for cached offline analysis.
3. OSM fallback when GRB access is not implemented yet.

Codex may build an abstraction ReferenceDataProvider so GRB and OSM can both satisfy the same downstream analysis contracts.

User uploaded raster imagery

Supported formats

• GeoTIFF .tif, .tiff
• Cloud Optimized GeoTIFF if possible
• JPEG/PNG only if accompanied by georeferencing metadata or used as non-geospatial demo input

Required metadata

dataset_id
file_path
file_size_bytes
raster_driver
width
height
band_count
crs
transform
bounds
resolution_x
resolution_y
nodata_values
dtype_per_band
statistics_per_band
color_interpretation
is_georeferenced
created_at

Required uses

• Map overlay preview.
• Raster metadata inspection.
• Clipping to area of interest.
• Tiling for object detection and segmentation.
• Remote sensing index calculations when bands support it.

User uploaded vector data

Supported formats

• GeoJSON
• Shapefile ZIP
• GeoPackage .gpkg
• KML/KMZ later

Required metadata

dataset_id
source_filename
format
layer_names
feature_count
geometry_types
crs
bounds
attributes_schema
created_at

Required uses

• Reference data.
• Manual annotations.
• User-supplied areas of interest.
• Comparison layers for QA/QC.

OpenStreetMap

Purpose

OSM is used for fast open-data enrichment and fallback when official sources are not yet implemented.

Canonical groups

• osm_buildings
• osm_roads
• osm_water
• osm_green
• osm_landuse
• osm_poi

Required fields

osm_id
tags_json
canonical_group
geometry
area_m2
length_m
fetched_at

Sentinel-2

Purpose

Sentinel-2 supports remote sensing indices and temporal change analysis.

Required indices later

• NDVI = (NIR - Red) / (NIR + Red)
• NDWI = (Green - NIR) / (Green + NIR)
• NDBI = (SWIR - NIR) / (SWIR + NIR)

V1 rule

Prepare architecture, but do not block V1 on a full Sentinel downloader.

DHMV / DEM / DSM

Purpose

Height intelligence: slope, low points, terrain context, DSM minus DEM for object height approximations.

Required outputs later

• min_elevation_m
• max_elevation_m
• mean_elevation_m
• slope_mean_deg
• low_point_polygons
• height_profile_samples

Data provenance requirements

Every derived dataset must keep:

source_dataset_ids
processing_pipeline
parameters_json
software_versions
created_at
created_by_job_id
crs
quality_flags

V1 non-negotiables

• Do not hardcode temporary demo assumptions into data models.
• Every dataset must have metadata.
• Every derived output must reference its source dataset and analysis run.
• AI outputs must be stored as geospatial outputs, not only image overlays.