228 lines
6.4 KiB
Markdown
228 lines
6.4 KiB
Markdown
# GeoIntel Kempen — Data Catalog v1.0
|
|
|
|
This catalog is the implementation reference for every dataset that GeoIntel may ingest, cache, analyse, validate against, or export. Codex must treat this file as source-of-truth when implementing data ingestion and analysis modules.
|
|
|
|
## Data strategy
|
|
|
|
GeoIntel is built around a professional reference-data-first strategy:
|
|
|
|
1. Use official Flemish geodata where available.
|
|
2. Use OpenStreetMap as a supplemental and fallback source.
|
|
3. Use raster/satellite/aerial imagery for AI and remote-sensing pipelines.
|
|
4. Cache all expensive or external requests in PostGIS and/or filesystem storage.
|
|
5. Store provenance metadata for every derived result.
|
|
|
|
## Priority sources
|
|
|
|
| Priority | Source | Role | Data type | V1 status |
|
|
|---|---|---|---|---|
|
|
| P0 | GRB / Basiskaart Vlaanderen | official reference geometry | vector | required as primary reference target |
|
|
| P0 | User uploaded GeoTIFF / imagery | model input | raster | required |
|
|
| P0 | User uploaded GeoJSON/Shapefile/GPKG | vector input/reference | vector | required |
|
|
| P1 | OpenStreetMap | fallback and additional context | vector | required in V1 if GRB connector is not ready |
|
|
| P1 | Gebouwenregister | building identifiers and metadata | vector/API | prepare architecture |
|
|
| P2 | Sentinel-2 | NDVI/NDWI/NDBI and temporal analysis | raster | roadmap |
|
|
| P2 | DHMV / height products | DEM/DSM/slope/height analysis | raster/point cloud | roadmap |
|
|
| P3 | LAS/LAZ point clouds | LiDAR workbench | point cloud | roadmap |
|
|
|
|
## GRB — Basiskaart Vlaanderen
|
|
|
|
### Purpose
|
|
GRB is the main official reference layer for GeoIntel Kempen. It is used for QA/QC, validation, feature comparison, and map context. It contains accurately measured reference objects such as buildings, parcels, roads and road inrichting, watercourses, railway beds and road networks. It is a cost-free authentic source managed by Digitaal Vlaanderen.
|
|
|
|
### Implementation role
|
|
GRB is not just a background map. It must become a validation and reference backbone:
|
|
|
|
- Compare AI building detections with official building footprints.
|
|
- Identify false positives and false negatives.
|
|
- Create QA/QC dashboards.
|
|
- Provide high-quality vector context for map overlays.
|
|
- Support change-detection workflows where official snapshots are available.
|
|
|
|
### Required layer groups
|
|
The exact service layer names must be discovered during implementation through the GRB WFS/WMS capabilities endpoint or downloaded package metadata. The application-level canonical layer groups are:
|
|
|
|
| Canonical group | Expected geometry | Usage |
|
|
|---|---|---|
|
|
| `grb_buildings` | polygon | building reference, QA/QC, footprint analysis |
|
|
| `grb_roads` | line/polygon | road/infrastructure context, pressure metrics |
|
|
| `grb_water` | line/polygon | water context, water proximity, hydro overlays |
|
|
| `grb_railways` | line/polygon | infrastructure barrier/context |
|
|
| `grb_parcels` | polygon | optional parcel context, not required for V1 |
|
|
|
|
### Required normalized fields
|
|
For every imported GRB feature, normalize at least:
|
|
|
|
```yaml
|
|
id: internal UUID
|
|
source: "GRB"
|
|
source_layer: original layer name
|
|
source_feature_id: original feature identifier when available
|
|
canonical_group: grb_buildings | grb_roads | grb_water | grb_railways | grb_parcels
|
|
geometry: PostGIS geometry
|
|
geometry_type: Polygon | MultiPolygon | LineString | MultiLineString
|
|
crs_original: original CRS
|
|
crs_storage: EPSG:31370 or EPSG:4326 depending DB policy
|
|
attributes_json: raw attributes
|
|
fetched_at: timestamp
|
|
source_updated_at: timestamp when available
|
|
bbox: calculated bounds
|
|
area_m2: calculated for polygon features
|
|
length_m: calculated for linear features
|
|
```
|
|
|
|
### Access strategy
|
|
V1 may implement either:
|
|
|
|
1. WFS area-of-interest fetch, preferred for targeted analysis.
|
|
2. Downloaded package import for cached offline analysis.
|
|
3. OSM fallback when GRB access is not implemented yet.
|
|
|
|
Codex may build an abstraction `ReferenceDataProvider` so GRB and OSM can both satisfy the same downstream analysis contracts.
|
|
|
|
## User uploaded raster imagery
|
|
|
|
### Supported formats
|
|
|
|
- GeoTIFF `.tif`, `.tiff`
|
|
- Cloud Optimized GeoTIFF if possible
|
|
- JPEG/PNG only if accompanied by georeferencing metadata or used as non-geospatial demo input
|
|
|
|
### Required metadata
|
|
|
|
```yaml
|
|
dataset_id
|
|
file_path
|
|
file_size_bytes
|
|
raster_driver
|
|
width
|
|
height
|
|
band_count
|
|
crs
|
|
transform
|
|
bounds
|
|
resolution_x
|
|
resolution_y
|
|
nodata_values
|
|
dtype_per_band
|
|
statistics_per_band
|
|
color_interpretation
|
|
is_georeferenced
|
|
created_at
|
|
```
|
|
|
|
### Required uses
|
|
|
|
- Map overlay preview.
|
|
- Raster metadata inspection.
|
|
- Clipping to area of interest.
|
|
- Tiling for object detection and segmentation.
|
|
- Remote sensing index calculations when bands support it.
|
|
|
|
## User uploaded vector data
|
|
|
|
### Supported formats
|
|
|
|
- GeoJSON
|
|
- Shapefile ZIP
|
|
- GeoPackage `.gpkg`
|
|
- KML/KMZ later
|
|
|
|
### Required metadata
|
|
|
|
```yaml
|
|
dataset_id
|
|
source_filename
|
|
format
|
|
layer_names
|
|
feature_count
|
|
geometry_types
|
|
crs
|
|
bounds
|
|
attributes_schema
|
|
created_at
|
|
```
|
|
|
|
### Required uses
|
|
|
|
- Reference data.
|
|
- Manual annotations.
|
|
- User-supplied areas of interest.
|
|
- Comparison layers for QA/QC.
|
|
|
|
## OpenStreetMap
|
|
|
|
### Purpose
|
|
OSM is used for fast open-data enrichment and fallback when official sources are not yet implemented.
|
|
|
|
### Canonical groups
|
|
|
|
- `osm_buildings`
|
|
- `osm_roads`
|
|
- `osm_water`
|
|
- `osm_green`
|
|
- `osm_landuse`
|
|
- `osm_poi`
|
|
|
|
### Required fields
|
|
|
|
```yaml
|
|
osm_id
|
|
tags_json
|
|
canonical_group
|
|
geometry
|
|
area_m2
|
|
length_m
|
|
fetched_at
|
|
```
|
|
|
|
## Sentinel-2
|
|
|
|
### Purpose
|
|
Sentinel-2 supports remote sensing indices and temporal change analysis.
|
|
|
|
### Required indices later
|
|
|
|
- NDVI = `(NIR - Red) / (NIR + Red)`
|
|
- NDWI = `(Green - NIR) / (Green + NIR)`
|
|
- NDBI = `(SWIR - NIR) / (SWIR + NIR)`
|
|
|
|
### V1 rule
|
|
Prepare architecture, but do not block V1 on a full Sentinel downloader.
|
|
|
|
## DHMV / DEM / DSM
|
|
|
|
### Purpose
|
|
Height intelligence: slope, low points, terrain context, DSM minus DEM for object height approximations.
|
|
|
|
### Required outputs later
|
|
|
|
- `min_elevation_m`
|
|
- `max_elevation_m`
|
|
- `mean_elevation_m`
|
|
- `slope_mean_deg`
|
|
- `low_point_polygons`
|
|
- `height_profile_samples`
|
|
|
|
## Data provenance requirements
|
|
|
|
Every derived dataset must keep:
|
|
|
|
```yaml
|
|
source_dataset_ids
|
|
processing_pipeline
|
|
parameters_json
|
|
software_versions
|
|
created_at
|
|
created_by_job_id
|
|
crs
|
|
quality_flags
|
|
```
|
|
|
|
## V1 non-negotiables
|
|
|
|
- Do not hardcode temporary demo assumptions into data models.
|
|
- Every dataset must have metadata.
|
|
- Every derived output must reference its source dataset and analysis run.
|
|
- AI outputs must be stored as geospatial outputs, not only image overlays.
|