Initial GeoIntel V1 foundation
This commit is contained in:
Codex
@@ -0,0 +1,227 @@
|
||||
# GeoIntel Kempen — Data Catalog v1.0
|
||||
|
||||
This catalog is the implementation reference for every dataset that GeoIntel may ingest, cache, analyse, validate against, or export. Codex must treat this file as source-of-truth when implementing data ingestion and analysis modules.
|
||||
|
||||
## Data strategy
|
||||
|
||||
GeoIntel is built around a professional reference-data-first strategy:
|
||||
|
||||
1. Use official Flemish geodata where available.
|
||||
2. Use OpenStreetMap as a supplemental and fallback source.
|
||||
3. Use raster/satellite/aerial imagery for AI and remote-sensing pipelines.
|
||||
4. Cache all expensive or external requests in PostGIS and/or filesystem storage.
|
||||
5. Store provenance metadata for every derived result.
|
||||
|
||||
## Priority sources
|
||||
|
||||
| Priority | Source | Role | Data type | V1 status |
|
||||
|---|---|---|---|---|
|
||||
| P0 | GRB / Basiskaart Vlaanderen | official reference geometry | vector | required as primary reference target |
|
||||
| P0 | User uploaded GeoTIFF / imagery | model input | raster | required |
|
||||
| P0 | User uploaded GeoJSON/Shapefile/GPKG | vector input/reference | vector | required |
|
||||
| P1 | OpenStreetMap | fallback and additional context | vector | required in V1 if GRB connector is not ready |
|
||||
| P1 | Gebouwenregister | building identifiers and metadata | vector/API | prepare architecture |
|
||||
| P2 | Sentinel-2 | NDVI/NDWI/NDBI and temporal analysis | raster | roadmap |
|
||||
| P2 | DHMV / height products | DEM/DSM/slope/height analysis | raster/point cloud | roadmap |
|
||||
| P3 | LAS/LAZ point clouds | LiDAR workbench | point cloud | roadmap |
|
||||
|
||||
## GRB — Basiskaart Vlaanderen
|
||||
|
||||
### Purpose
|
||||
GRB is the main official reference layer for GeoIntel Kempen. It is used for QA/QC, validation, feature comparison, and map context. It contains accurately measured reference objects such as buildings, parcels, roads and road inrichting, watercourses, railway beds and road networks. It is a cost-free authentic source managed by Digitaal Vlaanderen.
|
||||
|
||||
### Implementation role
|
||||
GRB is not just a background map. It must become a validation and reference backbone:
|
||||
|
||||
- Compare AI building detections with official building footprints.
|
||||
- Identify false positives and false negatives.
|
||||
- Create QA/QC dashboards.
|
||||
- Provide high-quality vector context for map overlays.
|
||||
- Support change-detection workflows where official snapshots are available.
|
||||
|
||||
### Required layer groups
|
||||
The exact service layer names must be discovered during implementation through the GRB WFS/WMS capabilities endpoint or downloaded package metadata. The application-level canonical layer groups are:
|
||||
|
||||
| Canonical group | Expected geometry | Usage |
|
||||
|---|---|---|
|
||||
| `grb_buildings` | polygon | building reference, QA/QC, footprint analysis |
|
||||
| `grb_roads` | line/polygon | road/infrastructure context, pressure metrics |
|
||||
| `grb_water` | line/polygon | water context, water proximity, hydro overlays |
|
||||
| `grb_railways` | line/polygon | infrastructure barrier/context |
|
||||
| `grb_parcels` | polygon | optional parcel context, not required for V1 |
|
||||
|
||||
### Required normalized fields
|
||||
For every imported GRB feature, normalize at least:
|
||||
|
||||
```yaml
|
||||
id: internal UUID
|
||||
source: "GRB"
|
||||
source_layer: original layer name
|
||||
source_feature_id: original feature identifier when available
|
||||
canonical_group: grb_buildings | grb_roads | grb_water | grb_railways | grb_parcels
|
||||
geometry: PostGIS geometry
|
||||
geometry_type: Polygon | MultiPolygon | LineString | MultiLineString
|
||||
crs_original: original CRS
|
||||
crs_storage: EPSG:31370 or EPSG:4326 depending DB policy
|
||||
attributes_json: raw attributes
|
||||
fetched_at: timestamp
|
||||
source_updated_at: timestamp when available
|
||||
bbox: calculated bounds
|
||||
area_m2: calculated for polygon features
|
||||
length_m: calculated for linear features
|
||||
```
|
||||
|
||||
### Access strategy
|
||||
V1 may implement either:
|
||||
|
||||
1. WFS area-of-interest fetch, preferred for targeted analysis.
|
||||
2. Downloaded package import for cached offline analysis.
|
||||
3. OSM fallback when GRB access is not implemented yet.
|
||||
|
||||
Codex may build an abstraction `ReferenceDataProvider` so GRB and OSM can both satisfy the same downstream analysis contracts.
|
||||
|
||||
## User uploaded raster imagery
|
||||
|
||||
### Supported formats
|
||||
|
||||
- GeoTIFF `.tif`, `.tiff`
|
||||
- Cloud Optimized GeoTIFF if possible
|
||||
- JPEG/PNG only if accompanied by georeferencing metadata or used as non-geospatial demo input
|
||||
|
||||
### Required metadata
|
||||
|
||||
```yaml
|
||||
dataset_id
|
||||
file_path
|
||||
file_size_bytes
|
||||
raster_driver
|
||||
width
|
||||
height
|
||||
band_count
|
||||
crs
|
||||
transform
|
||||
bounds
|
||||
resolution_x
|
||||
resolution_y
|
||||
nodata_values
|
||||
dtype_per_band
|
||||
statistics_per_band
|
||||
color_interpretation
|
||||
is_georeferenced
|
||||
created_at
|
||||
```
|
||||
|
||||
### Required uses
|
||||
|
||||
- Map overlay preview.
|
||||
- Raster metadata inspection.
|
||||
- Clipping to area of interest.
|
||||
- Tiling for object detection and segmentation.
|
||||
- Remote sensing index calculations when bands support it.
|
||||
|
||||
## User uploaded vector data
|
||||
|
||||
### Supported formats
|
||||
|
||||
- GeoJSON
|
||||
- Shapefile ZIP
|
||||
- GeoPackage `.gpkg`
|
||||
- KML/KMZ later
|
||||
|
||||
### Required metadata
|
||||
|
||||
```yaml
|
||||
dataset_id
|
||||
source_filename
|
||||
format
|
||||
layer_names
|
||||
feature_count
|
||||
geometry_types
|
||||
crs
|
||||
bounds
|
||||
attributes_schema
|
||||
created_at
|
||||
```
|
||||
|
||||
### Required uses
|
||||
|
||||
- Reference data.
|
||||
- Manual annotations.
|
||||
- User-supplied areas of interest.
|
||||
- Comparison layers for QA/QC.
|
||||
|
||||
## OpenStreetMap
|
||||
|
||||
### Purpose
|
||||
OSM is used for fast open-data enrichment and fallback when official sources are not yet implemented.
|
||||
|
||||
### Canonical groups
|
||||
|
||||
- `osm_buildings`
|
||||
- `osm_roads`
|
||||
- `osm_water`
|
||||
- `osm_green`
|
||||
- `osm_landuse`
|
||||
- `osm_poi`
|
||||
|
||||
### Required fields
|
||||
|
||||
```yaml
|
||||
osm_id
|
||||
tags_json
|
||||
canonical_group
|
||||
geometry
|
||||
area_m2
|
||||
length_m
|
||||
fetched_at
|
||||
```
|
||||
|
||||
## Sentinel-2
|
||||
|
||||
### Purpose
|
||||
Sentinel-2 supports remote sensing indices and temporal change analysis.
|
||||
|
||||
### Required indices later
|
||||
|
||||
- NDVI = `(NIR - Red) / (NIR + Red)`
|
||||
- NDWI = `(Green - NIR) / (Green + NIR)`
|
||||
- NDBI = `(SWIR - NIR) / (SWIR + NIR)`
|
||||
|
||||
### V1 rule
|
||||
Prepare architecture, but do not block V1 on a full Sentinel downloader.
|
||||
|
||||
## DHMV / DEM / DSM
|
||||
|
||||
### Purpose
|
||||
Height intelligence: slope, low points, terrain context, DSM minus DEM for object height approximations.
|
||||
|
||||
### Required outputs later
|
||||
|
||||
- `min_elevation_m`
|
||||
- `max_elevation_m`
|
||||
- `mean_elevation_m`
|
||||
- `slope_mean_deg`
|
||||
- `low_point_polygons`
|
||||
- `height_profile_samples`
|
||||
|
||||
## Data provenance requirements
|
||||
|
||||
Every derived dataset must keep:
|
||||
|
||||
```yaml
|
||||
source_dataset_ids
|
||||
processing_pipeline
|
||||
parameters_json
|
||||
software_versions
|
||||
created_at
|
||||
created_by_job_id
|
||||
crs
|
||||
quality_flags
|
||||
```
|
||||
|
||||
## V1 non-negotiables
|
||||
|
||||
- Do not hardcode temporary demo assumptions into data models.
|
||||
- Every dataset must have metadata.
|
||||
- Every derived output must reference its source dataset and analysis run.
|
||||
- AI outputs must be stored as geospatial outputs, not only image overlays.
|
||||
Reference in New Issue
Block a user