Initial GeoIntel V1 foundation

2026-06-16 23:36:32 +02:00
commit 6ea3586a3e
605 changed files with 45284 additions and 0 deletions
@@ -0,0 +1,255 @@
+# GeoIntel Kempen — Raster Operations Specification v1.0
+
+Raster operations are implemented with Rasterio, GDAL-compatible tooling, NumPy and optional OpenCV.
+
+## Global requirements
+
+- Never assume band order without metadata or explicit user selection.
+- Preserve georeferencing for all derived rasters.
+- Use Cloud Optimized GeoTIFF when generating large outputs if practical.
+- Store previews separately from analytical rasters.
+- Keep nodata values and masks explicit.
+
+## Required V1 operations
+
+## 1. Metadata extraction
+
+Input: raster file.
+
+Output:
+
+```json
+{
+  "driver": "GTiff",
+  "width": 4096,
+  "height": 4096,
+  "band_count": 4,
+  "crs": "EPSG:31370",
+  "bounds": [xmin, ymin, xmax, ymax],
+  "resolution": [0.25, 0.25],
+  "dtype": ["uint8"],
+  "nodata": [null],
+  "transform": [..]
+}
+```
+
+## 2. Band statistics
+
+For each band:
+
+- min
+- max
+- mean
+- std
+- nodata ratio
+- histogram bins
+- nodata count
+- valid pixel count
+- dtype
+- optional histogram bins approximation
+
+## 3. Preview generation
+
+Generate browser-friendly previews:
+
+- PNG preview for full raster or overview.
+- Optional tile pyramid later.
+- Support contrast stretch.
+
+## 4. Clip by polygon
+
+Input:
+
+- raster dataset
+- area polygon
+
+Output:
+
+- clipped GeoTIFF
+- metadata
+- preview
+
+## 5. Reproject
+
+Input:
+
+- raster dataset
+- target CRS
+- target CRS default: `EPSG:31370` when not explicitly set in request
+- resampling method: nearest, bilinear, cubic
+
+Output:
+
+- reprojected GeoTIFF
+
+Failure behavior:
+
+- if rasterio is unavailable, return dependency-aware error (`RASTER_PROCESSING_UNAVAILABLE`)
+- if source CRS is missing, return invalid CRS error
+
+## 6. Tile generation for AI
+
+Input:
+
+- raster dataset
+- tile size, default 512 (configured in runtime defaults)
+- overlap, default 64
+- optional area polygon
+
+Output:
+
+```json
+{
+  "tile_set_id": "uuid",
+  "source_dataset_id": "uuid",
+  "source_raster_id": "uuid",
+  "bounds": [xmin, ymin, xmax, ymax],
+  "tile_size": 512,
+  "overlap": 64,
+  "parameters": {},
+  "count": 16,
+  "tile_paths": ["storage/tiles/.../tile_0001.tif"],
+  "tiles": [
+    {
+      "path": "storage/tiles/.../tile_0001.tif",
+      "pixel_window": [x, y, width, height],
+      "bounds": [xmin, ymin, xmax, ymax],
+      "transform": []
+    }
+  ],
+  "ai_inference": false,
+  "tile_server": null
+}
+
+```
+
+Failure behavior:
+
+- if rasterio is unavailable: dependency-aware error
+- if no raster tiles are generated: empty-result failure
+
+## 7. Index calculations
+
+### NDVI
+
+Required inputs:
+
+- red band index
+- NIR band index
+- output name (optional)
+- formula: `(nir - red) / (nir + red)`
+- nodata/NaN handling: invalid pixels and division-by-zero values are written as `NaN`
+- output dtype: `float32`
+- provenance fields:
+  - `source_dataset_id`
+  - `operation`
+  - `band_mapping`
+  - `formula`
+  - `output_dtype`
+  - `nodata_strategy`
+  - `value_range_note`
+- failure behavior:
+  - band indices must be positive integers
+  - band indices must exist in source dataset
+  - `RASTER_PROCESSING_UNAVAILABLE` when rasterio/numpy missing
+
+Formula:
+
+```text
+NDVI = (NIR - Red) / (NIR + Red)
+```
+
+### NDWI
+
+Required inputs:
+
+- NIR band index
+- green band index
+- output name (optional)
+- formula: `(nir - green) / (nir + green)`
+- same provenance strategy as NDVI
+
+Formula:
+
+```text
+NDWI = (NIR - Green) / (NIR + Green)
+```
+
+### NDBI
+
+Required inputs:
+
+- SWIR band index
+- NIR band index
+- output name (optional)
+- formula: `(swir - nir) / (swir + nir)`
+- same provenance strategy as NDVI
+
+Formula:
+
+```text
+NDBI = (SWIR - NIR) / (SWIR + NIR)
+```
+
+Derived metadata contract:
+
+- `operation`: one of `raster.ndvi`, `raster.ndwi`, `raster.ndbi`
+- `source_dataset_id`
+- `band_mapping`
+- `formula`
+- `output_dtype`
+- `nodata_strategy`
+- `value_range_note`
+- `created_at`
+- `output_dataset_id`
+- `path`
+
+## 8. Threshold to mask
+
+Input:
+
+- raster/index
+- threshold operator
+- threshold value
+
+Output:
+
+- binary mask raster
+- optional vector polygons
+
+## Storage output policy
+
+Derived rasters go to:
+
+```text
+storage/rasters/derived/{project_id}/{dataset_id}/
+```
+
+Tiles go to:
+
+```text
+storage/tiles/{project_id}/{dataset_id}/{tile_set_id}/
+```
+
+Previews go to:
+
+```text
+storage/previews/{project_id}/{dataset_id}/
+```
+
+## API shape
+
+```http
+GET /datasets/{id}/raster/metadata
+POST /datasets/{id}/raster/clip
+POST /datasets/{id}/raster/reproject
+POST /datasets/{id}/raster/tile
+POST /datasets/{id}/raster/indices/ndvi
+POST /datasets/{id}/raster/indices/ndwi
+POST /datasets/{id}/raster/indices/ndbi
+POST /datasets/{id}/raster/threshold
+```
+
+## V1 target
+
+V1 must implement metadata, preview, clip and tile generation. NDVI/NDWI/NDBI are implemented on local raster files with explicit band selection.