Initial GeoIntel V1 foundation
This commit is contained in:
Codex
@@ -0,0 +1,151 @@
|
||||
# Canonical Domain Models
|
||||
|
||||
This document defines shared concepts. All backend services, frontend types and tests must use these definitions.
|
||||
|
||||
## Project
|
||||
|
||||
A Project is an investigation container.
|
||||
|
||||
Required behavior:
|
||||
|
||||
- contains Areas, Datasets, AnalysisRuns and Exports;
|
||||
- has a human-readable name and optional description;
|
||||
- may contain demo or real data, but the data mode must be visible;
|
||||
- does not directly store geospatial features except through child entities.
|
||||
|
||||
## Area
|
||||
|
||||
An Area is a geospatial boundary used to clip, filter and analyze data.
|
||||
|
||||
Required fields:
|
||||
|
||||
- `id`
|
||||
- `project_id`
|
||||
- `name`
|
||||
- `geometry`
|
||||
- `geometry_crs`
|
||||
- `area_m2`
|
||||
- `bounds_4326`
|
||||
- `created_at`
|
||||
|
||||
Rules:
|
||||
|
||||
- API representation is GeoJSON EPSG:4326.
|
||||
- Internal metric operations should use EPSG:31370 for Kempen/Belgium.
|
||||
- Area can be a free polygon, municipality-derived polygon or demo fixture.
|
||||
|
||||
## Dataset
|
||||
|
||||
A Dataset is a registered data source or uploaded file.
|
||||
|
||||
Dataset types:
|
||||
|
||||
- `raster`
|
||||
- `vector`
|
||||
- `reference_vector`
|
||||
- `model_output`
|
||||
- `mask`
|
||||
- `export`
|
||||
|
||||
Required behavior:
|
||||
|
||||
- original artifact is preserved;
|
||||
- metadata extraction produces a metadata record;
|
||||
- validation state is explicit;
|
||||
- derived datasets reference parents.
|
||||
|
||||
## Layer
|
||||
|
||||
A Layer is a map-renderable view of a dataset or analysis output.
|
||||
|
||||
Rules:
|
||||
|
||||
- layers have styling metadata;
|
||||
- layers do not own authoritative geometry;
|
||||
- layer visibility is frontend state, not processing state.
|
||||
|
||||
## AnalysisRun
|
||||
|
||||
An AnalysisRun is one execution of a processing pipeline.
|
||||
|
||||
Examples:
|
||||
|
||||
- raster metadata extraction;
|
||||
- vector clipping;
|
||||
- object detection;
|
||||
- segmentation;
|
||||
- QA/QC;
|
||||
- export generation.
|
||||
|
||||
Required fields:
|
||||
|
||||
- `id`
|
||||
- `project_id`
|
||||
- `area_id` optional
|
||||
- `analysis_type`
|
||||
- `status`
|
||||
- `parameters_json`
|
||||
- `started_at`
|
||||
- `finished_at`
|
||||
- `error_code` optional
|
||||
- `error_message` optional
|
||||
|
||||
## Detection
|
||||
|
||||
A Detection is a candidate object produced or imported as an analysis output.
|
||||
|
||||
Required fields:
|
||||
|
||||
- class name;
|
||||
- confidence;
|
||||
- geometry;
|
||||
- bbox;
|
||||
- source analysis run;
|
||||
- model metadata if AI-produced;
|
||||
- source tile if tiled inference was used.
|
||||
|
||||
## Segmentation
|
||||
|
||||
A Segmentation is a polygon or raster mask representing class coverage.
|
||||
|
||||
Required fields:
|
||||
|
||||
- class name;
|
||||
- geometry or mask path;
|
||||
- area_m2;
|
||||
- confidence/score if available;
|
||||
- source analysis run;
|
||||
- model metadata if AI-produced.
|
||||
|
||||
## ReferenceFeature
|
||||
|
||||
A ReferenceFeature is an authoritative or semi-authoritative feature used for validation.
|
||||
|
||||
V1 examples:
|
||||
|
||||
- GRB-like building polygons;
|
||||
- demo reference buildings;
|
||||
- OSM fallback buildings.
|
||||
|
||||
## QualityCheck
|
||||
|
||||
A QualityCheck compares candidate outputs to reference data or validates dataset integrity.
|
||||
|
||||
Required outputs:
|
||||
|
||||
- metric values;
|
||||
- method;
|
||||
- thresholds;
|
||||
- matched/unmatched features where applicable;
|
||||
- pass/fail or warning status.
|
||||
|
||||
## Export
|
||||
|
||||
An Export is a generated artifact derived from persisted state.
|
||||
|
||||
Allowed V1 exports:
|
||||
|
||||
- GeoJSON;
|
||||
- CSV metrics;
|
||||
- simple HTML/Markdown report if trivial;
|
||||
- not mandatory: complex PDF.
|
||||
@@ -0,0 +1,46 @@
|
||||
# Data Lifecycle
|
||||
|
||||
## Original dataset lifecycle
|
||||
|
||||
1. User or system registers dataset.
|
||||
2. Original file/reference is preserved.
|
||||
3. Dataset enters `UPLOADED` or `CREATED`.
|
||||
4. Validation extracts basic file and geospatial metadata.
|
||||
5. Metadata is persisted.
|
||||
6. Dataset becomes `READY`, `REQUIRES_CRS` or failure state.
|
||||
|
||||
## Derived dataset lifecycle
|
||||
|
||||
1. Operation is requested from a ready parent dataset.
|
||||
2. AnalysisRun is created.
|
||||
3. Job produces derived artifact.
|
||||
4. Derived dataset references parent dataset and analysis run.
|
||||
5. Derived metadata is extracted.
|
||||
6. Derived layer can be rendered/exported.
|
||||
|
||||
## Detection lifecycle
|
||||
|
||||
1. Detection run is created.
|
||||
2. Inputs and parameters are stored.
|
||||
3. Model/import produces candidate features.
|
||||
4. Features are georeferenced.
|
||||
5. Features are validated.
|
||||
6. Features are persisted as detections.
|
||||
7. Optional QA/QC compares them to reference features.
|
||||
8. Detections may be exported.
|
||||
|
||||
## Segmentation lifecycle
|
||||
|
||||
1. Segmentation run is created.
|
||||
2. Raster and model parameters are stored.
|
||||
3. Model/import produces masks.
|
||||
4. Masks are georeferenced.
|
||||
5. Optional polygonization creates vector features.
|
||||
6. Features and/or mask paths are persisted.
|
||||
7. QA/QC and export may run.
|
||||
|
||||
## Deletion and archival
|
||||
|
||||
- Deleting a project should not immediately physically delete files unless cleanup is requested.
|
||||
- Archival removes items from active lists but keeps reproducibility.
|
||||
- Physical cleanup must never leave dangling database references.
|
||||
@@ -0,0 +1,54 @@
|
||||
# Error Catalog
|
||||
|
||||
Use stable error codes across backend and frontend.
|
||||
|
||||
## Dataset errors
|
||||
|
||||
- `DATASET_001_INVALID_FILE`: file cannot be read.
|
||||
- `DATASET_002_UNSUPPORTED_TYPE`: file type is unsupported.
|
||||
- `DATASET_003_MISSING_CRS`: CRS cannot be determined.
|
||||
- `DATASET_004_INVALID_GEOMETRY`: geometry validation failed.
|
||||
- `DATASET_005_METADATA_FAILED`: metadata extraction failed.
|
||||
- `DATASET_006_STORAGE_FAILED`: artifact could not be stored.
|
||||
|
||||
## Raster errors
|
||||
|
||||
- `RASTER_001_NOT_GEOREFERENCED`: transform/CRS missing.
|
||||
- `RASTER_002_BAND_OUT_OF_RANGE`: selected band does not exist.
|
||||
- `RASTER_003_CLIP_FAILED`: clipping failed.
|
||||
- `RASTER_004_TILE_FAILED`: tiling failed.
|
||||
- `RASTER_005_REPROJECT_FAILED`: reprojection failed.
|
||||
|
||||
## Vector errors
|
||||
|
||||
- `VECTOR_001_EMPTY_LAYER`: no features found.
|
||||
- `VECTOR_002_SCHEMA_UNSUPPORTED`: attribute schema unsupported.
|
||||
- `VECTOR_003_CRS_TRANSFORM_FAILED`: reprojection failed.
|
||||
- `VECTOR_004_OPERATION_FAILED`: spatial operation failed.
|
||||
|
||||
## Analysis errors
|
||||
|
||||
- `ANALYSIS_001_INVALID_INPUT_STATE`: input not ready.
|
||||
- `ANALYSIS_002_JOB_FAILED`: background job failed.
|
||||
- `ANALYSIS_003_TIMEOUT`: processing timed out.
|
||||
- `ANALYSIS_004_PARTIAL_OUTPUT`: output incomplete.
|
||||
|
||||
## Detection errors
|
||||
|
||||
- `DETECTION_001_MODEL_UNAVAILABLE`: configured model unavailable.
|
||||
- `DETECTION_002_INFERENCE_FAILED`: inference failed.
|
||||
- `DETECTION_003_GEOREFERENCE_FAILED`: output could not be mapped to coordinates.
|
||||
- `DETECTION_004_NO_DETECTIONS`: valid run produced no detections.
|
||||
|
||||
## QA/QC errors
|
||||
|
||||
- `QAQC_001_REFERENCE_MISSING`: reference layer missing.
|
||||
- `QAQC_002_CANDIDATES_MISSING`: candidate features missing.
|
||||
- `QAQC_003_MATCHING_FAILED`: matching algorithm failed.
|
||||
- `QAQC_004_THRESHOLD_INVALID`: threshold invalid.
|
||||
|
||||
## Export errors
|
||||
|
||||
- `EXPORT_001_NO_OUTPUTS`: nothing to export.
|
||||
- `EXPORT_002_FORMAT_UNSUPPORTED`: unsupported export format.
|
||||
- `EXPORT_003_GENERATION_FAILED`: export failed.
|
||||
@@ -0,0 +1,58 @@
|
||||
# GIS Standards
|
||||
|
||||
## CRS standards
|
||||
|
||||
- API GeoJSON: EPSG:4326.
|
||||
- Internal Belgian metric calculations: EPSG:31370.
|
||||
- Raster native operations preserve source CRS unless reproject requested.
|
||||
- Store both source CRS and normalized CRS.
|
||||
|
||||
## Units
|
||||
|
||||
- Area: square meters internally, hectares/km² as display derivatives.
|
||||
- Length: meters internally, kilometers as display derivative.
|
||||
- Resolution: meters/pixel where projected CRS allows it; otherwise state units clearly.
|
||||
|
||||
## Geometry validity
|
||||
|
||||
All vector geometries must pass validation before becoming `READY`.
|
||||
|
||||
Allowed repair strategies:
|
||||
|
||||
1. `make_valid` where available.
|
||||
2. zero-width buffer only if documented.
|
||||
3. reject and require user correction.
|
||||
|
||||
Repairs must be recorded in metadata.
|
||||
|
||||
## Geometry type normalization
|
||||
|
||||
- Polygon inputs may become MultiPolygon.
|
||||
- LineString inputs may become MultiLineString.
|
||||
- GeometryCollections require explicit extraction or rejection.
|
||||
|
||||
## Spatial operations
|
||||
|
||||
All spatial operations must define:
|
||||
|
||||
- input CRS;
|
||||
- output CRS;
|
||||
- unit assumptions;
|
||||
- tolerance;
|
||||
- geometry repair behavior;
|
||||
- empty-result behavior.
|
||||
|
||||
## Precision
|
||||
|
||||
Store full precision in database. Simplify only derived visualization layers.
|
||||
|
||||
## Boundary handling
|
||||
|
||||
For clipping/intersections, include features that intersect area boundary. Record whether metrics use full feature geometry or clipped geometry.
|
||||
|
||||
## Source priority for V1 reference workflows
|
||||
|
||||
1. GRB-like reference fixtures / GRB when implemented.
|
||||
2. Official Flemish datasets where available.
|
||||
3. OSM fallback.
|
||||
4. User-uploaded reference layer.
|
||||
@@ -0,0 +1,29 @@
|
||||
# Canonical Performance Budgets
|
||||
|
||||
These are V1 local-development budgets. They are guidance, not hard production SLAs.
|
||||
|
||||
## API responsiveness
|
||||
|
||||
- simple GET list/detail: < 500 ms on demo data;
|
||||
- project/area creation: < 1 s;
|
||||
- metadata retrieval: < 500 ms once extracted;
|
||||
- job status polling: < 300 ms.
|
||||
|
||||
## Processing budgets on demo fixtures
|
||||
|
||||
- vector metadata extraction: < 2 s;
|
||||
- vector clip on demo area: < 5 s;
|
||||
- raster metadata extraction on small GeoTIFF: < 3 s;
|
||||
- tile manifest generation on demo raster: < 10 s;
|
||||
- QA/QC on demo building fixtures: < 5 s;
|
||||
- GeoJSON export on demo fixtures: < 5 s.
|
||||
|
||||
## UI budgets
|
||||
|
||||
- initial shell render: < 2 s in local dev;
|
||||
- map layer toggle on demo data: < 500 ms;
|
||||
- metrics panel update after API response: < 300 ms.
|
||||
|
||||
## Scaling note
|
||||
|
||||
Large real rasters or full municipality layers may exceed these budgets. In that case, the app must show async job state rather than blocking.
|
||||
@@ -0,0 +1,76 @@
|
||||
# Raster Standards
|
||||
|
||||
## Accepted V1 raster inputs
|
||||
|
||||
- GeoTIFF preferred.
|
||||
- TIFF only if georeferencing metadata exists or sidecar world file is provided.
|
||||
- JPG/PNG only as non-georeferenced preview unless georeferencing is explicitly supplied.
|
||||
|
||||
## Metadata required
|
||||
|
||||
- width;
|
||||
- height;
|
||||
- band count;
|
||||
- dtype;
|
||||
- CRS;
|
||||
- transform;
|
||||
- bounds in source CRS;
|
||||
- bounds in EPSG:4326 where possible;
|
||||
- nodata value;
|
||||
- resolution;
|
||||
- file size;
|
||||
- checksum/hash.
|
||||
|
||||
## Tile defaults
|
||||
|
||||
- tile size: 512x512 pixels;
|
||||
- overlap: 64 pixels for detection/segmentation;
|
||||
- padding strategy: reflect or constant nodata, recorded in parameters;
|
||||
- tile IDs must be stable and reproducible.
|
||||
|
||||
## Derived raster outputs
|
||||
|
||||
Derived rasters must reference:
|
||||
|
||||
- parent dataset;
|
||||
- operation;
|
||||
- parameters;
|
||||
- timestamp;
|
||||
- CRS;
|
||||
- nodata handling.
|
||||
|
||||
## Raster statistics
|
||||
|
||||
V1 statistics:
|
||||
|
||||
- min;
|
||||
- max;
|
||||
- mean;
|
||||
- std where feasible;
|
||||
- nodata count;
|
||||
- histogram bins for display.
|
||||
|
||||
## Reprojection
|
||||
|
||||
Reprojection must not overwrite originals. It creates a derived dataset.
|
||||
|
||||
## Clipping
|
||||
|
||||
Raster clipping to Area creates a derived raster and stores the clipping geometry reference.
|
||||
|
||||
## AI inference preparation
|
||||
|
||||
Before inference:
|
||||
|
||||
- dataset must be georeferenced;
|
||||
- CRS must be known;
|
||||
- transform must be available;
|
||||
- tiling parameters must be stored;
|
||||
- band selection must be explicit.
|
||||
|
||||
## Non-goals for V1
|
||||
|
||||
- perfect Sentinel processing pipeline;
|
||||
- cloud optimized GeoTIFF production;
|
||||
- large-scale distributed raster processing;
|
||||
- LiDAR-derived raster generation.
|
||||
@@ -0,0 +1,108 @@
|
||||
# State Machines
|
||||
|
||||
Use these states exactly. Do not invent ad-hoc alternatives.
|
||||
|
||||
## Dataset state machine
|
||||
|
||||
```text
|
||||
CREATED
|
||||
↓
|
||||
UPLOADING
|
||||
↓
|
||||
UPLOADED
|
||||
↓
|
||||
VALIDATING
|
||||
↓
|
||||
METADATA_EXTRACTED
|
||||
↓
|
||||
READY
|
||||
```
|
||||
|
||||
Failure states:
|
||||
|
||||
```text
|
||||
VALIDATION_FAILED
|
||||
METADATA_FAILED
|
||||
PROCESSING_FAILED
|
||||
REQUIRES_CRS
|
||||
ARCHIVED
|
||||
DELETED
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- `READY` means metadata and validation are good enough for compatible operations.
|
||||
- `REQUIRES_CRS` blocks geospatial operations but may allow file inspection.
|
||||
- `ARCHIVED` keeps data but hides it from active workflows.
|
||||
- `DELETED` means logical deletion unless physical cleanup is explicitly run.
|
||||
|
||||
## AnalysisRun state machine
|
||||
|
||||
```text
|
||||
QUEUED
|
||||
↓
|
||||
RUNNING
|
||||
↓
|
||||
SUCCEEDED
|
||||
```
|
||||
|
||||
Failure/cancel states:
|
||||
|
||||
```text
|
||||
FAILED
|
||||
CANCELLED
|
||||
PARTIAL
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- `PARTIAL` is allowed only when outputs are explicitly incomplete and marked as such.
|
||||
- Failed runs must retain logs/error code.
|
||||
- Retrying creates a new run unless retry semantics are explicitly implemented.
|
||||
|
||||
## Job state machine
|
||||
|
||||
```text
|
||||
PENDING
|
||||
↓
|
||||
STARTED
|
||||
↓
|
||||
FINISHED
|
||||
```
|
||||
|
||||
Failure states:
|
||||
|
||||
```text
|
||||
FAILED
|
||||
RETRYING
|
||||
CANCELLED
|
||||
TIMEOUT
|
||||
```
|
||||
|
||||
## Export state machine
|
||||
|
||||
```text
|
||||
REQUESTED
|
||||
↓
|
||||
GENERATING
|
||||
↓
|
||||
READY
|
||||
```
|
||||
|
||||
Failure states:
|
||||
|
||||
```text
|
||||
FAILED
|
||||
EXPIRED
|
||||
```
|
||||
|
||||
## Frontend page state model
|
||||
|
||||
Every data-driven page must handle:
|
||||
|
||||
- `empty`
|
||||
- `loading`
|
||||
- `ready`
|
||||
- `error`
|
||||
- `partial`
|
||||
- `offline/unavailable` where external service is involved.
|
||||
Reference in New Issue
Block a user