geointel/docs/DATABASE_IMPLEMENTATION_PLAN.md

Database Implementation Plan

Database: PostgreSQL + PostGIS.

Rules

• Store geometries in PostGIS with explicit SRID.
• Preserve original CRS metadata even when normalized geometry is stored as EPSG:4326 or a local projected CRS.
• Prefer UUID primary keys.
• Store large raster/mask/model files in filesystem or object storage; store metadata and paths in PostgreSQL.
• Keep analysis outputs reproducible by storing parameters JSON.

Required extensions

CREATE EXTENSION IF NOT EXISTS postgis;
CREATE EXTENSION IF NOT EXISTS postgis_topology;
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";

Core tables

projects

• id uuid primary key
• name text not null
• description text
• region text default 'Kempen'
• status text default 'active'
• created_at timestamptz
• updated_at timestamptz

areas

• id uuid primary key
• project_id uuid references projects(id)
• name text not null
• geometry geometry(MultiPolygon, 4326) not null
• original_crs text
• area_m2 double precision
• bbox geometry(Polygon, 4326)
• created_at timestamptz

Spatial index required on geometry.

datasets

• id uuid primary key
• project_id uuid references projects(id)
• area_id uuid nullable references areas(id)
• name text not null
• dataset_type text not null
• source text not null
• storage_path text
• derived_from_dataset_id uuid nullable references datasets(id)
• crs text
• bounds_json jsonb
• resolution_json jsonb
• bands_json jsonb
• metadata_json jsonb
• status text default 'created'
• created_at timestamptz

vector_features

Used for imported vector datasets and derived vector outputs when feature-level storage is needed. Original files remain source artifacts; this table is the queryable PostGIS state for vector features.

• id uuid primary key
• dataset_id uuid references datasets(id) on delete cascade
• feature_class text
• source_feature_id text
• properties_json jsonb
• geometry geometry(Geometry, 4326) not null
• created_at timestamptz

Required indexes:

• dataset_id
• GiST index on geometry

analysis_runs

• id uuid primary key
• project_id uuid references projects(id)
• area_id uuid references areas(id)
• dataset_id uuid nullable references datasets(id)
• job_id uuid nullable references jobs(id)
• analysis_type text not null
• status text not null
• model_name text nullable
• model_version text nullable
• parameters_json jsonb not null
• result_json jsonb nullable
• created_at timestamptz
• started_at timestamptz
• finished_at timestamptz
• error_message text

Analysis runs are domain lifecycle records. Jobs track execution state; analysis runs track reproducibility, model metadata, parameters and result summaries.

detections

• id uuid primary key
• project_id uuid references projects(id)
• dataset_id uuid nullable references datasets(id)
• analysis_run_id uuid nullable references analysis_runs(id)
• job_id uuid nullable references jobs(id)
• model_name text not null
• model_version text nullable
• class_name text not null
• confidence double precision not null
• geometry geometry(Geometry, 4326)
• bbox_json jsonb
• source_tile_path text nullable
• properties_json jsonb
• created_at timestamptz

Required indexes:

• project_id
• dataset_id
• analysis_run_id
• class_name
• GiST index on geometry

Sprint 8 persists detections as first-class PostGIS records. Detections are never stored only in jobs.result_json.

segmentations

• id uuid primary key
• project_id uuid references projects(id)
• dataset_id uuid nullable references datasets(id)
• job_id uuid nullable references jobs(id)
• analysis_run_id uuid nullable references analysis_runs(id)
• model_name text not null
• model_version text nullable
• class_name text not null
• confidence double precision nullable
• geometry geometry(MultiPolygon, 4326) not null
• bbox_json jsonb
• area_m2 double precision
• mask_path text
• source_tile_path text
• tile_index integer
• properties_json jsonb
• provenance_json jsonb
• created_at timestamptz

Required indexes:

• project_id
• dataset_id
• analysis_run_id
• job_id
• class_name
• GiST index on geometry

Sprint 9 persists segmentation outputs as first-class PostGIS records. Mask paths are artifact/provenance references only; map display, QA and GeoJSON output use segmentations.geometry.

metrics

• id uuid primary key
• quality_check_id uuid nullable references quality_checks(id)
• analysis_run_id uuid nullable references analysis_runs(id)
• metric_key text not null
• metric_value double precision
• metric_unit text
• label text
• metadata_json jsonb
• created_at timestamptz

Metrics may belong to a quality check, an analysis run, or both. Sprint 7A persists QA/QC metrics through quality_check_id.

quality_checks

• id uuid primary key
• project_id uuid references projects(id)
• job_id uuid nullable references jobs(id)
• analysis_run_id uuid nullable references analysis_runs(id)
• candidate_dataset_id uuid nullable references datasets(id)
• reference_dataset_id uuid references datasets(id)
• check_type text not null
• status text not null
• score double precision
• parameters_json jsonb
• findings_json jsonb
• created_at timestamptz
• completed_at timestamptz nullable

Quality checks are domain records. Jobs track execution state; quality checks track the persisted QA/QC result; metrics track individual measurements.

exports

• id uuid primary key
• project_id uuid references projects(id)
• analysis_run_id uuid nullable references analysis_runs(id)
• export_type text not null
• storage_path text not null
• metadata_json jsonb
• created_at timestamptz

Migration strategy

• Use Alembic.
• First migration creates extensions and core tables.
• Second migration adds spatial indexes.
• Seed script may create a sample project and sample area only if explicitly run.

Sprint 7B provider-to-dataset mapping

Provider integration is a contract layer only in Sprint 7B. Providers do not write directly to vector_features; future provider output must flow through DatasetService and VectorFeatureService so dataset provenance, storage metadata and feature persistence remain consistent.

• grb: maps to dataset_role='reference', source_name='grb'.
• osm: maps to dataset_role='source' by default, or dataset_role='reference' only when explicitly requested; source_name='osm'.
• manual: maps to dataset_role='reference', source_name='manual'.
• fixture: maps to dataset_role='reference', source_name='fixture'.

GRB and OSM live imports are intentionally not_configured in Sprint 7B. Manual and fixture reference datasets use existing upload and fixture flows.

Geometry normalization

• User-drawn polygons arrive as EPSG:4326.
• Uploaded vector data may arrive in another CRS; preserve original CRS and reproject to EPSG:4326 for storage.
• Area calculations should use a projected CRS suitable for Belgium, preferably EPSG:31370 or another documented Belgian projection.

Out of scope for V1

• Raster-in-database storage.
• Multi-tenant row-level security.
• User accounts.
• Full model registry tables.