Jens/geointel
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial GeoIntel V1 foundation
GeoIntel CI / docs-smoke (push) Has been cancelled
Details
GeoIntel CI / contract-smoke (push) Has been cancelled
Details

Browse Source
This commit is contained in:
Codex
2026-06-16 23:36:32 +02:00
commit 6ea3586a3e
605 changed files with 45284 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.env.example
+22
View File
@@ -0,0 +1,22 @@
# Backend
GEOINTEL_ENV=development
GEOINTEL_API_PREFIX=/api/v1
DATABASE_URL=postgresql+psycopg://geointel:geointel@localhost:5432/geointel?connect_timeout=1
STORAGE_ROOT=./storage
MAX_UPLOAD_MB=500
CORS_ORIGINS=http://localhost:1202,http://127.0.0.1:1202
YOLO_ENABLED=false
YOLO_MODEL_PATH=
YOLO_MODEL_VERSION=
YOLO_MAX_TILES=100
ENABLE_GRB_WFS=false
GRB_WFS_URL=
OSM_OVERPASS_URL=https://overpass-api.de/api/interpreter
# Install backend raster dependencies when needed:
# python -m pip install rasterio
# Frontend
VITE_API_BASE_URL=
VITE_API_PROXY_TARGET=http://localhost:8000
VITE_MAP_STYLE_URL=https://demotiles.maplibre.org/style.json
.gitattributes
+12
View File
@@ -0,0 +1,12 @@
*.sh text eol=lf
*.py text eol=lf
*.yml text eol=lf
*.yaml text eol=lf
*.toml text eol=lf
*.ini text eol=lf
Dockerfile text eol=lf
*.md text eol=lf
*.tsx text eol=lf
*.ts text eol=lf
*.css text eol=lf
*.json text eol=lf
.github/ISSUE_TEMPLATE/bug_report.md
+43
View File
@@ -0,0 +1,43 @@
---
name: Bug report
about: Report a reproducible defect
---
## Summary
## Steps to reproduce
1.
2.
3.
## Expected behavior
## Actual behavior
## Affected module
- [ ] Backend
- [ ] Frontend
- [ ] Database
- [ ] Raster
- [ ] Vector
- [ ] AI/Detection
- [ ] QA/QC
- [ ] Export
- [ ] Docs
## Logs/screenshots
## Data involved
- Dataset:
- CRS:
- Geometry type:
## Risk
- [ ] Blocks build
- [ ] Data correctness issue
- [ ] UX issue
- [ ] Documentation issue
.github/ISSUE_TEMPLATE/feature_request.md
+26
View File
@@ -0,0 +1,26 @@
---
name: Feature request
about: Propose an improvement without breaking scope
---
## Problem
## Proposed solution
## Scope category
- [ ] V1 in scope
- [ ] V1 adjacent
- [ ] V2+
- [ ] RFC required
## Affected modules
## Acceptance criteria
- [ ]
- [ ]
## Risks
## Notes
.github/pull_request_template.md
+21
View File
@@ -0,0 +1,21 @@
# Summary
## Changed files
## Acceptance criteria
- [ ] Meets pass prompt
- [ ] Meets M6 quality gates
- [ ] Tests run
- [ ] Docs updated
- [ ] No architecture drift
## Tests
```bash
# commands
```
## Known limitations
## Next pass recommendation
.github/workflows/ci.yml
+30
View File
@@ -0,0 +1,30 @@
name: GeoIntel CI
on:
push:
branches: [ main, develop, 'build/**' ]
pull_request:
branches: [ main, develop ]
jobs:
docs-smoke:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Validate repository docs
run: |
python scripts/smoke_docs.py
python scripts/validate_fixtures.py
contract-smoke:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Run contract smoke checks
run: python scripts/smoke_contracts.py
.gitignore
+40
View File
@@ -0,0 +1,40 @@
# Python
__pycache__/
*.py[cod]
.venv/
venv/
.env
*.egg-info/
# Node
node_modules/
dist/
build/
*.tsbuildinfo
# Large local data
/datasets/raw/*
/datasets/processed/*
/datasets/cache/*
/storage/uploads/*
/storage/tiles/*
/storage/masks/*
/storage/reports/*
/storage/exports/*
/storage/rasters/*
/exports/*
/models/*
/backend/storage/uploads/*
/backend/storage/tiles/*
/backend/storage/masks/*
/backend/storage/reports/*
/backend/storage/exports/*
# Keep folder placeholders
!**/.gitkeep
!**/README.md
# OS/editor
.DS_Store
.vscode/
.idea/
.gitkeep
View File
AGENTS.md
+42
View File
@@ -0,0 +1,42 @@
# AI Agent Instructions for GeoIntel
## Project identity
GeoIntel Kempen is a GeoAI Workbench, not a generic CRUD app and not a generic dashboard.
## Required behavior
- Read `docs/CODEX_BOOTSTRAP_PROMPT.md` first.
- Respect `docs/V1_SCOPE_FREEZE.md`.
- Use `docs/API_CONTRACTS.md` as source of truth for endpoints.
- Use `docs/DATABASE_IMPLEMENTATION_PLAN.md` as source of truth for persistence.
- Use `docs/DEFINITION_OF_DONE.md` to decide whether work is complete.
## Agent roles
### Architecture Agent
Owns repository layout, API contracts, database migrations and service boundaries.
### GIS Agent
Owns GeoPandas, Shapely, Rasterio, CRS, clipping, buffering, spatial joins and metadata extraction.
### AI Agent
Owns YOLO/SAM abstractions, inference contracts, model configuration, detection/segmentation persistence and `not_configured` behavior.
### QA Agent
Owns tests, QA/QC metrics, regression checks and acceptance criteria.
### Frontend Agent
Owns React, TypeScript, MapLibre, API client, UI states and workbench UX.
## Never do this
- Do not fake production AI outputs.
- Do not silently skip geospatial validation.
- Do not add auth/multi-user/LiDAR/training before V1 foundation is stable.
- Do not remove documentation to avoid conflicts.
CHANGELOG.md
+310
View File
@@ -0,0 +1,310 @@
# M13 — Codex Optimization Pack
- Added reusable Codex skills under `skills/`.
- Added prompt discipline, token/context budget policy, secrets policy and parallel-agent strategy.
- Added M13 day-one optimized master prompt and pass completion report prompt.
- Added M13 validation script and included it in readiness checks.
# Changelog
## Release hardening audit pass (2026-06-15)
- Replaced remaining backend `datetime.utcnow()` usage with timezone-aware UTC timestamps.
- Verified the affected backend tests with `DeprecationWarning` promoted to errors.
- Split the frontend production bundle into explicit app, React vendor and MapLibre vendor chunks.
- Raised the Vite chunk warning threshold to match the isolated MapLibre GIS dependency rather than masking app-code growth.
- Hardened the main readiness gate so backend deprecation warnings fail release readiness.
- Added API contract smoke validation to the main readiness gate.
- Hardened the pass-end placeholder scan to skip dependency, build-output and bytecode-cache folders.
- Added backend tests for the release/readiness script expectations.
- Updated `docs/TODO.md` with current implementation status while preserving older planning context.
- No API contracts, migrations, product features, AI dependencies or provider behavior were changed.
## Docker runtime hardening (2026-06-15)
- Fixed the backend Docker build by copying `README.md` and `app/` before `pip install .`.
- Removed mandatory Compose `.env` references so `docker compose up` works with checked-in local defaults.
- Published the Docker Compose frontend on host port `1202`.
- Added backend CORS defaults for `http://localhost:1202` and `http://127.0.0.1:1202`.
- Stopped publishing PostGIS on host port `5432`; backend uses Docker-internal `db:5432`.
- Added a PostGIS healthcheck and made the backend wait for a healthy database.
- Added a backend Docker start script that retries a real SQL connection before running migrations, avoiding first-start database race conditions.
- Made the backend container run `alembic upgrade head` before starting Uvicorn.
- Added backend/frontend `.dockerignore` files to keep dependency folders, build outputs and caches out of Docker build contexts.
- Added Docker runtime configuration regression tests.
- Fixed Alembic logging format so Docker migration logs no longer print literal `%(levelname)` formatter strings.
- Changed the frontend API default to same-origin requests and added a Vite proxy for `/api` and `/health`, with Docker routing to `http://backend:8000`.
- Added a browser runtime verification script that fails when the frontend `/api` proxy returns Vite HTML instead of the backend JSON envelope.
- Updated environment and local runbook documentation so Docker/LAN browser clients use same-origin API calls through the frontend proxy by default.
- Corrected example YOLO environment variables to the names the backend actually reads: `YOLO_ENABLED`, `YOLO_MODEL_PATH` and `YOLO_MAX_TILES`.
- Replaced the Docker frontend runtime with an nginx-served production build and explicit `/api` plus `/health` reverse proxy to the backend service, avoiding Vite HTML fallback for API requests.
## M6 — Codex Autonomy Pack
Added:
- M6 autonomy boundaries.
- M6 quality gates.
- Codex self-review checklist.
- Failure recovery playbook.
- Gap registry.
- Next-day execution checklist.
- Final handoff template.
- Codex pass prompts PASS 00 through PASS 12.
- GitHub issue templates and PR template.
- GitHub Actions docs/contract smoke workflow.
- Codex preflight and pass-end scripts.
Purpose:
- Prepare the repository so Codex can build with strict guidance and bounded improvement freedom.
## M7 - Implementation Control Layer
Added:
- M7 implementation control layer.
- Locked build sequence.
- Regression trap catalogue.
- Codex self-review checklist.
- Geospatial calculation rules.
- Frontend state rules.
- API response rules.
- Module completion matrix.
- Codex decision boundaries.
- Proposed improvements backlog.
- End-of-pass review prompts.
- Regression and contract drift audit prompts.
- Module contracts for project/area/dataset, detection boundary and QA/QC.
- M7 self-review scripts.
## M8 - Tomorrow Execution Pack
- Added Day 1 Codex execution pack.
- Added pass-by-pass Day 1 prompts.
- Added autonomy boundaries, failure recovery and quality gate matrix.
- Added operator checklist and smoke script scaffold.
- Added next-pass guidance for Day 2 GeoAI loop.
## v0.9 — M9 Max Preparation
- Added Codex day-one master prompt.
- Added autonomous build doctrine and pass scorecards.
- Added real-vs-demo data policy and detailed data contracts.
- Added geospatial edge cases, UI state spec and API validation examples.
- Added implementation review script, regression map and gap-to-task conversion rules.
- Added final pre-code checklist and long-form prompt variants.
## Sprint 1 readiness hardening (2026-06-11)
- Added cross-platform backend/runtime scripts with `python`/`python3` fallback in readiness tooling.
- Fixed backend packaging metadata so editable install works in current flat repo layout.
- Added dependency and smoke test script updates for Sprint 1 services.
- Added minimal Sprint 1 tests for health endpoint, GeoJSON metadata extraction, invalid payload rejection, and dataset content reads.
- Fixed frontend README doc reference typo for repository conventions.
- Updated Sprint 1 docs to include backend import smoke and concrete local setup commands.
## Sprint 2 foundation (2026-06-11)
- Added canonical vector/raster dataset handling and lifecycle status transitions (`uploaded`, `validating`, `ready`, `failed`).
- Added vector metadata extraction (feature count, geometry types, bounds, approximate area, CRS assumptions).
- Added raster metadata extraction service with dependency-aware fallback (`RASTER_PROCESSING_UNAVAILABLE`).
- Added deterministic storage metadata capture (`original_filename`, `stored_filename`, `content_type`, `size_bytes`, `checksum_sha256`) and upload folder layout.
- Added dataset inspection/vector summary/raster metadata API endpoints and frontend detail panel support.
- Added Sprint 2 tests for vector metadata, invalid GeoJSON handling, legacy geojson compatibility and raster dependency fallback.
## Sprint 3 foundation (2026-06-11)
- Added job model and database migration for queued/running/success/failed operations.
- Added job APIs for create/list/read/status under project scope.
- Added vector operation services and route wiring for inspect/bbox/stats/clip/buffer/intersect.
- Added raster operation scaffolding for inspect/metadata/preview, with dependency-aware clip/tile unavailability.
- Added frontend operation controls, job status display, and derived dataset link-through in dataset detail panel.
- Updated API contracts and execution log for Sprint 3 foundations.
## Sprint 5 raster analytics hardening (2026-06-11)
- Added raster band statistics operation:
- min, max, mean, std, nodata count, nodata ratio, valid pixel count, dtype, band index and optional histogram.
- chunked raster reads to reduce memory pressure and explicit dependency-aware unavailable mode when raster libs are missing.
- Added raster reproject operation foundation with CRS validation:
- supports target CRS selection via explicit parameter,
- persists derived output dataset with operation provenance,
- records operation parameters and error details when invalid.
- Hardened raster clip and tile manifest flow:
- explicit empty clip failure behavior,
- bounds/metadata refresh and improved tile manifest fields.
- Added raster operation job persistence tests:
- result_json and error_message persistence,
- dependency-aware statistics failure behavior,
- invalid CRS handling,
- output linkage for reprojected datasets.
- Extended dataset UI dataset detail job panel:
- raster metadata visibility (CRS, bounds, resolution),
- raster band statistics rendering,
- reproject form and job result visibility.
## Sprint 6 local spectral indices (2026-06-12)
- Added local spectral index operations:
- NDVI endpoint
- NDWI endpoint
- NDBI endpoint
- Added explicit spectral index input validation:
- positive integer checks
- source raster band-count bounds checks
- Implemented dependency-aware index execution for missing raster dependencies.
- Added local spectral raster output generation using float32 and `NaN` invalid handling.
- Stored index-derived dataset provenance metadata:
- `source_dataset_id`
- `operation` (`raster.ndvi`, `raster.ndwi`, `raster.ndbi`)
- `band_mapping`
- `formula`
- `output_dtype`
- `nodata_strategy`
- `value_range_note`
- `created_at`
- `output_dataset_id`
- `path`
- Extended dataset detail UI with spectral index controls and result dataset actions.
- Updated:
- `docs/API_CONTRACTS.md`
- `docs/RASTER_OPERATIONS_SPEC.md`
- `backend/README.md`
- `frontend/README.md`
- `docs/CODEX_EXECUTION_LOG.md`
## Sprint 7A persistence and QA foundation (2026-06-12)
- Added `vector_features` as first-class queryable vector state while preserving original uploaded files as source artifacts.
- Added `quality_checks` and `metrics` as persisted QA/QC domain records.
- Added Alembic migration `202606120700_sprint7a_persistence_foundation.py` for vector features, quality checks, metrics and required indexes.
- Persisted uploaded vector GeoJSON feature properties and geometries into PostGIS-backed feature rows.
- Updated QA candidate-vs-reference jobs to persist quality checks and metric rows and return `quality_check_id`.
- Hardened GRB/OSM provider contracts as honest `not_configured` capability stubs only.
- Added tests for Sprint 7A persistence, dataset role validation, provider contracts, migration integrity and QA route persistence.
## Sprint 7B provider integration skeleton (2026-06-12)
- Added central provider registry entries for `grb`, `osm`, `manual` and `fixture`.
- Added provider capability, layer, status and future import-contract endpoints using the existing API envelope style.
- Kept GRB and OSM as explicit `not_configured` providers with no live WFS, Overpass or download behavior.
- Added provider-to-dataset mapping rules for future imports through `DatasetService` and `VectorFeatureService`.
- Added lightweight frontend Provider Capabilities panel with status, authority, layers, query modes and limitations.
- Added live PostGIS migration smoke script for opt-in local database verification.
- Added Sprint 7B provider registry/API tests.
## Sprint 8 Detection Lab foundation (2026-06-12)
- Added `detections` as first-class persisted PostGIS records linked to project, dataset, job and analysis run.
- Hardened `analysis_runs` with dataset/job/model/result metadata for future detection and segmentation workflows.
- Added model registry capabilities for `yolo-placeholder` and `manual-fixture-detector`.
- Added Detection Lab service and API foundation with dependency-aware `DETECTION_MODEL_UNAVAILABLE` responses.
- Added explicit fixture detector mode for tests/demo fixtures only; no fake production inference was introduced.
- Added minimal frontend Detection Lab panel for model status, raster dataset selection, confidence threshold and run status.
- Added Sprint 8 tests for persistence, model capabilities, unavailable model behavior, invalid dataset validation, explicit fixture persistence and API envelope shape.
## Sprint 8B configured YOLO foundation (2026-06-12)
- Added optional `ai` backend dependency group for Ultralytics/Torch without making AI dependencies mandatory for normal startup.
- Added `yolo-configured` model registry capability with `not_configured`, `dependency_unavailable` and `configured` status behavior.
- Added import-safe YOLO adapter that loads only an existing local model path and does not auto-download weights.
- Added raster tile manifest validation, configured tile limits and pixel bbox to EPSG:4326 detection polygon conversion.
- Added mocked YOLO persistence tests that verify first-class detection records without requiring YOLO dependencies.
- Added Detection Lab tile manifest path input for configured YOLO runs.
- Updated AI/API/backend/frontend docs for Sprint 8B configuration and limitations.
## Sprint 8C detection visualization and QA integration (2026-06-12)
- Added detection result review endpoints for run lists, filtered detections, detection detail and GeoJSON FeatureCollection output.
- Added detection QA against persisted reference `vector_features` using existing `quality_checks` and `metrics`.
- Added frontend Detection Lab run selection, detection table, class/confidence filters and MapLibre detection GeoJSON overlay.
- Added frontend detection QA controls and metric summary display.
- Added tests for detection GeoJSON shape, filters, detail, QA persistence, no-match QA and Sprint 8B manifest edge cases.
- Segmentation, LiDAR, Copilot, Training Studio and Reports remain out of scope.
## Sprint 9 Segmentation Lab foundation (2026-06-12)
- Added `segmentations` as first-class persisted PostGIS MultiPolygon records linked to project, dataset, job and analysis run.
- Added deterministic segmentation mask path convention under `storage/masks/{project_id}/{analysis_run_id}/tile_{tile_index}/`.
- Added segmentation model registry capabilities:
- `segmentation-placeholder`
- `fixture-segmenter`
- `yolo-seg-configured`
- `sam-configured`
- Added Segmentation Lab service and API foundation for model listing, run creation, run/result listing, detail, GeoJSON output and reference QA.
- Added segmentation QA against persisted reference `vector_features` using existing `quality_checks` and `metrics`.
- Added minimal frontend Segmentation Lab UI with model status, raster selection, run/result table, map overlay and QA metric display.
- Real SAM, real YOLO-seg, model downloads, new AI dependencies, LiDAR, Copilot, Training Studio and Reports remain out of scope.
## Sprint 10 release hardening and modularization (2026-06-13)
- Extracted project, area, provider capabilities, Detection Lab and Segmentation Lab UI sections from `frontend/src/App.tsx` into focused components.
- Preserved existing API client usage, state ownership, MapLibre overlay behavior and workbench UX.
- Hardened readiness checks to include Alembic head verification and `scripts/live_migration_smoke.sh` syntax validation.
- No new product features, migrations, AI dependencies or live external provider fetching were introduced.
## Sprint 11 live Docker/PostGIS runtime validation (2026-06-13)
- Hardened `scripts/live_migration_smoke.sh` so fresh databases run Alembic migrations before checking `PostGIS_Version()`.
- Added live runtime schema-object checks for core migrated tables and geometry indexes.
- Added backend tests that lock the live migration smoke ordering and schema-check contract.
- Documented exact Docker/PostGIS validation commands, expected `DATABASE_URL` and local cleanup commands.
- Docker was unavailable in the current shell, so live container execution remains pending on a Docker-enabled machine.
## Sprint 12 QA/QC golden dataset and benchmarking (2026-06-15)
- Added deterministic golden building QA/QC fixtures and expected metric baseline.
- Added `scripts/run_golden_qa_benchmark.py` to run existing QA/QC logic against the golden fixtures and fail on metric drift.
- Added backend tests covering expected golden metrics and `QualityCheck`/`Metric` persistence verification.
- Documented benchmark purpose, command, expected outputs, tolerance and limitations.
- No product features, API contracts, migrations, live providers, AI models or new dependencies were introduced.
## Sprint 13 real YOLO operational hardening (2026-06-15)
- Added `YoloPreflightService` for local configured-YOLO readiness checks without loading models or running inference.
- Added `scripts/yolo_preflight.py` for checking enabled state, optional dependency availability, local model path, tile manifest validity, tile limit and referenced tile paths.
- Added backend tests for disabled, dependency-unavailable and ready preflight states plus CLI JSON output.
- Documented preflight usage in backend and AI pipeline docs.
- No model downloads, API contracts, migrations, new dependencies, segmentation behavior or provider fetching were introduced.
## Sprint 14 Docker GIS runtime enablement (2026-06-16)
- Added a backend `gis` optional dependency group for the approved raster/vector runtime stack.
- Updated the backend Docker image to install the `gis` extra plus GDAL/GEOS/PROJ system packages.
- Added `scripts/verify_gis_runtime.sh` to verify browser-facing PostGIS, Rasterio and GeoPandas capabilities through the frontend proxy.
- Added `scripts/gis_import_smoke.py` and made the backend Docker build fail if Rasterio, GeoPandas or pyogrio cannot be imported.
- Moved the Docker build-time GIS import smoke into the backend build context and kept the root script as a local wrapper.
- Included the GIS runtime script syntax check in the main readiness gate.
- Added backend and frontend Docker Compose healthchecks and made the frontend wait for a healthy backend.
- Added regression tests for Docker GIS dependency installation and capability verification script coverage.
- Documented Docker GIS runtime verification commands for local and LAN deployments.
- No API contracts, migrations, AI dependencies, provider fetching or product features were changed.
## Sprint 15 explicit demo workflow seed (2026-06-16)
- Added `POST /api/v1/demo/workflow` to seed or return an explicit offline demo workflow.
- The demo workflow creates a project, AOI, fixture reference building dataset, fixture candidate building dataset and persisted QA/QC metrics.
- Added `scripts/seed_demo_workflow.py` for CLI-based demo seeding.
- Added frontend "Load demo workflow" action in the Projects panel.
- Added tests for the demo endpoint envelope and fixture contract.
- No live GRB/OSM fetching, AI inference, migrations or new dependencies were introduced.
## Sprint 16 QA/QC result visibility (2026-06-16)
- Added `GET /api/v1/projects/{project_id}/quality-checks` to list persisted quality checks and metric rows.
- Added a frontend QA/QC Results panel for project-level persisted QA output.
- Demo workflow loading and QA actions now refresh visible QA/QC results.
- Added backend tests for quality check listing and canonical response envelopes.
- No migrations, new dependencies, live provider fetching or AI inference were introduced.
## Sprint 17 export foundation (2026-06-16)
- Hardened `POST /api/v1/exports/geojson` so exports persist `Export` rows instead of returning dataset ids as export ids.
- Added GeoJSON export support for vector datasets, detection runs and segmentation runs using existing persisted geometry services.
- Added project metadata JSON export, lightweight project report HTML export and export list/read/content/download endpoints.
- Added a frontend Export Center panel for creating exports, listing export records, previewing JSON artifact content and downloading artifacts.
- Added export history to project metadata/report artifacts.
- Added `scripts/verify_demo_export_workflow.sh` to smoke test demo seeding, QA/QC visibility, metadata/report/vector exports, export listing and artifact downloads through the browser-facing URL.
- Added backend tests for export persistence, artifact writing, raster rejection, HTML report creation, canonical export envelopes and raw file downloads.
- No migrations, new dependencies, live provider fetching, AI inference, LiDAR, Copilot, Training Studio or separate Reports module were introduced.
CODEX_START.md
+60
View File
@@ -0,0 +1,60 @@
# CODEX START — Use This First
This is the shortest possible entry point for the first implementation run.
## Mandatory order
1. Read `docs/00-start/START_HERE.md`.
2. Read `docs/30-codex-optimization/CODEX_RUN_CHECKLIST.md`.
3. Read `docs/30-codex-optimization/PROMPT_DISCIPLINE.md`.
4. Read `docs/20-run-readiness/RUN_READINESS_FINAL.md`.
5. Read `docs/20-run-readiness/CODEX_TOMORROW_RUNBOOK.md`.
6. Use `prompts/codex/m14/CODEX_FIRST_DAY_MASTER_PROMPT.md` as the first Codex prompt.
7. Follow `docs/20-run-readiness/PASS_SEQUENCE_FINAL.md` exactly.
8. Select the relevant skill from `skills/` for the active pass.
## First build objective
Build the V1 foundation vertical slice:
Project → Area → Dataset metadata → Reference polygons → Predicted detections → QA/QC → GeoJSON export → Minimal UI.
Do not start with heavy AI inference, LiDAR, training, MLOps, Sentinel automation, or advanced report generation before the foundation passes.
## Pass completion rule
A pass is not done until:
- commands were run;
- tests/smoke checks were attempted;
- docs/status were updated;
- limitations are explicit;
- next pass is clear.
## M13 additions
Before implementing, Codex must respect:
- `docs/30-codex-optimization/CODEX_OPTIMIZATION_OVERVIEW.md`
- `docs/30-codex-optimization/TOKEN_BUDGET_POLICY.md`
- `docs/30-codex-optimization/SECRETS_AND_ENV_POLICY.md`
- `docs/30-codex-optimization/PARALLEL_AGENT_STRATEGY.md` when using multiple agents/worktrees
- `docs/30-codex-optimization/CODEX_SKILLS_INDEX.md`
The preferred first prompt is now:
- `prompts/codex/m14/CODEX_FIRST_DAY_MASTER_PROMPT.md`
## M14 launch controls
Before the first implementation pass, Codex must read:
- `docs/40-build-launch/SPRINT_1_SCOPE_FREEZE.md`
- `docs/40-build-launch/BUILD_SUCCESS_DEFINITION.md`
- `docs/40-build-launch/BUILD_ORDER_GRAPH.md`
- `docs/40-build-launch/CODEX_STOP_RULES.md`
- `docs/40-build-launch/MODULE_ACCEPTANCE_CRITERIA.md`
The first implementation run is Sprint 1 only. Do not implement detection, segmentation, Sentinel, LiDAR, training, AI Copilot or advanced reports during Sprint 1.
M10_UPDATE_MANIFEST.txt
+44
View File
@@ -0,0 +1,44 @@
docs/18-ultra-prep/README.md
docs/18-ultra-prep/AUTONOMOUS_BUILD_CHARTER.md
docs/18-ultra-prep/BUILD_PASS_TEMPLATE.md
docs/18-ultra-prep/CRITICAL_PATH_TO_V1.md
docs/18-ultra-prep/CODEX_START_HERE.md
prompts/codex/M10_MASTER_AUTONOMOUS_PROMPT.md
prompts/codex/M10_PASS_SEQUENCE.md
docs/18-ultra-prep/FEATURE_FLAG_STRATEGY.md
docs/18-ultra-prep/ERROR_TAXONOMY.md
docs/18-ultra-prep/GEOMETRY_CONTRACTS.md
docs/18-ultra-prep/CRS_POLICY.md
docs/18-ultra-prep/SECURITY_AND_SECRET_HANDLING.md
docs/18-ultra-prep/PERFORMANCE_BUDGETS.md
docs/18-ultra-prep/OBSERVABILITY_PLAN.md
docs/18-ultra-prep/CONNECTOR_IMPLEMENTATION_GUIDE.md
docs/18-ultra-prep/MODEL_ADAPTER_GUIDE.md
docs/18-ultra-prep/QA_QC_MATCHING_ALGORITHM.md
docs/18-ultra-prep/FRONTEND_STATE_MACHINE.md
docs/18-ultra-prep/UI_COPY_BANK.md
docs/18-ultra-prep/REPO_HYGIENE_RULES.md
docs/18-ultra-prep/RELEASE_GATE_V1.md
docs/18-ultra-prep/KNOWN_LIMITATIONS_TEMPLATE.md
docs/18-ultra-prep/FINAL_PRE_CODEX_CHECKLIST.md
tickets/TICKET_INDEX.md
tickets/T-001-backend-skeleton.md
tickets/T-002-database-foundation.md
tickets/T-003-project-area-domain.md
tickets/T-010-dataset-manager.md
tickets/T-011-vector-processing.md
tickets/T-012-raster-processing.md
tickets/T-020-frontend-foundation.md
tickets/T-021-map-workbench.md
tickets/T-022-dataset-ui.md
tickets/T-030-detection-adapter.md
tickets/T-031-qaqc-engine.md
tickets/T-032-export-engine.md
tickets/T-033-demo-workflow.md
contracts/api/examples/project_create.json
contracts/api/examples/area_create.geojson
contracts/api/examples/error_feature_disabled.json
contracts/api/examples/qaqc_result.json
scripts/smoke_m10.sh
docs/TODO.md
RELEASE_NOTES/M10_ultra_preparation.md
M11_UPDATE_MANIFEST.txt
+23
View File
@@ -0,0 +1,23 @@
M11 Architect Audit & Control Layer
Added:
- docs/00-start/START_HERE.md
- docs/governance/GEOINTEL_CONSTITUTION.md
- docs/governance/FORBIDDEN_DECISIONS.md
- docs/governance/ARCHITECTURE_INVARIANTS.md
- docs/governance/DECISION_PRECEDENCE.md
- docs/specs/CANONICAL_DOMAIN_MODELS.md
- docs/specs/GIS_STANDARDS.md
- docs/specs/RASTER_STANDARDS.md
- docs/specs/STATE_MACHINES.md
- docs/specs/DATA_LIFECYCLE.md
- docs/specs/ERROR_CATALOG.md
- docs/specs/PERFORMANCE_BUDGETS_CANONICAL.md
- docs/workflows/GOLDEN_PATHS.md
- docs/build/BUILD_ORDER_DEPENDENCY_GRAPH.md
- docs/build/CODEX_OPERATING_SYSTEM.md
- docs/19-architect-audit/ARCHITECT_AUDIT_REPORT_M11.md
- prompts/codex/M11_ARCHITECT_MASTER_PROMPT.md
Changed:
- README.md now points to the single canonical M11 start path.
M12_UPDATE_MANIFEST.txt
+20
View File
@@ -0,0 +1,20 @@
M12 Final Run Readiness Layer
Added:
- CODEX_START.md
- docs/20-run-readiness/RUN_READINESS_FINAL.md
- docs/20-run-readiness/PASS_SEQUENCE_FINAL.md
- docs/20-run-readiness/CODEX_TOMORROW_RUNBOOK.md
- docs/20-run-readiness/IMPLEMENTATION_READINESS_CHECKLIST.md
- docs/20-run-readiness/REPO_CONFLICT_RESOLUTION.md
- prompts/codex/final/DAY_1_MASTER_PROMPT.md
- prompts/codex/final/PASS_00_REPO_AUDIT_FINAL.md
- prompts/codex/final/PASS_01_BACKEND_FOUNDATION_FINAL.md
- prompts/codex/final/PASS_02_DOMAIN_DATABASE_FINAL.md
- scripts/preimplementation_audit.py
- scripts/run_readiness_check.sh
- Makefile
- RELEASE_NOTES/v0.12-m12-final-run-readiness.md
Changed:
- README.md
M13_UPDATE_MANIFEST.txt
+25
View File
@@ -0,0 +1,25 @@
M13 — Codex Optimization Pack
Purpose:
- Improve Codex execution quality after M12 final run readiness.
- Add reusable skills, prompt discipline, token policy, secrets policy, parallel agent strategy and pass completion prompts.
Added:
- docs/30-codex-optimization/CODEX_OPTIMIZATION_OVERVIEW.md
- docs/30-codex-optimization/CODEX_RUN_CHECKLIST.md
- docs/30-codex-optimization/PROMPT_DISCIPLINE.md
- docs/30-codex-optimization/TOKEN_BUDGET_POLICY.md
- docs/30-codex-optimization/SECRETS_AND_ENV_POLICY.md
- docs/30-codex-optimization/PARALLEL_AGENT_STRATEGY.md
- docs/30-codex-optimization/CODEX_SKILLS_INDEX.md
- docs/30-codex-optimization/M13_HANDOFF_SUMMARY.md
- skills/*/SKILL.md
- prompts/codex/m13/*.md
- scripts/validate_m13_codex_assets.py
Updated:
- README.md
- CODEX_START.md
- Makefile
- scripts/run_readiness_check.sh
- CHANGELOG.md
M14_UPDATE_MANIFEST.txt
+25
View File
@@ -0,0 +1,25 @@
M14 Build Launch Package
Added:
- docs/40-build-launch/BUILD_SUCCESS_DEFINITION.md
- docs/40-build-launch/SPRINT_1_SCOPE_FREEZE.md
- docs/40-build-launch/DATA_ACQUISITION_PLAYBOOK.md
- docs/40-build-launch/GOLDEN_DATASET_PACKAGE.md
- docs/40-build-launch/BUILD_ORDER_GRAPH.md
- docs/40-build-launch/MODULE_ACCEPTANCE_CRITERIA.md
- docs/40-build-launch/CODEX_STOP_RULES.md
- docs/40-build-launch/RELEASE_STRATEGY.md
- docs/40-build-launch/RISK_REGISTER.md
- docs/40-build-launch/BACKLOG_PRIORITIES_MOSCOW.md
- docs/40-build-launch/FOLDER_OWNERSHIP.md
- prompts/codex/m14/CODEX_FIRST_DAY_MASTER_PROMPT.md
- checklists/SPRINT_1_OPERATOR_CHECKLIST.md
- release/v0.1-foundation-target.md
- scripts/validate_m14_launch_assets.py
Updated:
- README.md
- CODEX_START.md
- docs/00-start/START_HERE.md
- Makefile
- scripts/run_readiness_check.sh
M5_UPDATE_MANIFEST.txt
+29
View File
@@ -0,0 +1,29 @@
docs/OBSERVABILITY_PLAN.md
docs/TROUBLESHOOTING_RUNBOOK.md
docs/RELEASE_PROCESS.md
docs/ROLLBACK_AND_RECOVERY.md
docs/DEPENDENCY_LOCK_PLAN.md
docs/SECURITY_CHECKLIST.md
docs/DATA_PRIVACY_AND_LICENSING.md
docs/EXTERNAL_SERVICES_ADAPTERS.md
docs/GEOSPATIAL_VALIDATION_RULES.md
docs/BUILD_GOVERNANCE.md
docs/M5_OPERATIONAL_READINESS.md
docs/CI_CD_SPECIFICATION.md
docs/HEALTHCHECK_CONTRACTS.md
docs/CODEX_PASS_0_REPO_AUDIT.md
docs/CODEX_PASS_1_BACKEND_FOUNDATION.md
docs/CODEX_PASS_2_DATABASE_AND_MODELS.md
docs/CODEX_PASS_3_DATASET_MANAGER.md
docs/CODEX_PASS_4_RASTER_VECTOR_CORE.md
docs/CODEX_PASS_5_FRONTEND_WORKBENCH_SHELL.md
docs/CODEX_PASS_6_DETECTION_QA_SKELETON.md
docs/CODEX_PROMPT_M5_LONG_AUTONOMOUS_BUILD.md
scripts/check_repo_structure.sh
scripts/smoke_backend_import.sh
scripts/smoke_docs.py
scripts/smoke_contracts.py
scripts/validate_fixtures.py
RELEASE_NOTES/v0.5-m5-operational-readiness.md
CHANGELOG.md
docs/TODO.md
M9_UPDATE_MANIFEST.txt
+23
View File
@@ -0,0 +1,23 @@
# M9 Update Manifest
New/changed files:
- `docs/17-max-prep/M9_MAX_PREPARATION_PACK.md`
- `prompts/codex/M9_DAY_ONE_MASTER_PROMPT.md`
- `docs/17-max-prep/M9_AUTONOMOUS_BUILD_DOCTRINE.md`
- `docs/17-max-prep/M9_PASS_SCORECARDS.md`
- `docs/17-max-prep/M9_BUILD_BLOCKERS_AND_RECOVERY.md`
- `docs/17-max-prep/M9_REAL_VS_DEMO_DATA_POLICY.md`
- `docs/17-max-prep/M9_DATA_CONTRACTS_DETAILED.md`
- `docs/17-max-prep/M9_GEOSPATIAL_EDGE_CASES.md`
- `docs/17-max-prep/M9_UI_STATE_SPEC.md`
- `docs/17-max-prep/M9_API_VALIDATION_EXAMPLES.md`
- `docs/17-max-prep/M9_IMPLEMENTATION_REVIEW_SCRIPT.md`
- `docs/17-max-prep/M9_REGRESSION_MAP.md`
- `docs/17-max-prep/M9_GAP_TO_TASK_CONVERSION.md`
- `docs/17-max-prep/M9_MODULE_DATAFLOW_CHECKLIST.md`
- `docs/17-max-prep/M9_FINAL_PRE_CODE_CHECKLIST.md`
- `docs/17-max-prep/M9_LONG_FORM_CODEX_PROMPT_VARIANTS.md`
- `docs/IMPLEMENTATION_GAP_REPORT.md`
- `RELEASE_NOTES/v0.9-m9-max-preparation.md`
- `CHANGELOG.md`
Makefile
+50
View File
@@ -0,0 +1,50 @@
PYTHON_BIN := $(shell command -v python3 >/dev/null 2>&1 && echo python3 || echo python)
.PHONY: readiness docs fixtures preflight backend-install backend-test backend-dev frontend-install frontend-typecheck frontend-build m13 m14
readiness:
bash scripts/run_readiness_check.sh
backend-install:
cd backend && \
$(PYTHON_BIN) -m pip install -e .[dev]
backend-test:
cd backend && \
$(PYTHON_BIN) -m pytest
backend-dev:
cd backend && \
$(PYTHON_BIN) -m uvicorn app.main:app --reload
frontend-install:
cd frontend && \
npm install
frontend-typecheck:
cd frontend && \
npm run typecheck
frontend-build:
cd frontend && \
npm run build
docs:
$(PYTHON_BIN) scripts/smoke_docs.py
fixtures:
$(PYTHON_BIN) scripts/validate_fixtures.py
preflight:
bash scripts/codex_preflight.sh || true
$(PYTHON_BIN) scripts/preimplementation_audit.py
.PHONY: m13
m13:
$(PYTHON_BIN) scripts/validate_m13_codex_assets.py
.PHONY: m14
m14:
$(PYTHON_BIN) scripts/validate_m14_launch_assets.py
README.md
+250
View File
@@ -0,0 +1,250 @@
# GeoIntel Kempen
GeoIntel Kempen is a GeoAI Workbench for the Belgian Kempen. It is designed as a portfolio-grade project combining GIS, remote sensing, raster/vector processing, computer vision, QA/QC and geospatial exports.
GeoIntel is not a generic dashboard or chatbot. The core product is:
> data → processing → geospatial output → QA/QC → export
## Current milestone
**M14 — Build Launch Package**
The canonical start point is now:
- `CODEX_START.md`
- `docs/00-start/START_HERE.md`
- `docs/40-build-launch/SPRINT_1_SCOPE_FREEZE.md`
- `docs/40-build-launch/BUILD_SUCCESS_DEFINITION.md`
- `docs/40-build-launch/BUILD_ORDER_GRAPH.md`
Older M0-M13 handoff files are retained as historical preparation artifacts. The M14 build launch docs, M13 optimization docs, M12 final run-readiness docs, M11 governance docs and canonical specs take precedence.
## Core V1 vertical slice
The first implementation target is:
1. Project + Area creation.
2. Dataset registration/upload and metadata extraction.
3. Reference building layer loading.
4. Predicted detection layer loading/import.
5. QA/QC matching against reference polygons.
6. Metrics and false positive/false negative outputs.
7. GeoJSON export.
8. Minimal map/workbench UI.
## Primary stack
- Frontend: React, TypeScript, MapLibre GL, Deck.gl, Tailwind.
- Backend: FastAPI, Python.
- Database: PostgreSQL + PostGIS.
- GIS processing: GeoPandas, Shapely, Rasterio, PyProj, GDAL.
- AI: PyTorch, Ultralytics YOLO, SAM-compatible architecture.
- Jobs: Redis + RQ.
- Storage: local filesystem first, MinIO-compatible later.
## Codex instructions
Codex must start with:
1. `docs/00-start/START_HERE.md`
2. `prompts/codex/M11_ARCHITECT_MASTER_PROMPT.md`
Then follow the build order in:
- `docs/build/BUILD_ORDER_DEPENDENCY_GRAPH.md`
- `docs/build/CODEX_OPERATING_SYSTEM.md`
Before every implementation pass, run available preflight/smoke scripts where applicable.
## Repo principle
This is a documentation-driven engineering repo. The documentation is not decorative; it is the control system for autonomous implementation.
## Fastest Day 1 command path
```bash
make readiness
```
## Sprint 2 quick start
- Update dependencies:
```bash
python -m pip install -e backend/.[dev]
cd frontend && npm install
```
- Run full readiness checks (with no scope expansion):
```bash
python -m compileall backend/app
cd backend && python -m pytest
cd ../frontend && npm run typecheck && npm run build
bash scripts/run_readiness_check.sh
```
- Raster workflow validation command (backend only):
```bash
bash scripts/smoke_backend_import.sh
cd backend && python -c "from app.main import app; print(app.title)"
```
If `rasterio` is not installed, raster metadata endpoints return `RASTER_PROCESSING_UNAVAILABLE` and the frontend displays the
state as failed until the dependency is added.
## Sprint 4 raster foundation
- Raster operations now support:
- raster metadata extraction,
- raster preview generation,
- raster clip by area (with provenance on derived datasets),
- raster tile generation with manifest output.
- Raster services are dependency-aware:
- if `rasterio` is unavailable, endpoints return `RASTER_PROCESSING_UNAVAILABLE`.
- if preview dependencies (`numpy`, `pillow`) are unavailable, preview generation is unavailable with a clear error.
- Enable raster stack explicitly when needed:
```bash
cd backend && python -m pip install -e .[dev,raster]
```
## Sprint 5 raster analytics hardening
- Added raster band statistics (min/max/mean/std, nodata ratio/count, valid pixel count, dtype, optional histograms).
- Added raster reproject workflow with CRS validation and provenance persistence.
- Extended tile manifest expectations (`tile_set_id`, `tile_size`, `overlap`, `bounds`, `source_raster_id`, `tile_paths`, `tile_server`).
- Clarified raster operation availability in frontend/backend docs (`RASTER_PROCESSING_UNAVAILABLE` and invalid-CRS cases).
- Raster workflow command set (where available):
```bash
cd backend
python -m pip install -e .[dev,raster]
python -m pytest
cd ../frontend
npm run typecheck
npm run build
```
Then give Codex the prompt in:
- `prompts/codex/final/DAY_1_MASTER_PROMPT.md`
## M13 Codex optimization
For the first serious Codex build run, use:
- `prompts/codex/m13/DAY_1_OPTIMIZED_MASTER_PROMPT.md`
Codex should also use the relevant reusable skill under `skills/` for each implementation pass. Validate the optimization assets with:
```bash
make m13
```
The full readiness path remains:
```bash
make readiness
```
## M14 Build Launch
For the first serious implementation run, use:
- `docs/40-build-launch/SPRINT_1_SCOPE_FREEZE.md`
- `docs/40-build-launch/BUILD_SUCCESS_DEFINITION.md`
- `docs/40-build-launch/CODEX_STOP_RULES.md`
- `prompts/codex/m14/CODEX_FIRST_DAY_MASTER_PROMPT.md`
Validate launch assets with:
```bash
make m14
```
Full readiness remains:
```bash
make readiness
```
## Sprint 1 execution (Sprint 1 only)
From a clean machine:
```bash
cd backend && python -m pip install -e .[dev]
cd ..
make backend-install
make frontend-install
make readiness
```
Copy `.env.example` to `.env` only when you want local overrides. Docker Compose has safe defaults for the local PostGIS/backend/frontend stack and does not require a root `.env` file to exist.
With Docker Compose, open the workbench at `http://localhost:1202`.
The Docker frontend is served by nginx and proxies `/api` and `/health` to the backend container, so browser clients should use the frontend URL only, for example `http://192.168.10.150:1202` on a LAN host.
Runtime containers include healthchecks for PostGIS, backend and frontend. After
startup, inspect them with:
```bash
docker compose ps
```
Verify the browser-facing API proxy after rebuilding Docker images:
```bash
bash scripts/verify_browser_runtime.sh http://localhost:1202 http://localhost:8000/health
```
Verify the Docker GIS runtime after rebuilding the backend image:
```bash
bash scripts/verify_gis_runtime.sh http://localhost:1202
```
On the LAN host use the published browser URL, for example:
```bash
bash scripts/verify_gis_runtime.sh http://192.168.10.150:1202
```
Load the explicit offline demo workflow:
```bash
curl -X POST http://192.168.10.150:1202/api/v1/demo/workflow
```
If `/api/v1/projects` returns frontend HTML instead of a JSON envelope, rebuild
and restart the frontend container.
Useful direct verification commands:
```bash
python -m compileall backend/app
cd backend && python -c "from app.main import app; print(app.title)"
python -m pytest
cd ../frontend && npm run typecheck
cd ../frontend && npm run build
docker compose config
bash scripts/run_readiness_check.sh
```
If `make` or `docker` are unavailable in your shell, run the equivalent script entrypoints directly:
```bash
bash scripts/backend_install.sh
bash scripts/backend_test.sh
bash scripts/frontend_install.sh
bash scripts/frontend_typecheck.sh
bash scripts/frontend_build.sh
bash scripts/run_readiness_check.sh
```
RELEASE_NOTES/M10_ultra_preparation.md
+25
View File
@@ -0,0 +1,25 @@
# M10 Ultra Preparation
M10 adds a stronger Codex autonomy layer:
- autonomous build charter;
- Codex start-here guide;
- pass sequence;
- master prompt;
- geometry contracts;
- CRS policy;
- security and secret handling;
- performance budgets;
- observability plan;
- connector guide;
- model adapter guide;
- QA/QC matching algorithm;
- frontend state machine;
- UI copy bank;
- repo hygiene rules;
- V1 release gate;
- implementation tickets;
- API example payloads;
- final pre-Codex checklist.
This milestone aims to make tomorrow's Codex build significantly more autonomous while preserving strict product boundaries.
RELEASE_NOTES/v0.0-M2.md
+11
View File
@@ -0,0 +1,11 @@
# Release Notes — v0.0 M2 Engineering Package
This is not an application release. It is a repository preparation milestone for autonomous Codex development.
## Main value
Codex now has fewer architecture choices to invent. The repo contains decision records, contracts, engineering rules, fixtures, and build prompts.
## Next recommended action
Run Codex Pass 01 using `prompts/codex/PASS_01_BACKEND_FOUNDATION.md`.
RELEASE_NOTES/v0.0-M3.md
+22
View File
@@ -0,0 +1,22 @@
# Release Notes — v0.0 M3 Implementation Readiness
This is a documentation and repository preparation release.
## Added
- implementation epics
- build tickets
- migration plan
- seed data plan
- local dev runbook
- backend package map
- frontend route map
- module contracts
- job lifecycle
- Codex pass matrix
- additional Codex prompts
- known limitations
## Purpose
Prepare the repository for Codex-driven implementation without requiring major architecture decisions during coding.
RELEASE_NOTES/v0.11-m11-architect-audit-control-layer.md
+18
View File
@@ -0,0 +1,18 @@
# v0.11 — M11 Architect Audit & Control Layer
This release turns the GeoIntel preparation repo into a stricter architecture-controlled implementation repo.
## Highlights
- One canonical `START_HERE` document.
- Constitution, forbidden decisions and architecture invariants.
- Canonical domain model definitions.
- GIS/raster standards.
- State machines and data lifecycle.
- Golden paths and build dependency graph.
- Error catalog and canonical performance budgets.
- M11 Codex architect master prompt.
## Purpose
The goal is to reduce Codex ambiguity before implementation starts. Older handoff documents remain available, but M11 defines the precedence and operating model.
RELEASE_NOTES/v0.12-m12-final-run-readiness.md
+21
View File
@@ -0,0 +1,21 @@
# v0.12 — M12 Final Run Readiness Layer
This release turns the M11 architect audit repo into a directly executable Codex preparation package.
## Added
- Root `CODEX_START.md` as the shortest canonical entry point.
- Final run-readiness docs under `docs/20-run-readiness/`.
- Final Day 1 Codex master prompt under `prompts/codex/final/`.
- Final pass prompts for Pass 00, Pass 01 and Pass 02.
- `scripts/preimplementation_audit.py`.
- `scripts/run_readiness_check.sh`.
- Root `Makefile` with `make readiness`.
## Changed
- README now points to M12 and the final run path.
## Intent
Reduce manual work tomorrow by giving Codex one obvious entry point, one pass sequence, one first prompt, and a simple readiness command.
RELEASE_NOTES/v0.4-m4-autonomous-build-readiness.md
+17
View File
@@ -0,0 +1,17 @@
# v0.4 — M4 Autonomous Build Readiness
This release adds the documentation and fixtures required for longer autonomous Codex implementation passes.
## Highlights
- Clear sprint board.
- Module build contracts.
- Acceptance tests.
- Service IO contracts.
- UI route/state contracts.
- Job lifecycle contract.
- Demo model registry seed.
- Geel demo fixtures.
- Codex prompts per pass.
## Next
M5 should add concrete migration SQL, OpenAPI draft, component prop contracts and test skeletons.
RELEASE_NOTES/v0.5-m5-operational-readiness.md
+22
View File
@@ -0,0 +1,22 @@
# GeoIntel v0.5 — M5 Operational Readiness
## Toegevoegd
- Operational readiness documentatie.
- CI/CD-specificatie.
- Healthcheck-contracten.
- Observability plan.
- Troubleshooting runbook.
- Releaseproces.
- Rollback- en recoveryregels.
- Dependency lock plan.
- Security checklist.
- Data privacy en licensing notities.
- External services adaptercontracten.
- Geospatial validation rules.
- Build governance.
- Codex passdocumenten voor Pass 0 tot Pass 6.
- Long autonomous Codex build prompt.
- Smoke scripts voor repo/docs/contracts/backend import.
## Volgende logische stap
M6 kan zich richten op echte code-scaffolding: backend app, database migrations, API schemas, frontend shell en eerste project/dataset flows.
RELEASE_NOTES/v0.9-m9-max-preparation.md
+27
View File
@@ -0,0 +1,27 @@
# GeoIntel v0.9 — M9 Max Preparation
This release adds a heavy preparation layer intended to maximize Codex autonomy before implementation.
## Added
- M9 max preparation pack.
- Day-one Codex master prompt.
- Autonomous build doctrine.
- Build pass scorecards.
- Build blocker and recovery guide.
- Real vs demo data policy.
- Detailed data contracts.
- Geospatial edge case catalog.
- UI state specification.
- API validation examples.
- Implementation review script.
- Regression map.
- Gap-to-task conversion rules.
- Module dataflow checklist.
- Final pre-code checklist.
- Long-form Codex prompt variants.
- Implementation gap report template.
## Purpose
Make the repository as ready as possible for a long autonomous Codex build session.
adr/ADR-001-technology-stack.md
+25
View File
@@ -0,0 +1,25 @@
# ADR-001 — Technology Stack
## Status
Accepted for V1.
## Context
GeoIntel Kempen must demonstrate modern web development, geospatial processing, and GeoAI engineering. The stack must be realistic for a portfolio project while remaining close to professional workflows.
## Decision
Use:
- Frontend: React + TypeScript.
- Map UI: MapLibre GL with Deck.gl where advanced overlays are useful.
- Backend: FastAPI.
- Database: PostgreSQL + PostGIS.
- Processing: GeoPandas, Shapely, Rasterio, PyProj, GDAL-compatible tools.
- AI: PyTorch with Ultralytics YOLO first; SAM/segmentation later.
- Jobs: Redis + RQ for V1.
- Storage: local filesystem with explicit storage abstraction.
## Consequences
This stack keeps the first build achievable while matching the vacancy profile closely: Python, raster/vector processing, computer vision, AI pipelines, and GIS outputs.
## Non-goals
Do not introduce Django, Flask, MongoDB, Firebase, or a second frontend framework unless a future ADR explicitly replaces this decision.
adr/ADR-002-postgis-choice.md
+27
View File
@@ -0,0 +1,27 @@
# ADR-002 — PostGIS as Spatial Source of Truth
## Status
Accepted for V1.
## Context
GeoIntel stores areas, datasets, AI detections, segmentations, QA geometries, and exports. Spatial operations need to be queryable and persistent.
## Decision
Use PostgreSQL with PostGIS as the canonical database for:
- project areas,
- dataset spatial bounds,
- vector features,
- detection polygons/boxes,
- segmentation polygons,
- QA/QC geometries,
- spatial metadata,
- analysis outputs.
Raw rasters, tiles, masks, and large binary artifacts stay on disk/object storage. PostGIS stores metadata and vectorized results.
## Consequences
The backend can do spatial filtering, intersections, bounding-box queries, and QA matching without reloading every file. The portfolio visibly demonstrates professional GIS database skills.
## Non-goals
Do not store full large rasters as database blobs in V1.
adr/ADR-003-grb-strategy.md
+23
View File
@@ -0,0 +1,23 @@
# ADR-003 — GRB as Authoritative Reference Dataset
## Status
Accepted for V1 research and implementation planning.
## Context
The Basiskaart Vlaanderen / GRB is a professional Flemish geospatial reference dataset. GeoIntel is scoped to the Kempen, so Flemish official data is highly relevant.
## Decision
Treat GRB as the primary QA/QC reference where available. Use it for building/reference geometry validation and later for roads, water, and other base-map objects.
V1 integration strategy:
1. Implement a GRB provider abstraction.
2. Start with WFS or downloaded sample/cache depending on practical availability.
3. Normalize GRB features into a common `reference_features` model.
4. Compare AI detections against GRB with IoU/overlap metrics.
## Consequences
GeoIntel becomes more relevant to real Flemish GeoAI workflows than a generic OSM-only demo. GRB validation becomes a portfolio killer feature.
## Non-goals
Do not block the entire build on live GRB integration. Provide fixtures and provider interfaces first, then connect real GRB when endpoint details are tested.
adr/ADR-004-storage-strategy.md
+27
View File
@@ -0,0 +1,27 @@
# ADR-004 — Storage Strategy
## Status
Accepted for V1.
## Context
GeoIntel stores multiple artifact types: uploaded rasters, vector uploads, generated tiles, model outputs, masks, exports, and reports.
## Decision
Use local filesystem storage for V1 with a strict directory convention:
- `storage/uploads/` for original user uploads,
- `storage/originals/` for normalized source copies,
- `storage/tiles/` for generated raster tiles,
- `storage/masks/` for segmentation masks,
- `storage/derived/` for processed artifacts,
- `storage/exports/` for GeoJSON/COCO/YOLO exports,
- `storage/reports/` for reports,
- `storage/models/` for model artifacts.
Database rows reference files by relative path and content hash.
## Consequences
Simple local development and predictable repo behavior. Future MinIO/S3 migration remains possible because storage calls must go through a service boundary.
## Non-goals
No direct random file writes from routes or frontend-specific paths.
adr/ADR-005-ai-model-strategy.md
+18
View File
@@ -0,0 +1,18 @@
# ADR-005 — AI Model Strategy
## Status
Accepted for V1.
## Context
The vacancy emphasizes PyTorch, object detection, segmentation, and GeoAI. A portfolio build should show a real inference pipeline, not only AI text generation.
## Decision
Use Ultralytics YOLO as the first object detection runtime because it is practical, PyTorch-based, well documented, and fast to integrate. Add segmentation through YOLO-seg or SAM after the detection pipeline is reliable.
Model execution must be wrapped behind `ModelRegistryService` and `DetectionService` interfaces so the UI and API do not depend directly on Ultralytics internals.
## Consequences
GeoIntel can demonstrate model inference, georeferencing, output conversion, confidence thresholds, and QA/QC against GRB.
## Non-goals
Do not train a custom model in V1. Fine-tuning becomes V2/V3 after annotation and dataset export exist.
adr/ADR-006-job-processing.md
+25
View File
@@ -0,0 +1,25 @@
# ADR-006 — Job Processing
## Status
Accepted for V1.
## Context
Raster tiling, detection, segmentation, QA, and exports can take longer than a normal HTTP request.
## Decision
Use Redis + RQ for V1 background jobs. Every long-running operation creates an `analysis_run` or `job` record, updates status, stores outputs, and emits events.
Supported statuses:
- pending,
- queued,
- running,
- completed,
- failed,
- cancelled.
## Consequences
The UI can show progress and status without blocking. RQ is easier than Celery for an initial solo/portfolio project.
## Non-goals
No Kubernetes-native queues, no Airflow, no full workflow engine in V1.
adr/ADR-007-api-design.md
+28
View File
@@ -0,0 +1,28 @@
# ADR-007 — API Design
## Status
Accepted for V1.
## Context
The frontend must be API-driven and Codex must not invent inconsistent response shapes.
## Decision
Use REST-style FastAPI endpoints with typed Pydantic schemas. Responses use stable envelopes for long-running jobs and direct resources for simple CRUD operations.
Errors use a common structure:
```json
{
"error": {
"code": "DATASET_NOT_FOUND",
"message": "Dataset not found.",
"details": {}
}
}
```
## Consequences
Frontend API clients, tests, and docs stay consistent.
## Non-goals
No GraphQL in V1.
backend/.dockerignore
+10
View File
@@ -0,0 +1,10 @@
__pycache__
*.pyc
.pytest_cache
.mypy_cache
.ruff_cache
geointel_backend.egg-info
storage
dist
node_modules
.env
backend/.gitkeep
View File
backend/Dockerfile
+25
View File
@@ -0,0 +1,25 @@
FROM python:3.12-slim
WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends \
gcc \
gdal-bin \
libgdal-dev \
libgeos-dev \
libproj-dev \
libpq-dev \
proj-bin \
&& rm -rf /var/lib/apt/lists/*
COPY pyproject.toml README.md /app/
COPY app /app/app
RUN pip install --no-cache-dir --upgrade pip setuptools
RUN pip install --no-cache-dir ".[gis]"
COPY . /app
RUN python scripts/gis_import_smoke.py
EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
backend/README.md
+537
View File
@@ -0,0 +1,537 @@
# GeoIntel Backend (Sprint 3 foundation layer)
FastAPI backend for GeoIntel Kempen Foundation Sprints.
## Scope implemented
- Project CRUD
- Area CRUD with PostGIS geometry
- Vector and raster dataset upload/registration
- Deterministic local storage metadata capture
- PostGIS migration and database foundation
- Job foundation for async-ready GIS operations
## Sprint 2 additions
- Dataset typing and lifecycle support:
- `uploaded`
- `validating`
- `ready`
- `failed`
- Vector metadata extraction:
- feature count
- geometry type summary
- bounds
- approximate area
- CRS and CRS assumption
- Raster metadata endpoint:
- returns raster profile when `rasterio` is available
- returns clear `RASTER_PROCESSING_UNAVAILABLE` error when dependency is missing
- Deterministic storage metadata capture:
- original filename
- stored filename
- MIME/content type
- size bytes
- checksum SHA-256
## Sprint 3 additions
- Lightweight job architecture:
- `jobs` table and migrations
- job create/list/read/status API
- synchronous execution behind job abstraction
- Vector operations foundation:
- inspect
- bbox
- stats
- clip by area
- buffer
- intersect
- invalid geometry rejection with typed errors
- Raster operation foundation:
- inspect
- metadata
- preview readiness
- clip by area (dependency-aware with unavailable fallback)
- tile generation with manifest output
- real preview image generation when dependencies are installed
## Sprint 4 additions
- Raster foundation is now implemented with real extraction and deterministic artifact outputs:
- metadata returns width, height, band count, CRS, bounds, resolution, dtype, nodata, transform
- preview endpoint generates and reuses PNG previews with width/height
- clip operation persists a derived raster dataset with:
- `source_dataset_id`
- `operation`
- `operation_parameters`
- tile operation writes deterministic raster tiles under `tiles/{project_id}/{source_dataset_id}/{tile_set_id}`
- tile manifest includes tile path, pixel window, bounds, transform, and count
- Dependency behavior:
- when `rasterio` is missing, raster processing returns `RASTER_PROCESSING_UNAVAILABLE`
- preview endpoint additionally requires numpy/pillow and returns `RASTER_PROCESSING_UNAVAILABLE` when missing
## Sprint 5 additions
- Raster analytics hardening:
- raster band statistics now include:
- min, max, mean, std
- nodata count and ratio
- valid pixel count
- dtype
- optional histogram bins (default 16 bins)
- raster reproject operation implemented (CRS transform + rasterio reprojection) using dependency-aware raster processing checks.
- reproject failures are explicit (`INVALID_PARAMETERS`, `INVALID_DATASET_CRS`, `RASTER_PROCESSING_UNAVAILABLE`).
- Raster clip and tile hardening:
- clip validates area presence and CRS alignment constraints.
- tile manifest records `tile_set_id`, `tile_size`, `overlap`, `source_dataset_id`, `source_raster_id`, bounds, parameters, count, tile paths, `ai_inference`, and `tile_server`.
- Job result persistence for raster ops:
- raster clip/reproject/tile job payloads persist derived dataset references when outputs are produced.
## Sprint 6 additions
- Added local spectral index operations:
- NDVI endpoint: `POST /raster/indices/ndvi`
- NDWI endpoint: `POST /raster/indices/ndwi`
- NDBI endpoint: `POST /raster/indices/ndbi`
- Spectral index input validation:
- band parameters must be positive integers
- band parameters must exist in source raster band count
- Dependency-aware execution:
- returns `RASTER_PROCESSING_UNAVAILABLE` when rasterio or numpy are unavailable
- Real index output handling:
- local windowed float32 GeoTIFF generation
- `NaN` strategy for invalid pixels / division by zero
- Provenance capture for derived index datasets:
- `source_dataset_id`, `operation`, `band_mapping`, `formula`
- `output_dtype`, `nodata_strategy`, `value_range_note`
- `output_dataset_id`, `created_at`, `path`
## Sprint 7B additions
- Added provider registry skeleton for `grb`, `osm`, `manual` and `fixture`.
- Added provider capability endpoints:
- `GET /api/v1/external/providers`
- `GET /api/v1/external/providers/{provider_name}`
- `GET /api/v1/external/providers/{provider_name}/layers`
- `GET /api/v1/external/providers/{provider_name}/status`
- `POST /api/v1/external/providers/{provider_name}/import`
- GRB and OSM imports return explicit `not_configured` responses; no live WFS or Overpass calls are made.
- Manual and fixture providers describe existing upload/fixture flows only.
- Added live PostGIS migration smoke script for environments with a real database:
```bash
bash scripts/live_migration_smoke.sh
```
## Sprint 8 additions
- Added Detection Lab foundation:
- `detections` ORM model and Alembic migration with PostGIS geometry storage.
- hardened `analysis_runs` for dataset/job/model/result metadata.
- model registry capability service for `yolo-placeholder` and `manual-fixture-detector`.
- detection service boundary for creating jobs, analysis runs and dependency-aware unavailable responses.
- Added detection endpoints:
- `GET /api/v1/detection/models`
- `POST /api/v1/detection/run`
- `GET /api/v1/detection/runs/{analysis_run_id}`
- `GET /api/v1/detection/runs/{analysis_run_id}/detections`
- YOLO/PyTorch real inference is not enabled in Sprint 8.
- Fixture detector mode is test/demo-only and requires explicit `fixture_mode=true`.
## Sprint 8B additions
- Added optional configured YOLO integration foundation:
- `yolo-configured` model registry capability.
- import-safe adapter for local Ultralytics model files.
- raster tile manifest validation and tile limit enforcement.
- pixel bbox to EPSG:4326 detection polygon conversion.
- persisted detections through the existing detection/job/analysis-run path.
- YOLO dependencies are optional extras and are not required for backend startup.
- GeoIntel does not download YOLO model weights automatically.
## Sprint 8C additions
- Added detection visualization/review API support:
- list detection runs
- list detections by run or dataset with class/confidence filters
- get detection detail
- return persisted detections as GeoJSON FeatureCollections
- Added detection QA against reference vector datasets:
- compares persisted detection geometries against persisted `vector_features`
- persists `quality_checks` and `metrics`
- returns precision, recall, F1, mean IoU and false positive/negative counts
- Segmentation, LiDAR, AI Copilot, Training Studio and Reports remain out of scope.
## Sprint 9 additions
- Added Segmentation Lab foundation:
- `segmentations` ORM model and Alembic migration with PostGIS MultiPolygon geometry storage.
- segmentation model registry capabilities for `segmentation-placeholder`, `fixture-segmenter`, `yolo-seg-configured` and `sam-configured`.
- segmentation service boundary for creating jobs, analysis runs and unavailable model responses.
- explicit fixture segmenter mode for tests/demo fixtures only.
- Added segmentation endpoints:
- `GET /api/v1/segmentation/models`
- `POST /api/v1/segmentation/run`
- `GET /api/v1/segmentation/runs`
- `GET /api/v1/segmentation/runs/{analysis_run_id}`
- `GET /api/v1/segmentation/runs/{analysis_run_id}/segmentations`
- `GET /api/v1/segmentation/runs/{analysis_run_id}/geojson`
- `POST /api/v1/segmentation/runs/{analysis_run_id}/qa/reference`
- Real SAM and YOLO-seg inference are not enabled in Sprint 9.
- Mask paths are provenance/debug artifacts; persisted PostGIS geometry is authoritative for QA, map display and GeoJSON.
## Sprint 17 additions
- Added export foundation backed by the existing `exports` table.
- GeoJSON exports now persist export records and write JSON artifacts for:
- vector datasets
- detection analysis runs
- segmentation analysis runs
- Added project metadata JSON export for project, dataset and QA/QC summary state.
- Added export read/list/content endpoints:
- `POST /api/v1/exports/geojson`
- `POST /api/v1/exports/metadata`
- `GET /api/v1/exports/projects/{project_id}/exports`
- `GET /api/v1/exports/{export_id}`
- `GET /api/v1/exports/{export_id}/content`
- Exported detection and segmentation GeoJSON is generated from persisted first-class geometry rows.
- No new migrations, product lines, live providers or AI dependencies are introduced by this export pass.
## Run locally
### Prerequisites
- Python 3.11+
- PostgreSQL with PostGIS
### Install dependencies
```bash
cd backend
python -m pip install -e .[dev]
```
Optional AI dependencies for configured local YOLO inference:
```bash
cd backend
python -m pip install -e .[ai]
```
Configured YOLO requires:
```bash
YOLO_ENABLED=true
YOLO_MODEL_PATH=/absolute/path/to/local-model.pt
```
Optional tuning:
```bash
YOLO_MODEL_ID=yolo-configured
YOLO_MODEL_DISPLAY_NAME="Configured YOLO detector"
YOLO_MODEL_VERSION=local-v1
YOLO_DEVICE=cpu
YOLO_IMAGE_SIZE=640
YOLO_MAX_TILES=100
YOLO_BATCH_SIZE=1
```
### YOLO local preflight
Sprint 13 adds a local-only preflight for configured YOLO paths:
```bash
python scripts/yolo_preflight.py --model-path /absolute/path/to/local-model.pt --tile-manifest-path /absolute/path/to/manifest.json
```
Machine-readable output:
```bash
python scripts/yolo_preflight.py --model-path /absolute/path/to/local-model.pt --tile-manifest-path /absolute/path/to/manifest.json --json
```
To validate only local model/manifest paths on a machine without optional AI dependencies:
```bash
python scripts/yolo_preflight.py --model-path /absolute/path/to/local-model.pt --tile-manifest-path /absolute/path/to/manifest.json --assume-dependencies --json
```
The preflight checks configuration, dependency availability, local model file existence, tile manifest validity, tile count and referenced tile paths. It does not load a YOLO model, run inference or download weights.
### Run backend
```bash
cd backend
python -m uvicorn app.main:app --reload
```
### Run backend tests
```bash
cd backend
python -m pytest
```
For warning-sensitive release checks, the backend is expected to pass with Python deprecation warnings promoted to errors for the timestamp-heavy service paths:
```bash
cd backend
python -m pytest -W error::DeprecationWarning tests/test_geojson_dataset_service.py tests/test_qa_service.py tests/test_sprint7a_persistence_foundation.py tests/test_sprint8c_detection_visualization_qa.py tests/test_sprint9_segmentation_foundation.py tests/test_vector_operations_service.py
```
The repository readiness gate now applies the same warning policy to the full backend suite:
```bash
bash scripts/run_readiness_check.sh
```
That readiness gate also runs the API contract smoke check before backend/frontend compilation and tests.
### Golden QA/QC benchmark
Sprint 12 includes a deterministic QA/QC regression benchmark using explicit fixture data:
```bash
python scripts/run_golden_qa_benchmark.py
```
Machine-readable output:
```bash
python scripts/run_golden_qa_benchmark.py --json
```
The benchmark compares `fixtures/golden/predicted_buildings.geojson` against `fixtures/golden/reference_buildings.geojson` and fails on metric drift. Expected baseline:
- precision: `0.5`
- recall: `0.5`
- F1: `0.5`
- mean IoU: `0.8339768339761133`
- false positives: `1`
- false negatives: `1`
The command uses existing QA/QC service logic and verifies `QualityCheck`/`Metric` persistence through an in-memory test session. It does not require live providers, AI models, Docker or PostGIS.
### Demo workflow seed
Sprint 15 adds an explicit offline demo workflow seed. It creates or returns a
demo project, AOI, fixture reference buildings, fixture candidate buildings and
a persisted QA/QC result. It does not fetch live GRB/OSM data and does not run
AI inference.
API:
```bash
curl -X POST http://localhost:1202/api/v1/demo/workflow
```
CLI:
```bash
python scripts/seed_demo_workflow.py --json
```
In Docker Compose on a LAN host:
```bash
curl -X POST http://192.168.10.150:1202/api/v1/demo/workflow
```
### QA/QC result listing
Persisted project quality checks and metric rows can be listed with:
```bash
curl http://localhost:1202/api/v1/projects/{project_id}/quality-checks
```
The frontend QA/QC Results panel uses this endpoint after loading the demo
workflow or running QA.
### Export foundation
Persisted exports can be created from the existing workbench state:
```bash
curl -X POST http://localhost:1202/api/v1/exports/metadata \
-H "Content-Type: application/json" \
-d '{"project_id":"PROJECT_UUID"}'
```
Vector dataset GeoJSON export:
```bash
curl -X POST http://localhost:1202/api/v1/exports/geojson \
-H "Content-Type: application/json" \
-d '{"export_kind":"dataset","dataset_id":"DATASET_UUID"}'
```
Detection or segmentation run GeoJSON export:
```bash
curl -X POST http://localhost:1202/api/v1/exports/geojson \
-H "Content-Type: application/json" \
-d '{"export_kind":"detection_run","analysis_run_id":"ANALYSIS_RUN_UUID"}'
```
List and inspect exports:
```bash
curl http://localhost:1202/api/v1/exports/projects/PROJECT_UUID/exports
curl http://localhost:1202/api/v1/exports/EXPORT_UUID/content
```
Download an artifact as a browser/file response:
```bash
curl -OJ http://localhost:1202/api/v1/exports/EXPORT_UUID/download
```
Create a lightweight HTML project report artifact:
```bash
curl -X POST http://localhost:1202/api/v1/exports/report \
-H "Content-Type: application/json" \
-d '{"project_id":"PROJECT_UUID"}'
```
The report contains project, dataset, QA/QC summary and export history state
only. It is not a PDF designer and does not add a separate reporting module.
After rebuilding a Docker/LAN deployment, verify the end-to-end demo and export
flow through the browser-facing frontend proxy:
```bash
bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202
```
The script seeds the explicit demo workflow, verifies persisted QA/QC results,
creates metadata/report/vector GeoJSON exports, lists exports and downloads the
JSON/GeoJSON/HTML artifacts.
### Backend import smoke
```bash
cd backend
python -c "from app.main import app; print(app.title)"
```
### Dockerized backend
```bash
docker compose up --build backend db
```
The Docker Compose stack does not require a root `.env` file for the default local runtime. The database service exposes a container-internal Postgres healthcheck, and the backend also runs `docker_start.sh`, which retries an actual SQL `SELECT 1` connection before running `python -m alembic upgrade head` and starting Uvicorn.
PostGIS is not published on the host `5432` port by default. This avoids conflicts with existing Postgres/PostGIS services on NAS or server hosts. The backend connects over Docker networking with `db:5432`.
Backend and frontend Docker build contexts exclude dependency folders, build outputs and Python bytecode caches via `.dockerignore`.
The Docker Compose frontend is published at `http://localhost:1202`.
Compose healthchecks are enabled for all runtime services:
- `db` uses `pg_isready`.
- `backend` checks `http://127.0.0.1:8000/health` inside the container.
- `frontend` checks `http://127.0.0.1/health` through nginx, which also verifies the frontend-to-backend proxy path.
The frontend waits for a healthy backend before starting. Check runtime state:
```bash
docker compose ps
docker compose logs --tail=80 backend
docker compose logs --tail=80 frontend
```
The backend Docker image installs the approved GIS runtime extra (`.[gis]`) so
browser-facing Docker deployments can report raster/vector processing
capabilities accurately:
- `rasterio`
- `numpy`
- `pillow`
- `geopandas`
- `pyogrio`
- GDAL/GEOS/PROJ system libraries
After rebuilding the backend image, verify the LAN/browser runtime from the
repository root:
```bash
bash scripts/verify_gis_runtime.sh http://localhost:1202
```
On a NAS or server host, use the published LAN URL:
```bash
bash scripts/verify_gis_runtime.sh http://192.168.10.150:1202
```
The script calls `/api/v1/system/capabilities` through the frontend proxy and
fails if `postgis`, `rasterio` or `geopandas` are not reported as available.
The backend Docker build also runs:
```bash
python scripts/gis_import_smoke.py
```
Inside the backend Docker build context this resolves to
`backend/scripts/gis_import_smoke.py`. The root `scripts/gis_import_smoke.py`
wrapper calls the same smoke locally. The smoke imports `rasterio`, `geopandas`
and `pyogrio`; if one of those imports fails, the backend image build fails
before deployment.
### Live Docker/PostGIS migration smoke
Sprint 11 validates the real PostGIS runtime path with the existing database service. From the repository root:
```bash
docker compose config
docker compose up -d db
DATABASE_URL=postgresql+psycopg://geointel:geointel@localhost:5432/geointel bash scripts/live_migration_smoke.sh
```
The smoke script:
- opens a backend SQLAlchemy connection and runs `SELECT 1`
- runs `alembic upgrade head`
- checks `PostGIS_Version()` after migrations have created the extension
- verifies one Alembic head
- verifies required migrated tables and GiST indexes exist
Expected local environment:
```bash
DATABASE_URL=postgresql+psycopg://geointel:geointel@localhost:5432/geointel
```
If the database is not reachable, confirm Docker Desktop is running and that port `5432` is not already occupied. To clean up the local database container without deleting the named volume:
```bash
docker compose stop db
```
To remove the local PostGIS volume as well, use only when you explicitly want a fresh database:
```bash
docker compose down -v
```
## Key docs
- `docs/API_CONTRACTS.md`
- `docs/DATABASE_IMPLEMENTATION_PLAN.md`
- `docs/DEFINITION_OF_DONE.md`
- `docs/40-build-launch/SPRINT_1_SCOPE_FREEZE.md`
## Raster dependency note
Raster metadata and raster operations depend on local GDAL/rasterio availability.
To enable raster processing locally:
```bash
python -m pip install rasterio
```
If `rasterio` is unavailable:
- raster metadata responses return `503` with `RASTER_PROCESSING_UNAVAILABLE`
- raster clip/tile endpoints return explicit unavailable responses
## Helpful repository scripts
- `bash scripts/backend_install.sh`
- `bash scripts/backend_test.sh`
- `bash scripts/backend_dev.sh`
- `bash scripts/smoke_backend_import.sh`
backend/alembic.ini
+38
View File
@@ -0,0 +1,38 @@
[alembic]
script_location = alembic
prepend_sys_path = .
sqlalchemy.url = postgresql+psycopg://geointel:geointel@localhost:5432/geointel
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARNING
handlers = console
qualname =
[logger_sqlalchemy]
level = INFO
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
class_ = logging.Formatter
backend/alembic/env.py
+48
View File
@@ -0,0 +1,48 @@
from __future__ import annotations
import os
import sys
from logging.config import fileConfig
from alembic import context
from sqlalchemy import engine_from_config, pool
sys.path.append(os.path.realpath(os.path.join(os.path.dirname(__file__), '..')))
from app.core.config import get_settings
from app.db.base import Base
import app.models.entities # noqa: F401
settings = get_settings()
config = context.config
if config.config_file_name is not None:
fileConfig(config.config_file_name)
config.set_main_option("sqlalchemy.url", settings.database_url)
target_metadata = Base.metadata
def run_migrations_offline() -> None:
url = config.get_main_option("sqlalchemy.url")
context.configure(url=url, target_metadata=target_metadata, literal_binds=True)
with context.begin_transaction():
context.run_migrations()
def run_migrations_online() -> None:
connectable = engine_from_config(
config.get_section(config.config_ini_section, {}),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
with connectable.connect() as connection:
context.configure(connection=connection, target_metadata=target_metadata)
with context.begin_transaction():
context.run_migrations()
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()
backend/alembic/script.py.mako
+20
View File
@@ -0,0 +1,20 @@
"""
${message}
"""
from alembic import op
import sqlalchemy as sa
${imports}
revision = ${repr(revision)}
down_revision = ${repr(down_revision)}
branch_labels = ${repr(branch_labels)}
depends_on = ${repr(depends_on)}
def upgrade():
${upgrades if upgrades else "pass"}
def downgrade():
${downgrades if downgrades else "pass"}
backend/alembic/versions/202601110001_initial.py
+108
View File
@@ -0,0 +1,108 @@
"""Initial PostGIS schema for Sprint 1 foundation."""
from alembic import op
import sqlalchemy as sa
from geoalchemy2 import Geometry
revision = "202601110001"
down_revision = None
branch_labels = None
depends_on = None
def upgrade() -> None:
op.execute("CREATE EXTENSION IF NOT EXISTS postgis")
op.execute("CREATE EXTENSION IF NOT EXISTS postgis_topology")
op.execute('CREATE EXTENSION IF NOT EXISTS "uuid-ossp"')
op.create_table(
"projects",
sa.Column("id", sa.UUID(as_uuid=True), primary_key=True),
sa.Column("name", sa.Text(), nullable=False),
sa.Column("description", sa.Text(), nullable=True),
sa.Column("region", sa.Text(), nullable=False, server_default="Kempen"),
sa.Column("status", sa.Text(), nullable=False, server_default="active"),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("NOW()")),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.text("NOW()")),
)
op.create_table(
"areas",
sa.Column("id", sa.UUID(as_uuid=True), primary_key=True),
sa.Column("project_id", sa.UUID(as_uuid=True), sa.ForeignKey("projects.id", ondelete="CASCADE"), nullable=False),
sa.Column("name", sa.Text(), nullable=False),
sa.Column("geometry", Geometry("MULTIPOLYGON", srid=4326), nullable=False),
sa.Column("original_crs", sa.Text(), nullable=True),
sa.Column("area_m2", sa.Float(), nullable=True),
sa.Column("bbox", Geometry("POLYGON", srid=4326), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("NOW()")),
)
op.create_table(
"datasets",
sa.Column("id", sa.UUID(as_uuid=True), primary_key=True),
sa.Column("project_id", sa.UUID(as_uuid=True), sa.ForeignKey("projects.id", ondelete="CASCADE"), nullable=False),
sa.Column("area_id", sa.UUID(as_uuid=True), sa.ForeignKey("areas.id", ondelete="SET NULL"), nullable=True),
sa.Column("name", sa.Text(), nullable=False),
sa.Column("dataset_type", sa.Text(), nullable=False),
sa.Column("source", sa.Text(), nullable=False),
sa.Column("storage_path", sa.Text(), nullable=True),
sa.Column("derived_from_dataset_id", sa.UUID(as_uuid=True), sa.ForeignKey("datasets.id", ondelete="SET NULL"), nullable=True),
sa.Column("crs", sa.Text(), nullable=True),
sa.Column("bounds_json", sa.JSON(), nullable=True),
sa.Column("resolution_json", sa.JSON(), nullable=True),
sa.Column("bands_json", sa.JSON(), nullable=True),
sa.Column("metadata_json", sa.JSON(), nullable=True),
sa.Column("status", sa.Text(), nullable=False, server_default="created"),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("NOW()")),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.text("NOW()")),
)
op.create_table(
"dataset_versions",
sa.Column("id", sa.UUID(as_uuid=True), primary_key=True),
sa.Column("dataset_id", sa.UUID(as_uuid=True), sa.ForeignKey("datasets.id", ondelete="CASCADE"), nullable=False),
sa.Column("version", sa.Integer(), nullable=False, server_default="1"),
sa.Column("storage_path", sa.Text(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("NOW()")),
)
op.create_table(
"analysis_runs",
sa.Column("id", sa.UUID(as_uuid=True), primary_key=True),
sa.Column("project_id", sa.UUID(as_uuid=True), sa.ForeignKey("projects.id", ondelete="CASCADE"), nullable=False),
sa.Column("area_id", sa.UUID(as_uuid=True), sa.ForeignKey("areas.id", ondelete="SET NULL"), nullable=True),
sa.Column("analysis_type", sa.Text(), nullable=False),
sa.Column("status", sa.Text(), nullable=False),
sa.Column("parameters_json", sa.JSON(), nullable=False),
sa.Column("started_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("finished_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("error_message", sa.Text(), nullable=True),
)
op.create_table(
"exports",
sa.Column("id", sa.UUID(as_uuid=True), primary_key=True),
sa.Column("project_id", sa.UUID(as_uuid=True), sa.ForeignKey("projects.id", ondelete="CASCADE"), nullable=False),
sa.Column("analysis_run_id", sa.UUID(as_uuid=True), sa.ForeignKey("analysis_runs.id", ondelete="SET NULL"), nullable=True),
sa.Column("export_type", sa.Text(), nullable=False),
sa.Column("storage_path", sa.Text(), nullable=False),
sa.Column("metadata_json", sa.JSON(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("NOW()")),
)
op.create_index("ix_areas_geometry", "areas", ["geometry"], postgresql_using="gist")
op.create_index("ix_areas_project_id", "areas", ["project_id"])
op.create_index("ix_datasets_project_id", "datasets", ["project_id"])
def downgrade() -> None:
op.drop_index("ix_datasets_project_id", table_name="datasets")
op.drop_index("ix_areas_project_id", table_name="areas")
op.drop_index("ix_areas_geometry", table_name="areas", postgresql_using="gist")
op.drop_table("exports")
op.drop_table("analysis_runs")
op.drop_table("dataset_versions")
op.drop_table("datasets")
op.drop_table("areas")
op.drop_table("projects")
backend/alembic/versions/202601120001_dataset_storage_metadata.py
+27
View File
@@ -0,0 +1,27 @@
"""Add dataset storage metadata columns."""
from alembic import op
import sqlalchemy as sa
revision = "202601120001"
down_revision = "202601110001"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column("datasets", sa.Column("original_filename", sa.Text(), nullable=True))
op.add_column("datasets", sa.Column("stored_filename", sa.Text(), nullable=True))
op.add_column("datasets", sa.Column("content_type", sa.Text(), nullable=True))
op.add_column("datasets", sa.Column("size_bytes", sa.Integer(), nullable=True))
op.add_column("datasets", sa.Column("checksum_sha256", sa.Text(), nullable=True))
op.alter_column("datasets", "status", server_default="uploaded")
def downgrade() -> None:
op.drop_column("datasets", "checksum_sha256")
op.drop_column("datasets", "size_bytes")
op.drop_column("datasets", "content_type")
op.drop_column("datasets", "stored_filename")
op.drop_column("datasets", "original_filename")
backend/alembic/versions/20260611212435_add_jobs_table.py
+38
View File
@@ -0,0 +1,38 @@
"""Add lightweight job table for sprint-3 async architecture foundation."""
from alembic import op
import sqlalchemy as sa
revision = "20260611212435"
down_revision = "202601120001"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_table(
"jobs",
sa.Column("id", sa.UUID(as_uuid=True), primary_key=True),
sa.Column("job_type", sa.Text(), nullable=False),
sa.Column("status", sa.Text(), nullable=False, server_default="queued"),
sa.Column("project_id", sa.UUID(as_uuid=True), sa.ForeignKey("projects.id", ondelete="CASCADE"), nullable=False),
sa.Column("dataset_id", sa.UUID(as_uuid=True), sa.ForeignKey("datasets.id", ondelete="SET NULL"), nullable=True),
sa.Column("input_dataset_id", sa.UUID(as_uuid=True), sa.ForeignKey("datasets.id", ondelete="SET NULL"), nullable=True),
sa.Column("output_dataset_id", sa.UUID(as_uuid=True), sa.ForeignKey("datasets.id", ondelete="SET NULL"), nullable=True),
sa.Column("parameters_json", sa.JSON(), nullable=False),
sa.Column("result_json", sa.JSON(), nullable=True),
sa.Column("error_message", sa.Text(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("NOW()")),
sa.Column("started_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("finished_at", sa.DateTime(timezone=True), nullable=True),
)
op.create_index("ix_jobs_project_id", "jobs", ["project_id"])
op.create_index("ix_jobs_status", "jobs", ["status"])
def downgrade() -> None:
op.drop_index("ix_jobs_status", table_name="jobs")
op.drop_index("ix_jobs_project_id", table_name="jobs")
op.drop_table("jobs")
backend/alembic/versions/202606120001_add_dataset_reference_metadata.py
+28
View File
@@ -0,0 +1,28 @@
"""Add dataset reference and provenance metadata columns."""
from alembic import op
import sqlalchemy as sa
revision = "202606120001"
down_revision = "20260611212435"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column("datasets", sa.Column("dataset_role", sa.Text(), nullable=False, server_default="source"))
op.add_column("datasets", sa.Column("source_name", sa.Text(), nullable=True))
op.add_column("datasets", sa.Column("reference_layer_name", sa.Text(), nullable=True))
op.add_column("datasets", sa.Column("source_metadata", sa.JSON(), nullable=True))
op.add_column("datasets", sa.Column("provenance_metadata", sa.JSON(), nullable=True))
op.add_column("datasets", sa.Column("imported_at", sa.DateTime(timezone=True), server_default=sa.text("NOW()"), nullable=False))
def downgrade() -> None:
op.drop_column("datasets", "imported_at")
op.drop_column("datasets", "provenance_metadata")
op.drop_column("datasets", "source_metadata")
op.drop_column("datasets", "reference_layer_name")
op.drop_column("datasets", "source_name")
op.drop_column("datasets", "dataset_role")
backend/alembic/versions/202606120700_sprint7a_persistence_foundation.py
+76
View File
@@ -0,0 +1,76 @@
"""Add Sprint 7A vector feature and QA persistence foundation."""
from alembic import op
import sqlalchemy as sa
from geoalchemy2 import Geometry
revision = "202606120700"
down_revision = "202606120001"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_table(
"vector_features",
sa.Column("id", sa.UUID(as_uuid=True), primary_key=True),
sa.Column("dataset_id", sa.UUID(as_uuid=True), sa.ForeignKey("datasets.id", ondelete="CASCADE"), nullable=False),
sa.Column("feature_class", sa.Text(), nullable=True),
sa.Column("source_feature_id", sa.Text(), nullable=True),
sa.Column("properties_json", sa.JSON(), nullable=True),
sa.Column("geometry", Geometry("GEOMETRY", srid=4326, spatial_index=False), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("NOW()")),
)
op.create_index("ix_vector_features_dataset_id", "vector_features", ["dataset_id"])
op.create_index("ix_vector_features_geometry", "vector_features", ["geometry"], postgresql_using="gist")
op.create_table(
"quality_checks",
sa.Column("id", sa.UUID(as_uuid=True), primary_key=True),
sa.Column("project_id", sa.UUID(as_uuid=True), sa.ForeignKey("projects.id", ondelete="CASCADE"), nullable=False),
sa.Column("job_id", sa.UUID(as_uuid=True), sa.ForeignKey("jobs.id", ondelete="SET NULL"), nullable=True),
sa.Column("analysis_run_id", sa.UUID(as_uuid=True), sa.ForeignKey("analysis_runs.id", ondelete="SET NULL"), nullable=True),
sa.Column("candidate_dataset_id", sa.UUID(as_uuid=True), sa.ForeignKey("datasets.id", ondelete="SET NULL"), nullable=True),
sa.Column("reference_dataset_id", sa.UUID(as_uuid=True), sa.ForeignKey("datasets.id", ondelete="CASCADE"), nullable=False),
sa.Column("check_type", sa.Text(), nullable=False),
sa.Column("status", sa.Text(), nullable=False),
sa.Column("score", sa.Float(), nullable=True),
sa.Column("parameters_json", sa.JSON(), nullable=True),
sa.Column("findings_json", sa.JSON(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("NOW()")),
sa.Column("completed_at", sa.DateTime(timezone=True), nullable=True),
)
op.create_index("ix_quality_checks_project_id", "quality_checks", ["project_id"])
op.create_index("ix_quality_checks_reference_dataset_id", "quality_checks", ["reference_dataset_id"])
op.create_index("ix_quality_checks_candidate_dataset_id", "quality_checks", ["candidate_dataset_id"])
op.create_index("ix_quality_checks_analysis_run_id", "quality_checks", ["analysis_run_id"])
op.create_table(
"metrics",
sa.Column("id", sa.UUID(as_uuid=True), primary_key=True),
sa.Column("quality_check_id", sa.UUID(as_uuid=True), sa.ForeignKey("quality_checks.id", ondelete="CASCADE"), nullable=True),
sa.Column("analysis_run_id", sa.UUID(as_uuid=True), sa.ForeignKey("analysis_runs.id", ondelete="SET NULL"), nullable=True),
sa.Column("metric_key", sa.Text(), nullable=False),
sa.Column("metric_value", sa.Float(), nullable=True),
sa.Column("metric_unit", sa.Text(), nullable=True),
sa.Column("label", sa.Text(), nullable=True),
sa.Column("metadata_json", sa.JSON(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("NOW()")),
)
op.create_index("ix_metrics_quality_check_id", "metrics", ["quality_check_id"])
op.create_index("ix_metrics_analysis_run_id", "metrics", ["analysis_run_id"])
def downgrade() -> None:
op.drop_index("ix_metrics_analysis_run_id", table_name="metrics")
op.drop_index("ix_metrics_quality_check_id", table_name="metrics")
op.drop_table("metrics")
op.drop_index("ix_quality_checks_analysis_run_id", table_name="quality_checks")
op.drop_index("ix_quality_checks_candidate_dataset_id", table_name="quality_checks")
op.drop_index("ix_quality_checks_reference_dataset_id", table_name="quality_checks")
op.drop_index("ix_quality_checks_project_id", table_name="quality_checks")
op.drop_table("quality_checks")
op.drop_index("ix_vector_features_geometry", table_name="vector_features", postgresql_using="gist")
op.drop_index("ix_vector_features_dataset_id", table_name="vector_features")
op.drop_table("vector_features")
backend/alembic/versions/202606120800_sprint8_detection_foundation.py
+59
View File
@@ -0,0 +1,59 @@
"""Add Sprint 8 detection foundation."""
from alembic import op
import sqlalchemy as sa
from geoalchemy2 import Geometry
revision = "202606120800"
down_revision = "202606120700"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column("analysis_runs", sa.Column("dataset_id", sa.UUID(as_uuid=True), sa.ForeignKey("datasets.id", ondelete="SET NULL"), nullable=True))
op.add_column("analysis_runs", sa.Column("job_id", sa.UUID(as_uuid=True), sa.ForeignKey("jobs.id", ondelete="SET NULL"), nullable=True))
op.add_column("analysis_runs", sa.Column("model_name", sa.String(length=255), nullable=True))
op.add_column("analysis_runs", sa.Column("model_version", sa.String(length=120), nullable=True))
op.add_column("analysis_runs", sa.Column("result_json", sa.JSON(), nullable=True))
op.add_column("analysis_runs", sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("NOW()"), nullable=False))
op.create_table(
"detections",
sa.Column("id", sa.UUID(as_uuid=True), primary_key=True),
sa.Column("project_id", sa.UUID(as_uuid=True), sa.ForeignKey("projects.id", ondelete="CASCADE"), nullable=False),
sa.Column("dataset_id", sa.UUID(as_uuid=True), sa.ForeignKey("datasets.id", ondelete="SET NULL"), nullable=True),
sa.Column("analysis_run_id", sa.UUID(as_uuid=True), sa.ForeignKey("analysis_runs.id", ondelete="SET NULL"), nullable=True),
sa.Column("job_id", sa.UUID(as_uuid=True), sa.ForeignKey("jobs.id", ondelete="SET NULL"), nullable=True),
sa.Column("model_name", sa.String(length=255), nullable=False),
sa.Column("model_version", sa.String(length=120), nullable=True),
sa.Column("class_name", sa.String(length=120), nullable=False),
sa.Column("confidence", sa.Float(), nullable=False),
sa.Column("geometry", Geometry("GEOMETRY", srid=4326, spatial_index=False), nullable=False),
sa.Column("bbox_json", sa.JSON(), nullable=True),
sa.Column("source_tile_path", sa.String(length=500), nullable=True),
sa.Column("properties_json", sa.JSON(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("NOW()"), nullable=False),
)
op.create_index("ix_detections_project_id", "detections", ["project_id"])
op.create_index("ix_detections_dataset_id", "detections", ["dataset_id"])
op.create_index("ix_detections_analysis_run_id", "detections", ["analysis_run_id"])
op.create_index("ix_detections_class_name", "detections", ["class_name"])
op.create_index("ix_detections_geometry", "detections", ["geometry"], postgresql_using="gist")
def downgrade() -> None:
op.drop_index("ix_detections_geometry", table_name="detections", postgresql_using="gist")
op.drop_index("ix_detections_class_name", table_name="detections")
op.drop_index("ix_detections_analysis_run_id", table_name="detections")
op.drop_index("ix_detections_dataset_id", table_name="detections")
op.drop_index("ix_detections_project_id", table_name="detections")
op.drop_table("detections")
op.drop_column("analysis_runs", "created_at")
op.drop_column("analysis_runs", "result_json")
op.drop_column("analysis_runs", "model_version")
op.drop_column("analysis_runs", "model_name")
op.drop_column("analysis_runs", "job_id")
op.drop_column("analysis_runs", "dataset_id")
backend/alembic/versions/202606120900_sprint9_segmentation_foundation.py
+51
View File
@@ -0,0 +1,51 @@
"""Add Sprint 9 segmentation foundation."""
from alembic import op
import sqlalchemy as sa
from geoalchemy2 import Geometry
revision = "202606120900"
down_revision = "202606120800"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_table(
"segmentations",
sa.Column("id", sa.UUID(as_uuid=True), primary_key=True),
sa.Column("project_id", sa.UUID(as_uuid=True), sa.ForeignKey("projects.id", ondelete="CASCADE"), nullable=False),
sa.Column("dataset_id", sa.UUID(as_uuid=True), sa.ForeignKey("datasets.id", ondelete="SET NULL"), nullable=True),
sa.Column("job_id", sa.UUID(as_uuid=True), sa.ForeignKey("jobs.id", ondelete="SET NULL"), nullable=True),
sa.Column("analysis_run_id", sa.UUID(as_uuid=True), sa.ForeignKey("analysis_runs.id", ondelete="SET NULL"), nullable=True),
sa.Column("model_name", sa.String(length=255), nullable=False),
sa.Column("model_version", sa.String(length=120), nullable=True),
sa.Column("class_name", sa.String(length=120), nullable=False),
sa.Column("confidence", sa.Float(), nullable=True),
sa.Column("geometry", Geometry("MultiPolygon", srid=4326, spatial_index=False), nullable=False),
sa.Column("bbox_json", sa.JSON(), nullable=True),
sa.Column("area_m2", sa.Float(), nullable=True),
sa.Column("mask_path", sa.Text(), nullable=True),
sa.Column("source_tile_path", sa.String(length=500), nullable=True),
sa.Column("tile_index", sa.Integer(), nullable=True),
sa.Column("properties_json", sa.JSON(), nullable=True),
sa.Column("provenance_json", sa.JSON(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("NOW()"), nullable=False),
)
op.create_index("ix_segmentations_project_id", "segmentations", ["project_id"])
op.create_index("ix_segmentations_dataset_id", "segmentations", ["dataset_id"])
op.create_index("ix_segmentations_analysis_run_id", "segmentations", ["analysis_run_id"])
op.create_index("ix_segmentations_job_id", "segmentations", ["job_id"])
op.create_index("ix_segmentations_class_name", "segmentations", ["class_name"])
op.create_index("ix_segmentations_geometry", "segmentations", ["geometry"], postgresql_using="gist")
def downgrade() -> None:
op.drop_index("ix_segmentations_geometry", table_name="segmentations", postgresql_using="gist")
op.drop_index("ix_segmentations_class_name", table_name="segmentations")
op.drop_index("ix_segmentations_job_id", table_name="segmentations")
op.drop_index("ix_segmentations_analysis_run_id", table_name="segmentations")
op.drop_index("ix_segmentations_dataset_id", table_name="segmentations")
op.drop_index("ix_segmentations_project_id", table_name="segmentations")
op.drop_table("segmentations")
backend/app/.gitkeep
View File
backend/app/__init__.py
+3
View File
@@ -0,0 +1,3 @@
from app.models.entities import AnalysisRun, Area, Dataset, Export, Project
__all__ = ["AnalysisRun", "Area", "Dataset", "Export", "Project"]
backend/app/ai/.gitkeep
View File
backend/app/analysis/.gitkeep
View File
backend/app/api/.gitkeep
View File
backend/app/api/routes/.gitkeep
View File
backend/app/api/routes/__init__.py
+1
View File
@@ -0,0 +1 @@
__all__ = ["areas", "datasets", "health", "projects", "exports", "jobs", "external", "qa"]
backend/app/api/routes/areas.py
+58
View File
@@ -0,0 +1,58 @@
from __future__ import annotations
from uuid import UUID
from fastapi import APIRouter, Depends, Query
from fastapi import HTTPException
from sqlalchemy.orm import Session
from app.db.session import get_db
from app.models import Area
from app.schemas.area import AreaCreate, AreaRead, AreaUpdate
from app.services.area_service import AreaService
from app.utils.response import envelope
router = APIRouter(prefix="/projects/{project_id}/areas", tags=["areas"])
@router.get("", response_model=dict)
def list_areas(
project_id: UUID,
limit: int = Query(default=50, ge=1, le=200),
offset: int = Query(default=0, ge=0),
db: Session = Depends(get_db),
):
areas, total = AreaService.list_areas(db, project_id=project_id, limit=limit, offset=offset)
return envelope({"items": [AreaRead.model_validate(area).model_dump() for area in areas], "total": total, "limit": limit, "offset": offset})
@router.post("", status_code=201, response_model=dict)
def create_area(project_id: UUID, payload: AreaCreate, db: Session = Depends(get_db)):
area = AreaService.create_area(db, project_id, payload)
return envelope(AreaRead.model_validate(area).model_dump())
@router.get("/{area_id}", response_model=dict)
def get_area(
project_id: UUID,
area_id: UUID,
db: Session = Depends(get_db),
):
area = AreaService.get_area(db, area_id)
if area.project_id != project_id:
raise HTTPException(status_code=404, detail="Area not found")
return envelope(AreaRead.model_validate(area).model_dump())
@router.patch("/{area_id}", response_model=dict)
def update_area(
project_id: UUID,
area_id: UUID,
payload: AreaUpdate,
db: Session = Depends(get_db),
):
existing = db.get(Area, area_id)
if not existing or existing.project_id != project_id:
raise HTTPException(status_code=404, detail="Area not found")
area = AreaService.update_area(db, area_id, payload)
return envelope(AreaRead.model_validate(area).model_dump())
backend/app/api/routes/datasets.py
+489
View File
@@ -0,0 +1,489 @@
from __future__ import annotations
import json
from typing import Any
from uuid import UUID
from uuid import UUID as _UUID
from fastapi import APIRouter, Depends, File, Form, HTTPException, Query
from fastapi import UploadFile
from sqlalchemy.orm import Session
from app.models import Area
from app.core.errors import AppError
from app.db.session import get_db
from app.schemas import (
RasterClipRequest,
RasterStatsResponse,
RasterReprojectRequest,
RasterTileRequest,
RasterNdviRequest,
RasterNdwiRequest,
RasterNdbiRequest,
VectorBBoxResponse,
VectorBufferRequest,
VectorClipRequest,
VectorIntersectRequest,
)
from app.schemas.job import JobCreate
from app.schemas.dataset import DatasetCreateResponse
from app.schemas.operations import VectorOperationResult
from app.services.job_service import JobService
from app.services.raster_operations_service import RasterOperationsService
from app.services.vector_operations_service import VectorOperationsService
from app.services.dataset_service import DatasetService
from app.utils.response import envelope
router = APIRouter(prefix="/projects/{project_id}", tags=["datasets"])
def _parse_metadata_json(raw: str | None, field_name: str) -> dict | None:
if raw is None:
return None
raw = raw.strip()
if not raw:
return None
try:
value = json.loads(raw)
except json.JSONDecodeError as exc:
raise AppError(code="INVALID_JSON", message=f"Invalid JSON for {field_name}", details={"field": field_name}, status_code=400) from exc
if not isinstance(value, dict):
raise AppError(code="INVALID_JSON", message=f"{field_name} must be a JSON object", details={"field": field_name}, status_code=400)
return value
def _run_job_sync(
db: Session,
project_id: UUID,
input_dataset_id: UUID,
job_type: str,
parameters: dict[str, Any],
operation,
) -> dict[str, Any]:
return JobService.run_sync_job(
db=db,
project_id=project_id,
job_type=job_type,
parameters=parameters,
operation=operation,
input_dataset_id=input_dataset_id,
)
@router.post("/datasets/upload", status_code=201, response_model=dict)
async def upload_dataset(
project_id: UUID,
file: UploadFile = File(...),
dataset_type: str = Form(...),
source: str = Form("user_upload"),
area_id: UUID | None = Form(None),
dataset_role: str = Form("source"),
source_name: str | None = Form(None),
reference_layer_name: str | None = Form(None),
source_metadata_json: str | None = Form(None),
provenance_metadata_json: str | None = Form(None),
db: Session = Depends(get_db),
):
if area_id is not None:
area = db.get(Area, area_id)
if not area:
raise AppError(code="AREA_NOT_FOUND", message="Area not found", status_code=404)
if area.project_id != project_id:
raise AppError(code="INVALID_DATASET_SCOPE", message="Area does not belong to this project", status_code=400)
created = await DatasetService.upload_dataset(
db,
project_id=project_id,
file=file,
dataset_type=dataset_type,
source=source,
dataset_role=dataset_role,
source_name=source_name,
reference_layer_name=reference_layer_name,
source_metadata=_parse_metadata_json(source_metadata_json, "source_metadata_json"),
provenance_metadata=_parse_metadata_json(provenance_metadata_json, "provenance_metadata_json"),
area_id=area_id,
)
return envelope(created.model_dump())
@router.get("/datasets", response_model=dict)
def list_datasets(
project_id: UUID,
limit: int = Query(default=50, ge=1, le=200),
offset: int = Query(default=0, ge=0),
db: Session = Depends(get_db),
):
datasets, total = DatasetService.list_datasets(db, project_id, limit=limit, offset=offset)
return envelope({"items": [item.model_dump() for item in datasets], "total": total, "limit": limit, "offset": offset})
@router.get("/datasets/{dataset_id}", response_model=dict)
def get_dataset(
project_id: UUID,
dataset_id: UUID,
db: Session = Depends(get_db),
):
dataset = DatasetService.get_dataset(db, dataset_id)
if dataset.project_id != project_id:
raise HTTPException(status_code=404, detail="Dataset not found")
return envelope(DatasetCreateResponse.model_validate(dataset).model_dump())
@router.post("/datasets/{dataset_id}/metadata/refresh", response_model=dict)
def refresh_dataset_metadata(
project_id: UUID,
dataset_id: UUID,
db: Session = Depends(get_db),
):
dataset = DatasetService.get_dataset(db, dataset_id)
if dataset.project_id != project_id:
raise HTTPException(status_code=404, detail="Dataset not found")
refreshed = DatasetService.refresh_metadata(db, dataset_id)
return envelope(refreshed.model_dump())
@router.get("/datasets/{dataset_id}/vector/inspect", response_model=dict)
def inspect_vector_dataset(
project_id: UUID,
dataset_id: UUID,
db: Session = Depends(get_db),
):
dataset = DatasetService.get_dataset(db, dataset_id)
if dataset.project_id != project_id:
raise HTTPException(status_code=404, detail="Dataset not found")
return envelope(VectorOperationsService.inspect(db, dataset_id).model_dump())
@router.get("/datasets/{dataset_id}/vector/bbox", response_model=dict)
def vector_bbox(
project_id: UUID,
dataset_id: UUID,
db: Session = Depends(get_db),
):
dataset = DatasetService.get_dataset(db, dataset_id)
if dataset.project_id != project_id:
raise HTTPException(status_code=404, detail="Dataset not found")
payload = VectorOperationsService.bbox(db, dataset_id)
return envelope(VectorBBoxResponse(**payload).model_dump())
@router.get("/datasets/{dataset_id}/vector/stats", response_model=dict)
def vector_stats(
project_id: UUID,
dataset_id: UUID,
db: Session = Depends(get_db),
):
dataset = DatasetService.get_dataset(db, dataset_id)
if dataset.project_id != project_id:
raise HTTPException(status_code=404, detail="Dataset not found")
return envelope(VectorOperationsService.stats(db, dataset_id))
@router.post("/datasets/{dataset_id}/vector/clip", status_code=201, response_model=dict)
def clip_vector_dataset(
project_id: UUID,
dataset_id: UUID,
payload: VectorClipRequest,
db: Session = Depends(get_db),
):
dataset = DatasetService.get_dataset(db, dataset_id)
if dataset.project_id != project_id:
raise HTTPException(status_code=404, detail="Dataset not found")
job = _run_job_sync(
db=db,
project_id=project_id,
input_dataset_id=dataset_id,
job_type="vector.clip",
parameters=payload.model_dump(),
operation=lambda: VectorOperationsService.clip_by_area(
db,
dataset_id=dataset_id,
area_id=payload.area_id,
output_name=payload.output_name,
),
)
return envelope(job)
@router.post("/datasets/{dataset_id}/vector/buffer", status_code=201, response_model=dict)
def buffer_vector_dataset(
project_id: UUID,
dataset_id: UUID,
payload: VectorBufferRequest,
db: Session = Depends(get_db),
):
dataset = DatasetService.get_dataset(db, dataset_id)
if dataset.project_id != project_id:
raise HTTPException(status_code=404, detail="Dataset not found")
job = _run_job_sync(
db=db,
project_id=project_id,
input_dataset_id=dataset_id,
job_type="vector.buffer",
parameters=payload.model_dump(),
operation=lambda: VectorOperationsService.buffer(
db,
dataset_id=dataset_id,
distance_m=payload.distance_m,
dissolve=payload.dissolve,
output_name=payload.output_name,
),
)
return envelope(job)
@router.post("/datasets/{dataset_id}/vector/intersect", status_code=201, response_model=dict)
def intersect_vector_dataset(
project_id: UUID,
dataset_id: UUID,
payload: VectorIntersectRequest,
db: Session = Depends(get_db),
):
dataset = DatasetService.get_dataset(db, dataset_id)
if dataset.project_id != project_id:
raise HTTPException(status_code=404, detail="Dataset not found")
job = _run_job_sync(
db=db,
project_id=project_id,
input_dataset_id=dataset_id,
job_type="vector.intersect",
parameters=payload.model_dump(),
operation=lambda: VectorOperationsService.intersect(
db,
source_dataset_id=dataset_id,
target_dataset_id=UUID(payload.other_dataset_id),
output_name=payload.output_name,
),
)
return envelope(job)
@router.get("/datasets/{dataset_id}/vector/summary", response_model=dict)
def vector_dataset_summary(
project_id: UUID,
dataset_id: UUID,
db: Session = Depends(get_db),
):
dataset = DatasetService.get_dataset(db, dataset_id)
if dataset.project_id != project_id:
raise HTTPException(status_code=404, detail="Dataset not found")
return envelope(DatasetService.vector_summary(db, dataset_id))
@router.get("/datasets/{dataset_id}/raster/inspect", response_model=dict)
def raster_dataset_inspect(
project_id: UUID,
dataset_id: UUID,
db: Session = Depends(get_db),
):
dataset = DatasetService.get_dataset(db, dataset_id)
if dataset.project_id != project_id:
raise HTTPException(status_code=404, detail="Dataset not found")
payload = RasterOperationsService.inspect(db, dataset_id)
return envelope(payload)
@router.get("/datasets/{dataset_id}/raster/preview", response_model=dict)
def raster_preview_readiness(
project_id: UUID,
dataset_id: UUID,
db: Session = Depends(get_db),
):
dataset = DatasetService.get_dataset(db, dataset_id)
if dataset.project_id != project_id:
raise HTTPException(status_code=404, detail="Dataset not found")
return envelope(RasterOperationsService.preview(db, dataset_id))
@router.get("/datasets/{dataset_id}/raster/stats", response_model=dict)
def raster_stats(
project_id: UUID,
dataset_id: UUID,
db: Session = Depends(get_db),
):
dataset = DatasetService.get_dataset(db, dataset_id)
if dataset.project_id != project_id:
raise HTTPException(status_code=404, detail="Dataset not found")
payload = RasterOperationsService.stats(db, dataset_id)
return envelope(RasterStatsResponse(**payload).model_dump())
@router.post("/datasets/{dataset_id}/raster/reproject", status_code=201, response_model=dict)
def raster_reproject_dataset(
project_id: UUID,
dataset_id: UUID,
payload: RasterReprojectRequest,
db: Session = Depends(get_db),
):
dataset = DatasetService.get_dataset(db, dataset_id)
if dataset.project_id != project_id:
raise HTTPException(status_code=404, detail="Dataset not found")
job = _run_job_sync(
db=db,
project_id=project_id,
input_dataset_id=dataset_id,
job_type="raster.reproject",
parameters=payload.model_dump(),
operation=lambda: RasterOperationsService.reproject(
db,
dataset_id,
target_crs=payload.target_crs,
output_name=payload.output_name,
resampling=payload.resampling,
),
)
return envelope(job)
@router.post("/datasets/{dataset_id}/raster/clip", status_code=201, response_model=dict)
def raster_clip_dataset(
project_id: UUID,
dataset_id: UUID,
payload: RasterClipRequest,
db: Session = Depends(get_db),
):
dataset = DatasetService.get_dataset(db, dataset_id)
if dataset.project_id != project_id:
raise HTTPException(status_code=404, detail="Dataset not found")
job = _run_job_sync(
db=db,
project_id=project_id,
input_dataset_id=dataset_id,
job_type="raster.clip",
parameters=payload.model_dump(),
operation=lambda: RasterOperationsService.clip(db, dataset_id, UUID(payload.area_id), payload.output_name),
)
return envelope(job)
@router.post("/datasets/{dataset_id}/raster/tile", status_code=201, response_model=dict)
def raster_tile_dataset(
project_id: UUID,
dataset_id: UUID,
payload: RasterTileRequest,
db: Session = Depends(get_db),
):
dataset = DatasetService.get_dataset(db, dataset_id)
if dataset.project_id != project_id:
raise HTTPException(status_code=404, detail="Dataset not found")
job = _run_job_sync(
db=db,
project_id=project_id,
input_dataset_id=dataset_id,
job_type="raster.tile",
parameters=payload.model_dump(),
operation=lambda: RasterOperationsService.tile(
db,
dataset_id,
tile_size=payload.tile_size,
overlap=payload.overlap,
output_name=payload.output_name,
),
)
return envelope(job)
@router.post("/datasets/{dataset_id}/raster/indices/ndvi", status_code=201, response_model=dict)
def raster_ndvi_dataset(
project_id: UUID,
dataset_id: UUID,
payload: RasterNdviRequest,
db: Session = Depends(get_db),
):
dataset = DatasetService.get_dataset(db, dataset_id)
if dataset.project_id != project_id:
raise HTTPException(status_code=404, detail="Dataset not found")
job = _run_job_sync(
db=db,
project_id=project_id,
input_dataset_id=dataset_id,
job_type="raster.ndvi",
parameters=payload.model_dump(),
operation=lambda: RasterOperationsService.ndvi(
db,
dataset_id=dataset_id,
nir_band=payload.nir_band,
red_band=payload.red_band,
output_name=payload.output_name,
),
)
return envelope(job)
@router.post("/datasets/{dataset_id}/raster/indices/ndwi", status_code=201, response_model=dict)
def raster_ndwi_dataset(
project_id: UUID,
dataset_id: UUID,
payload: RasterNdwiRequest,
db: Session = Depends(get_db),
):
dataset = DatasetService.get_dataset(db, dataset_id)
if dataset.project_id != project_id:
raise HTTPException(status_code=404, detail="Dataset not found")
job = _run_job_sync(
db=db,
project_id=project_id,
input_dataset_id=dataset_id,
job_type="raster.ndwi",
parameters=payload.model_dump(),
operation=lambda: RasterOperationsService.ndwi(
db,
dataset_id=dataset_id,
green_band=payload.green_band,
nir_band=payload.nir_band,
output_name=payload.output_name,
),
)
return envelope(job)
@router.post("/datasets/{dataset_id}/raster/indices/ndbi", status_code=201, response_model=dict)
def raster_ndbi_dataset(
project_id: UUID,
dataset_id: UUID,
payload: RasterNdbiRequest,
db: Session = Depends(get_db),
):
dataset = DatasetService.get_dataset(db, dataset_id)
if dataset.project_id != project_id:
raise HTTPException(status_code=404, detail="Dataset not found")
job = _run_job_sync(
db=db,
project_id=project_id,
input_dataset_id=dataset_id,
job_type="raster.ndbi",
parameters=payload.model_dump(),
operation=lambda: RasterOperationsService.ndbi(
db,
dataset_id=dataset_id,
swir_band=payload.swir_band,
nir_band=payload.nir_band,
output_name=payload.output_name,
),
)
return envelope(job)
@router.get("/datasets/{dataset_id}/raster/metadata", response_model=dict)
def raster_dataset_metadata(
project_id: UUID,
dataset_id: UUID,
db: Session = Depends(get_db),
):
dataset = DatasetService.get_dataset(db, dataset_id)
if dataset.project_id != project_id:
raise HTTPException(status_code=404, detail="Dataset not found")
return envelope(RasterOperationsService.metadata(db, dataset_id))
@router.get("/datasets/{dataset_id}/content", response_model=dict)
def dataset_content(
project_id: UUID,
dataset_id: UUID,
db: Session = Depends(get_db),
):
dataset = DatasetService.get_dataset(db, dataset_id)
if dataset.project_id != project_id:
raise HTTPException(status_code=404, detail="Dataset not found")
return envelope(DatasetService.get_dataset_geojson(db, dataset_id))
backend/app/api/routes/demo.py
+17
View File
@@ -0,0 +1,17 @@
from __future__ import annotations
from fastapi import APIRouter, Depends, status
from sqlalchemy.orm import Session
from app.db.session import get_db
from app.schemas.demo import DemoWorkflowResponse
from app.services.demo_workflow_service import DemoWorkflowService
from app.utils.response import envelope
router = APIRouter(prefix="/demo", tags=["demo"])
@router.post("/workflow", status_code=status.HTTP_201_CREATED, response_model=dict)
def seed_demo_workflow(db: Session = Depends(get_db)) -> dict:
result: DemoWorkflowResponse = DemoWorkflowService.seed(db)
return envelope(result.model_dump())
backend/app/api/routes/detection.py
+145
View File
@@ -0,0 +1,145 @@
from __future__ import annotations
from uuid import UUID
from fastapi import APIRouter, Depends
from sqlalchemy.orm import Session
from app.db.session import get_db
from app.schemas import DetectionQaRequest, DetectionRunRequest
from app.services.detection_service import DetectionService
from app.services.model_registry_service import ModelRegistryService
from app.utils.response import envelope
router = APIRouter(prefix="/detection", tags=["detection"])
@router.get("/models", response_model=dict)
def list_detection_models() -> dict:
return envelope({"models": [model.model_dump() for model in ModelRegistryService.list_model_capabilities()]})
@router.post("/run", response_model=dict)
def run_detection(payload: DetectionRunRequest, db: Session = Depends(get_db)) -> dict:
result = DetectionService.run_detection(
db=db,
project_id=payload.project_id,
dataset_id=payload.dataset_id,
model_id=payload.model_id,
confidence_threshold=payload.confidence_threshold,
class_filter=payload.class_filter,
tile_manifest_path=payload.tile_manifest_path,
parameters_json=payload.parameters_json,
)
return envelope(result.model_dump())
@router.get("/runs", response_model=dict)
def list_detection_runs(
project_id: UUID | None = None,
dataset_id: UUID | None = None,
db: Session = Depends(get_db),
) -> dict:
return envelope(DetectionService.list_runs(db, project_id=project_id, dataset_id=dataset_id).model_dump())
@router.get("/runs/{analysis_run_id}", response_model=dict)
def get_detection_run(analysis_run_id: UUID, db: Session = Depends(get_db)) -> dict:
return envelope(DetectionService.get_run(db, analysis_run_id).model_dump())
@router.get("/runs/{analysis_run_id}/detections", response_model=dict)
def list_detection_run_detections(
analysis_run_id: UUID,
dataset_id: UUID | None = None,
class_name: str | None = None,
min_confidence: float | None = None,
db: Session = Depends(get_db),
) -> dict:
return envelope(
DetectionService.list_detections(
db,
analysis_run_id=analysis_run_id,
dataset_id=dataset_id,
class_name=class_name,
min_confidence=min_confidence,
).model_dump()
)
@router.get("/datasets/{dataset_id}/detections", response_model=dict)
def list_dataset_detections(
dataset_id: UUID,
analysis_run_id: UUID | None = None,
class_name: str | None = None,
min_confidence: float | None = None,
db: Session = Depends(get_db),
) -> dict:
return envelope(
DetectionService.list_detections(
db,
analysis_run_id=analysis_run_id,
dataset_id=dataset_id,
class_name=class_name,
min_confidence=min_confidence,
).model_dump()
)
@router.get("/detections/{detection_id}", response_model=dict)
def get_detection(detection_id: UUID, db: Session = Depends(get_db)) -> dict:
return envelope(DetectionService.get_detection(db, detection_id).model_dump())
@router.get("/runs/{analysis_run_id}/geojson", response_model=dict)
def get_detection_run_geojson(
analysis_run_id: UUID,
class_name: str | None = None,
min_confidence: float | None = None,
db: Session = Depends(get_db),
) -> dict:
return envelope(
DetectionService.detections_to_geojson(
db,
analysis_run_id=analysis_run_id,
class_name=class_name,
min_confidence=min_confidence,
)
)
@router.get("/datasets/{dataset_id}/geojson", response_model=dict)
def get_dataset_detection_geojson(
dataset_id: UUID,
analysis_run_id: UUID | None = None,
class_name: str | None = None,
min_confidence: float | None = None,
db: Session = Depends(get_db),
) -> dict:
return envelope(
DetectionService.detections_to_geojson(
db,
analysis_run_id=analysis_run_id,
dataset_id=dataset_id,
class_name=class_name,
min_confidence=min_confidence,
)
)
@router.post("/runs/{analysis_run_id}/qa/reference", response_model=dict)
def compare_detection_run_with_reference(
analysis_run_id: UUID,
payload: DetectionQaRequest,
db: Session = Depends(get_db),
) -> dict:
return envelope(
DetectionService.compare_detections_with_reference(
db=db,
analysis_run_id=analysis_run_id,
reference_dataset_id=payload.reference_dataset_id,
iou_threshold=payload.iou_threshold,
class_name=payload.class_name,
min_confidence=payload.min_confidence,
)
)
backend/app/api/routes/exports.py
+66
View File
@@ -0,0 +1,66 @@
from __future__ import annotations
from uuid import UUID
from fastapi import APIRouter, Depends, Query
from fastapi.responses import FileResponse
from sqlalchemy.orm import Session
from app.db.session import get_db
from app.schemas.export import GeoJsonExportRequest, MetadataExportRequest, ReportExportRequest
from app.services.export_service import ExportService
from app.utils.response import envelope
router = APIRouter(prefix="/exports", tags=["exports"])
@router.post("/geojson", response_model=dict)
def export_geojson(payload: GeoJsonExportRequest, db: Session = Depends(get_db)):
if payload.export_kind == "detection_run" and payload.analysis_run_id is not None:
return envelope(
ExportService.export_detection_run_geojson(db, payload.analysis_run_id, payload.name).model_dump(mode="json")
)
if payload.export_kind == "segmentation_run" and payload.analysis_run_id is not None:
return envelope(
ExportService.export_segmentation_run_geojson(db, payload.analysis_run_id, payload.name).model_dump(mode="json")
)
if payload.dataset_id is not None:
return envelope(ExportService.export_dataset_geojson(db, payload.dataset_id, payload.name).model_dump(mode="json"))
return envelope({})
@router.post("/metadata", response_model=dict)
def export_project_metadata(payload: MetadataExportRequest, db: Session = Depends(get_db)):
return envelope(ExportService.export_project_metadata(db, payload.project_id, payload.name).model_dump(mode="json"))
@router.post("/report", response_model=dict)
def export_project_report(payload: ReportExportRequest, db: Session = Depends(get_db)):
return envelope(ExportService.export_project_report(db, payload.project_id, payload.name).model_dump(mode="json"))
@router.get("/projects/{project_id}/exports", response_model=dict)
def list_project_exports(
project_id: UUID,
limit: int = Query(default=50, ge=1, le=100),
offset: int = Query(default=0, ge=0),
db: Session = Depends(get_db),
):
return envelope(ExportService.list_project_exports(db, project_id, limit=limit, offset=offset).model_dump(mode="json"))
@router.get("/{export_id}", response_model=dict)
def get_export(export_id: UUID, db: Session = Depends(get_db)):
return envelope(ExportService.get_export(db, export_id).model_dump(mode="json"))
@router.get("/{export_id}/download")
def download_export(export_id: UUID, db: Session = Depends(get_db)):
path = ExportService.get_export_download_path(db, export_id)
media_type = "text/html" if path.suffix.lower() in {".html", ".htm"} else "application/json"
return FileResponse(path, filename=path.name, media_type=media_type)
@router.get("/{export_id}/content", response_model=dict)
def get_export_content(export_id: UUID, db: Session = Depends(get_db)):
return envelope(ExportService.get_export_content(db, export_id).model_dump(mode="json"))
backend/app/api/routes/external.py
+122
View File
@@ -0,0 +1,122 @@
from __future__ import annotations
from fastapi import APIRouter, Depends
from sqlalchemy.orm import Session
from app.core.errors import AppError
from app.db.session import get_db
from app.models import Area, Project
from app.providers.registry import fetch_provider_data, get_provider, import_provider_dataset, list_provider_capabilities
from app.schemas import ExternalFetchRequest, ExternalFetchResponse, ProviderImportRequest
from app.utils.response import envelope
router = APIRouter(prefix="/external", tags=["external"])
def _validate_area_in_project(db: Session, project_id, area_id: str | None) -> None:
if area_id is None:
return
area = db.get(Area, area_id)
if not area:
raise AppError(code="AREA_NOT_FOUND", message="Area not found", status_code=404)
if area.project_id != project_id:
raise AppError(code="INVALID_DATASET_SCOPE", message="Area does not belong to this project", status_code=400)
def _assert_project_exists(db: Session, project_id):
project = db.get(Project, project_id)
if not project:
raise AppError(code="PROJECT_NOT_FOUND", message="Project not found", status_code=404)
def _normalize_layer_input(layers: list[str] | None) -> list[str]:
return [layer.strip() for layer in (layers or []) if isinstance(layer, str) and layer.strip()]
def _provider_payload(provider_name: str) -> dict:
return get_provider(provider_name).capability.to_dict()
@router.get("/providers")
def list_external_providers() -> dict:
return envelope({
"providers": [provider.to_dict() for provider in list_provider_capabilities()],
})
@router.get("/providers/capabilities")
def get_external_provider_capabilities() -> dict:
return envelope({
"providers": [provider.to_dict() for provider in list_provider_capabilities()],
})
@router.get("/providers/{provider_name}")
def get_external_provider(provider_name: str) -> dict:
return envelope(_provider_payload(provider_name))
@router.get("/providers/{provider_name}/layers")
def get_external_provider_layers(provider_name: str) -> dict:
provider = get_provider(provider_name)
return envelope({
"provider_name": provider.provider_name,
"layers": provider.supported_layers,
})
@router.get("/providers/{provider_name}/status")
def get_external_provider_status(provider_name: str) -> dict:
provider = get_provider(provider_name)
return envelope({
"provider_name": provider.provider_name,
"configured": provider.is_configured,
"status": provider.capability.status,
"limitation_message": provider.limitation_message,
})
@router.post("/providers/{provider_name}/import")
def import_external_provider_dataset(provider_name: str, payload: ProviderImportRequest) -> dict:
result = import_provider_dataset(
provider_name=provider_name,
project_id=payload.project_id,
area_id=payload.area_id,
layers=_normalize_layer_input(payload.layers),
requested_dataset_role=payload.dataset_role,
)
return envelope(result.model_dump())
def _run_fetch(payload: ExternalFetchRequest, provider_name: str) -> ExternalFetchResponse:
area_id_str = str(payload.area_id) if payload.area_id else None
response = fetch_provider_data(
provider_name=provider_name,
project_id=str(payload.project_id),
area_id=area_id_str,
layers=_normalize_layer_input(payload.layers),
)
return ExternalFetchResponse(
provider=provider_name,
status=response.get("status", "not_configured"),
message=response.get("message", "Provider fetch executed."),
requested_layers=_normalize_layer_input(payload.layers),
project_id=payload.project_id,
area_id=payload.area_id,
)
@router.post("/osm/fetch")
def fetch_osm(payload: ExternalFetchRequest, db: Session = Depends(get_db)) -> dict:
_assert_project_exists(db, payload.project_id)
_validate_area_in_project(db, payload.project_id, payload.area_id)
return envelope(_run_fetch(payload, "osm").model_dump())
@router.post("/grb/fetch")
def fetch_grb(payload: ExternalFetchRequest, db: Session = Depends(get_db)) -> dict:
_assert_project_exists(db, payload.project_id)
_validate_area_in_project(db, payload.project_id, payload.area_id)
return envelope(_run_fetch(payload, "grb").model_dump())
backend/app/api/routes/health.py
+45
View File
@@ -0,0 +1,45 @@
from __future__ import annotations
from importlib import import_module
from sqlalchemy import text
from fastapi import APIRouter
from app.schemas.health import HealthResponse, SystemCapabilities
from app.providers.registry import list_provider_capabilities
from app.db.session import get_engine
router = APIRouter()
def _dependency_enabled(module: str) -> bool:
try:
import_module(module)
return True
except Exception:
return False
@router.get("/health")
def readiness() -> HealthResponse:
db_status = "ok"
try:
with get_engine().connect() as connection:
connection.execute(text("SELECT 1"))
except Exception:
db_status = "degraded"
return HealthResponse(status="ok", service="geointel-backend", version="0.1.0", database=db_status)
@router.get("/api/v1/system/capabilities")
def capabilities() -> dict:
providers = [item.to_dict() for item in list_provider_capabilities()]
return {"data": SystemCapabilities(
postgis=True,
rasterio=_dependency_enabled("rasterio"),
geopandas=_dependency_enabled("geopandas"),
yolo=False,
sam=False,
grb="planned",
sentinel="planned",
providers=providers,
).model_dump()}
backend/app/api/routes/jobs.py
+67
View File
@@ -0,0 +1,67 @@
from __future__ import annotations
from uuid import UUID
from fastapi import APIRouter, Depends, HTTPException, Query
from sqlalchemy.orm import Session
from app.db.session import get_db
from app.schemas import JobCreate, JobList, JobRead, JobStatus
from app.services.job_service import JobService
from app.utils.response import envelope
router = APIRouter(prefix="/projects/{project_id}", tags=["jobs"])
@router.post("/jobs", status_code=201, response_model=dict)
def create_job(
project_id: UUID,
payload: JobCreate,
db: Session = Depends(get_db),
):
if payload.project_id != project_id:
raise HTTPException(status_code=400, detail="project_id mismatch")
return envelope(JobService.create_job(db, payload).model_dump())
@router.get("/jobs", response_model=dict)
def list_jobs(
project_id: UUID,
dataset_id: UUID | None = Query(default=None),
limit: int = Query(default=50, ge=1, le=200),
offset: int = Query(default=0, ge=0),
db: Session = Depends(get_db),
):
items, total = JobService.list_jobs(
db,
project_id=project_id,
dataset_id=dataset_id,
limit=limit,
offset=offset,
)
return envelope(JobList(items=items, total=total, limit=limit, offset=offset).model_dump())
@router.get("/jobs/{job_id}", response_model=dict)
def read_job(
project_id: UUID,
job_id: UUID,
db: Session = Depends(get_db),
):
job = JobService.get_job(db, job_id)
if job.project_id != project_id:
raise HTTPException(status_code=404, detail="Job not found")
return envelope(job.model_dump())
@router.get("/jobs/{job_id}/status", response_model=dict)
def read_job_status(
project_id: UUID,
job_id: UUID,
db: Session = Depends(get_db),
):
status_row = JobService.get_job_status(db, job_id)
if status_row["project_id"] != str(project_id):
raise HTTPException(status_code=404, detail="Job not found")
return envelope(JobStatus(**status_row).model_dump())
backend/app/api/routes/projects.py
+50
View File
@@ -0,0 +1,50 @@
from __future__ import annotations
from uuid import UUID
from fastapi import APIRouter, Depends, HTTPException, Query, status
from sqlalchemy.orm import Session
from app.db.session import get_db
from app.schemas.project import ProjectCreate, ProjectRead, ProjectUpdate
from app.services.project_service import ProjectService
from app.utils.response import envelope
router = APIRouter(prefix="/projects", tags=["projects"])
@router.get("", response_model=dict)
def list_projects(
limit: int = Query(default=50, ge=1, le=200),
offset: int = Query(default=0, ge=0),
db: Session = Depends(get_db),
):
projects, total = ProjectService.list_projects(db, limit=limit, offset=offset)
return envelope({"items": [ProjectRead.model_validate(item).model_dump() for item in projects], "total": total, "limit": limit, "offset": offset})
@router.post("", status_code=status.HTTP_201_CREATED, response_model=dict)
def create_project(payload: ProjectCreate, db: Session = Depends(get_db)):
project = ProjectService.create_project(db, payload)
return envelope(ProjectRead.model_validate(project).model_dump())
@router.get("/{project_id}", response_model=dict)
def get_project(project_id: UUID, db: Session = Depends(get_db)):
project = ProjectService.get_project(db, project_id)
if not project:
raise HTTPException(status_code=404, detail="Project not found")
return envelope(ProjectRead.model_validate(project).model_dump())
@router.patch("/{project_id}", response_model=dict)
def update_project(project_id: UUID, payload: ProjectUpdate, db: Session = Depends(get_db)):
project = ProjectService.update_project(db, project_id, payload)
return envelope(ProjectRead.model_validate(project).model_dump())
@router.delete("/{project_id}", status_code=status.HTTP_200_OK, response_model=dict)
def delete_project(project_id: UUID, db: Session = Depends(get_db)):
if not ProjectService.delete_project(db, project_id):
raise HTTPException(status_code=404, detail="Project not found")
return envelope({"deleted": True})
backend/app/api/routes/qa.py
+80
View File
@@ -0,0 +1,80 @@
from __future__ import annotations
import uuid
from fastapi import APIRouter, Depends
from sqlalchemy.orm import Session
from app.db.session import get_db
from app.core.errors import AppError
from app.models import Dataset, Job
from app.schemas import QaProviderComparisonRequest
from app.services.qa_service import QaService
from app.services.job_service import JobService
from app.services.quality_service import QualityService
from app.utils.response import envelope
router = APIRouter(prefix="/qa", tags=["qa"])
@router.post("/detections-vs-reference")
def compare_candidate_with_reference(
payload: QaProviderComparisonRequest,
db: Session = Depends(get_db),
) -> dict:
candidate_dataset = db.get(Dataset, payload.candidate_dataset_id)
if not candidate_dataset:
raise AppError(code="DATASET_NOT_FOUND", message="Candidate dataset not found", status_code=404)
job = JobService.run_sync_job(
db=db,
project_id=candidate_dataset.project_id,
job_type="qa.compare-candidate-with-reference",
parameters=payload.model_dump(mode="json"),
input_dataset_id=candidate_dataset.id,
operation=lambda: QaService.compare_candidate_with_reference(
db=db,
project_id=candidate_dataset.project_id,
candidate_dataset_id=payload.candidate_dataset_id,
reference_dataset_id=payload.reference_dataset_id,
iou_threshold=payload.iou_threshold,
area_id=payload.area_id,
).model_dump(mode="json"),
)
result_json = job.get("result_json") if isinstance(job, dict) else None
if isinstance(result_json, dict) and job.get("status") == "success":
quality_check = QualityService.persist_quality_check(
db=db,
project_id=candidate_dataset.project_id,
job_id=uuid.UUID(str(job["id"])),
candidate_dataset_id=payload.candidate_dataset_id,
reference_dataset_id=payload.reference_dataset_id,
check_type="candidate_vs_reference",
status=str(result_json.get("status", "ok")),
score=result_json.get("f1_score"),
parameters=payload.model_dump(mode="json"),
findings={
"matches": result_json.get("matches"),
"false_positives": result_json.get("false_positives"),
"false_negatives": result_json.get("false_negatives"),
"warnings": result_json.get("warnings", []),
"unsupported_geometry": result_json.get("unsupported_geometry", False),
"unsupported_geometries": result_json.get("unsupported_geometries", []),
},
metrics={
"precision": result_json.get("precision"),
"recall": result_json.get("recall"),
"f1": result_json.get("f1_score"),
"mean_iou": result_json.get("mean_iou"),
"false_positive_count": result_json.get("false_positives"),
"false_negative_count": result_json.get("false_negatives"),
},
)
result_json["quality_check_id"] = str(quality_check.id)
job_record = db.get(Job, uuid.UUID(str(job["id"])))
if job_record:
job_record.result_json = result_json
db.add(job_record)
db.commit()
return envelope(job)
backend/app/api/routes/quality_checks.py
+29
View File
@@ -0,0 +1,29 @@
from __future__ import annotations
from uuid import UUID
from fastapi import APIRouter, Depends, Query
from sqlalchemy.orm import Session
from app.db.session import get_db
from app.schemas.qa import QualityCheckList
from app.services.quality_check_service import QualityCheckService
from app.utils.response import envelope
router = APIRouter(prefix="/projects/{project_id}", tags=["quality-checks"])
@router.get("/quality-checks", response_model=dict)
def list_quality_checks(
project_id: UUID,
limit: int = Query(default=50, ge=1, le=200),
offset: int = Query(default=0, ge=0),
db: Session = Depends(get_db),
) -> dict:
items, total = QualityCheckService.list_quality_checks(
db,
project_id=project_id,
limit=limit,
offset=offset,
)
return envelope(QualityCheckList(items=items, total=total, limit=limit, offset=offset).model_dump())
backend/app/api/routes/segmentation.py
+145
View File
@@ -0,0 +1,145 @@
from __future__ import annotations
from uuid import UUID
from fastapi import APIRouter, Depends
from sqlalchemy.orm import Session
from app.db.session import get_db
from app.schemas import SegmentationQaRequest, SegmentationRunRequest
from app.services.model_registry_service import ModelRegistryService
from app.services.segmentation_service import SegmentationService
from app.utils.response import envelope
router = APIRouter(prefix="/segmentation", tags=["segmentation"])
@router.get("/models", response_model=dict)
def list_segmentation_models() -> dict:
return envelope({"models": [model.model_dump() for model in ModelRegistryService.list_model_capabilities(task_type="segmentation")]})
@router.post("/run", response_model=dict)
def run_segmentation(payload: SegmentationRunRequest, db: Session = Depends(get_db)) -> dict:
result = SegmentationService.run_segmentation(
db=db,
project_id=payload.project_id,
dataset_id=payload.dataset_id,
model_id=payload.model_id,
confidence_threshold=payload.confidence_threshold,
class_filter=payload.class_filter,
tile_manifest_path=payload.tile_manifest_path,
parameters_json=payload.parameters_json,
)
return envelope(result.model_dump())
@router.get("/runs", response_model=dict)
def list_segmentation_runs(
project_id: UUID | None = None,
dataset_id: UUID | None = None,
db: Session = Depends(get_db),
) -> dict:
return envelope(SegmentationService.list_runs(db, project_id=project_id, dataset_id=dataset_id).model_dump())
@router.get("/runs/{analysis_run_id}", response_model=dict)
def get_segmentation_run(analysis_run_id: UUID, db: Session = Depends(get_db)) -> dict:
return envelope(SegmentationService.get_run(db, analysis_run_id).model_dump())
@router.get("/runs/{analysis_run_id}/segmentations", response_model=dict)
def list_segmentation_run_outputs(
analysis_run_id: UUID,
dataset_id: UUID | None = None,
class_name: str | None = None,
min_confidence: float | None = None,
db: Session = Depends(get_db),
) -> dict:
return envelope(
SegmentationService.list_segmentations(
db,
analysis_run_id=analysis_run_id,
dataset_id=dataset_id,
class_name=class_name,
min_confidence=min_confidence,
).model_dump()
)
@router.get("/datasets/{dataset_id}/segmentations", response_model=dict)
def list_dataset_segmentations(
dataset_id: UUID,
analysis_run_id: UUID | None = None,
class_name: str | None = None,
min_confidence: float | None = None,
db: Session = Depends(get_db),
) -> dict:
return envelope(
SegmentationService.list_segmentations(
db,
analysis_run_id=analysis_run_id,
dataset_id=dataset_id,
class_name=class_name,
min_confidence=min_confidence,
).model_dump()
)
@router.get("/segmentations/{segmentation_id}", response_model=dict)
def get_segmentation(segmentation_id: UUID, db: Session = Depends(get_db)) -> dict:
return envelope(SegmentationService.get_segmentation(db, segmentation_id).model_dump())
@router.get("/runs/{analysis_run_id}/geojson", response_model=dict)
def get_segmentation_run_geojson(
analysis_run_id: UUID,
class_name: str | None = None,
min_confidence: float | None = None,
db: Session = Depends(get_db),
) -> dict:
return envelope(
SegmentationService.segmentations_to_geojson(
db,
analysis_run_id=analysis_run_id,
class_name=class_name,
min_confidence=min_confidence,
)
)
@router.get("/datasets/{dataset_id}/geojson", response_model=dict)
def get_dataset_segmentation_geojson(
dataset_id: UUID,
analysis_run_id: UUID | None = None,
class_name: str | None = None,
min_confidence: float | None = None,
db: Session = Depends(get_db),
) -> dict:
return envelope(
SegmentationService.segmentations_to_geojson(
db,
analysis_run_id=analysis_run_id,
dataset_id=dataset_id,
class_name=class_name,
min_confidence=min_confidence,
)
)
@router.post("/runs/{analysis_run_id}/qa/reference", response_model=dict)
def compare_segmentation_run_with_reference(
analysis_run_id: UUID,
payload: SegmentationQaRequest,
db: Session = Depends(get_db),
) -> dict:
return envelope(
SegmentationService.compare_segmentations_with_reference(
db=db,
analysis_run_id=analysis_run_id,
reference_dataset_id=payload.reference_dataset_id,
iou_threshold=payload.iou_threshold,
class_name=payload.class_name,
min_confidence=payload.min_confidence,
)
)
backend/app/core/.gitkeep
View File
backend/app/core/config.py
+52
View File
@@ -0,0 +1,52 @@
from pydantic import Field, field_validator
from pydantic_settings import BaseSettings, SettingsConfigDict
class Settings(BaseSettings):
model_config = SettingsConfigDict(
env_file=".env",
env_file_encoding="utf-8",
extra="ignore",
populate_by_name=True,
)
app_env: str = Field(default="development", validation_alias="GEOINTEL_ENV")
app_version: str = Field(default="0.1.0")
api_prefix: str = Field(default="/api/v1", validation_alias="GEOINTEL_API_PREFIX")
database_url: str = Field(
default="postgresql+psycopg://geointel:geointel@localhost:5432/geointel?connect_timeout=1",
validation_alias="DATABASE_URL",
)
storage_root: str = Field(default="./storage", validation_alias="STORAGE_ROOT")
max_upload_mb: int = Field(default=500, validation_alias="MAX_UPLOAD_MB")
redis_url: str | None = Field(default=None, validation_alias="REDIS_URL")
log_level: str = Field(default="INFO", validation_alias="GEOINTEL_LOG_LEVEL")
database_statement_timeout_ms: int = Field(default=5_000, validation_alias="DATABASE_STATEMENT_TIMEOUT_MS")
yolo_enabled: bool = Field(default=False, validation_alias="YOLO_ENABLED")
yolo_model_path: str | None = Field(default=None, validation_alias="YOLO_MODEL_PATH")
yolo_model_id: str = Field(default="yolo-configured", validation_alias="YOLO_MODEL_ID")
yolo_model_display_name: str = Field(default="Configured YOLO detector", validation_alias="YOLO_MODEL_DISPLAY_NAME")
yolo_model_version: str | None = Field(default=None, validation_alias="YOLO_MODEL_VERSION")
yolo_device: str = Field(default="cpu", validation_alias="YOLO_DEVICE")
yolo_image_size: int = Field(default=640, validation_alias="YOLO_IMAGE_SIZE")
yolo_max_tiles: int = Field(default=100, validation_alias="YOLO_MAX_TILES")
yolo_batch_size: int = Field(default=1, validation_alias="YOLO_BATCH_SIZE")
cors_origins: list[str] | str = Field(
default=["http://localhost:5173", "http://127.0.0.1:5173"],
validation_alias="CORS_ORIGINS",
)
@field_validator("cors_origins", mode="before")
@classmethod
def parse_cors_origins(cls, value: object) -> list[str]:
if isinstance(value, str):
return [item.strip() for item in value.split(",") if item.strip()]
if isinstance(value, list):
return value
if value is None:
return ["http://localhost:5173", "http://127.0.0.1:5173"]
return [str(value)]
def get_settings() -> Settings:
return Settings()
backend/app/core/errors.py
+15
View File
@@ -0,0 +1,15 @@
class AppError(Exception):
"""Domain error used by services to return canonical API errors."""
def __init__(
self,
code: str,
message: str,
details: dict | list | None = None,
status_code: int = 400,
) -> None:
super().__init__(message)
self.code = code
self.message = message
self.details = details or {}
self.status_code = status_code
backend/app/core/logging.py
+12
View File
@@ -0,0 +1,12 @@
import logging
import sys
def configure_logging(level: str = "INFO") -> None:
logging.basicConfig(
level=level,
format="%(asctime)s | %(levelname)s | %(name)s | %(message)s",
stream=sys.stdout,
)
for name in ["uvicorn", "uvicorn.error", "uvicorn.access", "sqlalchemy.engine"]:
logging.getLogger(name).setLevel(level)
backend/app/db/.gitkeep
View File
backend/app/db/__init__.py
+4
View File
@@ -0,0 +1,4 @@
from .base import Base
from .session import get_db, get_engine
__all__ = ["Base", "get_db", "get_engine"]
backend/app/db/base.py
+5
View File
@@ -0,0 +1,5 @@
from sqlalchemy.orm import DeclarativeBase
class Base(DeclarativeBase):
pass
backend/app/db/session.py
+20
View File
@@ -0,0 +1,20 @@
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker, Session
from app.core.config import get_settings
engine = create_engine(get_settings().database_url, pool_pre_ping=True, future=True)
SessionLocal = sessionmaker(bind=engine, autocommit=False, autoflush=False, future=True)
def get_db():
db: Session = SessionLocal()
try:
yield db
finally:
db.close()
def get_engine():
return engine
backend/app/geo/.gitkeep
View File
backend/app/main.py
+99
View File
@@ -0,0 +1,99 @@
from __future__ import annotations
from fastapi import FastAPI, HTTPException, Request
from fastapi.exceptions import RequestValidationError
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import JSONResponse
from app.api.routes import areas, datasets, demo, detection, exports, external, health, jobs, projects, qa, quality_checks, segmentation
from app.core.config import get_settings
from app.core.errors import AppError
from app.core.logging import configure_logging
def _to_error_payload(code: str, message: str, details: dict | list | None = None) -> dict:
return {
"error": {
"code": code,
"message": message,
"details": details or {},
},
}
def create_app() -> FastAPI:
settings = get_settings()
configure_logging(settings.log_level)
app = FastAPI(
title="GeoIntel Kempen",
version=settings.app_version,
docs_url="/docs",
redoc_url="/redoc",
)
app.add_middleware(
CORSMiddleware,
allow_origins=settings.cors_origins,
allow_methods=["*"],
allow_headers=["*"],
allow_credentials=True,
)
app.include_router(health.router)
app.include_router(projects.router, prefix=settings.api_prefix)
app.include_router(areas.router, prefix=settings.api_prefix)
app.include_router(datasets.router, prefix=settings.api_prefix)
app.include_router(jobs.router, prefix=settings.api_prefix)
app.include_router(quality_checks.router, prefix=settings.api_prefix)
app.include_router(exports.router, prefix=settings.api_prefix)
app.include_router(external.router, prefix=settings.api_prefix)
app.include_router(demo.router, prefix=settings.api_prefix)
app.include_router(qa.router, prefix=settings.api_prefix)
app.include_router(detection.router, prefix=settings.api_prefix)
app.include_router(segmentation.router, prefix=settings.api_prefix)
@app.exception_handler(AppError)
async def app_error(request: Request, exc: AppError): # noqa: ARG001
return JSONResponse(
status_code=exc.status_code,
content=_to_error_payload(exc.code, exc.message, exc.details),
)
@app.exception_handler(HTTPException)
async def http_error(request: Request, exc: HTTPException): # noqa: ARG001
return JSONResponse(
status_code=exc.status_code,
content=_to_error_payload("HTTP_ERROR", str(exc.detail), {}),
)
@app.exception_handler(RequestValidationError)
async def validation_error(request: Request, exc: RequestValidationError): # noqa: ARG001
return JSONResponse(
status_code=422,
content=_to_error_payload("VALIDATION_ERROR", "Validation failed", exc.errors()),
)
@app.exception_handler(Exception)
async def unexpected_error(request: Request, exc: Exception): # noqa: ARG001
return JSONResponse(
status_code=500,
content=_to_error_payload("INTERNAL_ERROR", "Unexpected server error", {"type": exc.__class__.__name__}),
)
return app
app = create_app()
def main() -> None:
import uvicorn
settings = get_settings()
uvicorn.run(
"app.main:app",
host="0.0.0.0",
port=8000,
reload=settings.app_env == "development",
)
backend/app/models.py
+1
View File
@@ -0,0 +1 @@
from app.models import *
backend/app/models/.gitkeep
View File
backend/app/models/__init__.py
+16
View File
@@ -0,0 +1,16 @@
from .entities import AnalysisRun, Area, Dataset, DatasetVersion, Detection, Export, Job, Metric, Project, QualityCheck, Segmentation, VectorFeature
__all__ = [
"AnalysisRun",
"Area",
"Dataset",
"DatasetVersion",
"Detection",
"Export",
"Job",
"Metric",
"Project",
"QualityCheck",
"Segmentation",
"VectorFeature",
]
backend/app/models/entities.py
+269
View File
@@ -0,0 +1,269 @@
from __future__ import annotations
import uuid
from datetime import datetime
from geoalchemy2 import Geometry
from sqlalchemy import DateTime, ForeignKey, Float, Index, JSON, String, Text, func
from sqlalchemy.sql.sqltypes import Integer
from sqlalchemy.dialects.postgresql import UUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.db.base import Base
class Project(Base):
__tablename__ = "projects"
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
name: Mapped[str] = mapped_column(String(255), nullable=False)
description: Mapped[str | None] = mapped_column(Text, nullable=True)
region: Mapped[str] = mapped_column(String(120), default="Kempen")
status: Mapped[str] = mapped_column(String(32), default="active")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
areas: Mapped[list["Area"]] = relationship("Area", back_populates="project", cascade="all, delete-orphan")
datasets: Mapped[list["Dataset"]] = relationship("Dataset", back_populates="project", cascade="all, delete-orphan")
class Area(Base):
__tablename__ = "areas"
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
project_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("projects.id", ondelete="CASCADE"), nullable=False)
name: Mapped[str] = mapped_column(String(255), nullable=False)
geometry: Mapped[str] = mapped_column(Geometry("MultiPolygon", srid=4326), nullable=False)
original_crs: Mapped[str | None] = mapped_column(String(64), nullable=True)
area_m2: Mapped[float | None] = mapped_column(Float, nullable=True)
bbox: Mapped[str | None] = mapped_column(Geometry("Polygon", srid=4326), nullable=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
project: Mapped[Project] = relationship("Project", back_populates="areas")
class Dataset(Base):
__tablename__ = "datasets"
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
project_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("projects.id", ondelete="CASCADE"), nullable=False)
area_id: Mapped[uuid.UUID | None] = mapped_column(UUID(as_uuid=True), ForeignKey("areas.id", ondelete="SET NULL"), nullable=True)
name: Mapped[str] = mapped_column(String(255), nullable=False)
dataset_type: Mapped[str] = mapped_column(String(64), nullable=False)
source: Mapped[str] = mapped_column(String(120), nullable=False)
storage_path: Mapped[str | None] = mapped_column(String(500), nullable=True)
original_filename: Mapped[str | None] = mapped_column(String(255), nullable=True)
stored_filename: Mapped[str | None] = mapped_column(String(255), nullable=True)
content_type: Mapped[str | None] = mapped_column(String(120), nullable=True)
size_bytes: Mapped[int | None] = mapped_column(Integer, nullable=True)
checksum_sha256: Mapped[str | None] = mapped_column(String(64), nullable=True)
derived_from_dataset_id: Mapped[uuid.UUID | None] = mapped_column(
UUID(as_uuid=True),
ForeignKey("datasets.id", ondelete="SET NULL"),
nullable=True,
)
crs: Mapped[str | None] = mapped_column(String(64), nullable=True)
bounds_json: Mapped[dict | None] = mapped_column(JSON, nullable=True)
resolution_json: Mapped[dict | None] = mapped_column(JSON, nullable=True)
bands_json: Mapped[dict | None] = mapped_column(JSON, nullable=True)
metadata_json: Mapped[dict | None] = mapped_column(JSON, nullable=True)
dataset_role: Mapped[str] = mapped_column(String(32), nullable=False, default="source", server_default="source")
source_name: Mapped[str | None] = mapped_column(String(120), nullable=True)
reference_layer_name: Mapped[str | None] = mapped_column(String(120), nullable=True)
source_metadata: Mapped[dict | None] = mapped_column(JSON, nullable=True)
provenance_metadata: Mapped[dict | None] = mapped_column(JSON, nullable=True)
imported_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
status: Mapped[str] = mapped_column(String(32), default="uploaded")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
project: Mapped[Project] = relationship("Project", back_populates="datasets")
versions: Mapped[list["DatasetVersion"]] = relationship(
"DatasetVersion",
back_populates="dataset",
cascade="all, delete-orphan",
)
vector_features: Mapped[list["VectorFeature"]] = relationship(
"VectorFeature",
back_populates="dataset",
cascade="all, delete-orphan",
)
class DatasetVersion(Base):
__tablename__ = "dataset_versions"
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
dataset_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("datasets.id", ondelete="CASCADE"), nullable=False)
version: Mapped[int] = mapped_column(Integer, default=1)
storage_path: Mapped[str | None] = mapped_column(String(500), nullable=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
dataset: Mapped[Dataset] = relationship("Dataset", back_populates="versions")
class VectorFeature(Base):
__tablename__ = "vector_features"
__table_args__ = (
Index("ix_vector_features_dataset_id", "dataset_id"),
Index("ix_vector_features_geometry", "geometry", postgresql_using="gist"),
)
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
dataset_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("datasets.id", ondelete="CASCADE"), nullable=False)
feature_class: Mapped[str | None] = mapped_column(String(120), nullable=True)
source_feature_id: Mapped[str | None] = mapped_column(String(255), nullable=True)
properties_json: Mapped[dict | None] = mapped_column(JSON, nullable=True)
geometry: Mapped[str] = mapped_column(Geometry("Geometry", srid=4326, spatial_index=False), nullable=False)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
dataset: Mapped[Dataset] = relationship("Dataset", back_populates="vector_features")
class AnalysisRun(Base):
__tablename__ = "analysis_runs"
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
project_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("projects.id", ondelete="CASCADE"), nullable=False)
area_id: Mapped[uuid.UUID | None] = mapped_column(UUID(as_uuid=True), ForeignKey("areas.id", ondelete="SET NULL"), nullable=True)
dataset_id: Mapped[uuid.UUID | None] = mapped_column(UUID(as_uuid=True), ForeignKey("datasets.id", ondelete="SET NULL"), nullable=True)
job_id: Mapped[uuid.UUID | None] = mapped_column(UUID(as_uuid=True), ForeignKey("jobs.id", ondelete="SET NULL"), nullable=True)
analysis_type: Mapped[str] = mapped_column(String(64), nullable=False)
status: Mapped[str] = mapped_column(String(32), nullable=False)
model_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
model_version: Mapped[str | None] = mapped_column(String(120), nullable=True)
parameters_json: Mapped[dict] = mapped_column(JSON, nullable=False)
result_json: Mapped[dict | None] = mapped_column(JSON, nullable=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
error_message: Mapped[str | None] = mapped_column(Text, nullable=True)
class Detection(Base):
__tablename__ = "detections"
__table_args__ = (
Index("ix_detections_project_id", "project_id"),
Index("ix_detections_dataset_id", "dataset_id"),
Index("ix_detections_analysis_run_id", "analysis_run_id"),
Index("ix_detections_class_name", "class_name"),
Index("ix_detections_geometry", "geometry", postgresql_using="gist"),
)
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
project_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("projects.id", ondelete="CASCADE"), nullable=False)
dataset_id: Mapped[uuid.UUID | None] = mapped_column(UUID(as_uuid=True), ForeignKey("datasets.id", ondelete="SET NULL"), nullable=True)
analysis_run_id: Mapped[uuid.UUID | None] = mapped_column(UUID(as_uuid=True), ForeignKey("analysis_runs.id", ondelete="SET NULL"), nullable=True)
job_id: Mapped[uuid.UUID | None] = mapped_column(UUID(as_uuid=True), ForeignKey("jobs.id", ondelete="SET NULL"), nullable=True)
model_name: Mapped[str] = mapped_column(String(255), nullable=False)
model_version: Mapped[str | None] = mapped_column(String(120), nullable=True)
class_name: Mapped[str] = mapped_column(String(120), nullable=False)
confidence: Mapped[float] = mapped_column(Float, nullable=False)
geometry: Mapped[str] = mapped_column(Geometry("Geometry", srid=4326, spatial_index=False), nullable=False)
bbox_json: Mapped[dict | None] = mapped_column(JSON, nullable=True)
source_tile_path: Mapped[str | None] = mapped_column(String(500), nullable=True)
properties_json: Mapped[dict | None] = mapped_column(JSON, nullable=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
class Segmentation(Base):
__tablename__ = "segmentations"
__table_args__ = (
Index("ix_segmentations_project_id", "project_id"),
Index("ix_segmentations_dataset_id", "dataset_id"),
Index("ix_segmentations_analysis_run_id", "analysis_run_id"),
Index("ix_segmentations_job_id", "job_id"),
Index("ix_segmentations_class_name", "class_name"),
Index("ix_segmentations_geometry", "geometry", postgresql_using="gist"),
)
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
project_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("projects.id", ondelete="CASCADE"), nullable=False)
dataset_id: Mapped[uuid.UUID | None] = mapped_column(UUID(as_uuid=True), ForeignKey("datasets.id", ondelete="SET NULL"), nullable=True)
job_id: Mapped[uuid.UUID | None] = mapped_column(UUID(as_uuid=True), ForeignKey("jobs.id", ondelete="SET NULL"), nullable=True)
analysis_run_id: Mapped[uuid.UUID | None] = mapped_column(UUID(as_uuid=True), ForeignKey("analysis_runs.id", ondelete="SET NULL"), nullable=True)
model_name: Mapped[str] = mapped_column(String(255), nullable=False)
model_version: Mapped[str | None] = mapped_column(String(120), nullable=True)
class_name: Mapped[str] = mapped_column(String(120), nullable=False)
confidence: Mapped[float | None] = mapped_column(Float, nullable=True)
geometry: Mapped[str] = mapped_column(Geometry("MultiPolygon", srid=4326, spatial_index=False), nullable=False)
bbox_json: Mapped[dict | None] = mapped_column(JSON, nullable=True)
area_m2: Mapped[float | None] = mapped_column(Float, nullable=True)
mask_path: Mapped[str | None] = mapped_column(Text, nullable=True)
source_tile_path: Mapped[str | None] = mapped_column(String(500), nullable=True)
tile_index: Mapped[int | None] = mapped_column(Integer, nullable=True)
properties_json: Mapped[dict | None] = mapped_column(JSON, nullable=True)
provenance_json: Mapped[dict | None] = mapped_column(JSON, nullable=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
class QualityCheck(Base):
__tablename__ = "quality_checks"
__table_args__ = (
Index("ix_quality_checks_project_id", "project_id"),
Index("ix_quality_checks_reference_dataset_id", "reference_dataset_id"),
Index("ix_quality_checks_candidate_dataset_id", "candidate_dataset_id"),
Index("ix_quality_checks_analysis_run_id", "analysis_run_id"),
)
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
project_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("projects.id", ondelete="CASCADE"), nullable=False)
job_id: Mapped[uuid.UUID | None] = mapped_column(UUID(as_uuid=True), ForeignKey("jobs.id", ondelete="SET NULL"), nullable=True)
analysis_run_id: Mapped[uuid.UUID | None] = mapped_column(UUID(as_uuid=True), ForeignKey("analysis_runs.id", ondelete="SET NULL"), nullable=True)
candidate_dataset_id: Mapped[uuid.UUID | None] = mapped_column(UUID(as_uuid=True), ForeignKey("datasets.id", ondelete="SET NULL"), nullable=True)
reference_dataset_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("datasets.id", ondelete="CASCADE"), nullable=False)
check_type: Mapped[str] = mapped_column(String(120), nullable=False)
status: Mapped[str] = mapped_column(String(32), nullable=False)
score: Mapped[float | None] = mapped_column(Float, nullable=True)
parameters_json: Mapped[dict | None] = mapped_column(JSON, nullable=True)
findings_json: Mapped[dict | None] = mapped_column(JSON, nullable=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
completed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
class Metric(Base):
__tablename__ = "metrics"
__table_args__ = (
Index("ix_metrics_quality_check_id", "quality_check_id"),
Index("ix_metrics_analysis_run_id", "analysis_run_id"),
)
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
quality_check_id: Mapped[uuid.UUID | None] = mapped_column(UUID(as_uuid=True), ForeignKey("quality_checks.id", ondelete="CASCADE"), nullable=True)
analysis_run_id: Mapped[uuid.UUID | None] = mapped_column(UUID(as_uuid=True), ForeignKey("analysis_runs.id", ondelete="SET NULL"), nullable=True)
metric_key: Mapped[str] = mapped_column(String(120), nullable=False)
metric_value: Mapped[float | None] = mapped_column(Float, nullable=True)
metric_unit: Mapped[str | None] = mapped_column(String(64), nullable=True)
label: Mapped[str | None] = mapped_column(String(120), nullable=True)
metadata_json: Mapped[dict | None] = mapped_column(JSON, nullable=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
class Export(Base):
__tablename__ = "exports"
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
project_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("projects.id", ondelete="CASCADE"), nullable=False)
analysis_run_id: Mapped[uuid.UUID | None] = mapped_column(UUID(as_uuid=True), ForeignKey("analysis_runs.id", ondelete="SET NULL"), nullable=True)
export_type: Mapped[str] = mapped_column(String(64), nullable=False)
storage_path: Mapped[str] = mapped_column(String(500), nullable=False)
metadata_json: Mapped[dict | None] = mapped_column(JSON, nullable=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
class Job(Base):
__tablename__ = "jobs"
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
job_type: Mapped[str] = mapped_column(String(128), nullable=False)
status: Mapped[str] = mapped_column(String(32), nullable=False, default="queued")
project_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), ForeignKey("projects.id", ondelete="CASCADE"), nullable=False)
dataset_id: Mapped[uuid.UUID | None] = mapped_column(UUID(as_uuid=True), ForeignKey("datasets.id", ondelete="SET NULL"), nullable=True)
input_dataset_id: Mapped[uuid.UUID | None] = mapped_column(UUID(as_uuid=True), ForeignKey("datasets.id", ondelete="SET NULL"), nullable=True)
output_dataset_id: Mapped[uuid.UUID | None] = mapped_column(UUID(as_uuid=True), ForeignKey("datasets.id", ondelete="SET NULL"), nullable=True)
parameters_json: Mapped[dict] = mapped_column(JSON, nullable=False, default=dict)
result_json: Mapped[dict | None] = mapped_column(JSON, nullable=True)
error_message: Mapped[str | None] = mapped_column(Text, nullable=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
backend/app/providers/.gitkeep
View File
backend/app/providers/__init__.py
+5
View File
@@ -0,0 +1,5 @@
from __future__ import annotations
from app.providers import base, fixture, grb, manual, osm, registry
__all__ = ["base", "fixture", "grb", "manual", "osm", "registry"]
backend/app/providers/base.py
+96
View File
@@ -0,0 +1,96 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import Any
@dataclass(frozen=True)
class ProviderCapability:
provider_name: str
display_name: str
authority_level: str
supported_layers: list[str]
supported_geometry_types: list[str]
supported_query_modes: list[str]
fetch_signature: str
configured: bool
status: str
limitation_message: str
attribution: str
license_note: str
not_configured_reason: str | None = None
def to_dict(self) -> dict[str, Any]:
return {
"provider_name": self.provider_name,
"display_name": self.display_name,
"authority_level": self.authority_level,
"supported_layers": self.supported_layers,
"supported_geometry_types": self.supported_geometry_types,
"supported_query_modes": self.supported_query_modes,
"fetch_signature": self.fetch_signature,
"configured": self.configured,
"status": self.status,
"limitation_message": self.limitation_message,
"attribution": self.attribution,
"license_note": self.license_note,
"not_configured_reason": self.not_configured_reason,
}
class BaseReferenceProvider:
def __init__(
self,
provider_name: str,
display_name: str,
authority_level: str,
supported_layers: list[str],
supported_geometry_types: list[str],
supported_query_modes: list[str],
fetch_signature: str,
limitation_message: str,
attribution: str,
license_note: str,
configured: bool = False,
) -> None:
self.provider_name = provider_name
self.display_name = display_name
self.authority_level = authority_level
self.supported_layers = supported_layers
self.supported_geometry_types = supported_geometry_types
self.supported_query_modes = supported_query_modes
self.fetch_signature = fetch_signature
self.limitation_message = limitation_message
self.attribution = attribution
self.license_note = license_note
self._configured = configured
@property
def capability(self) -> ProviderCapability:
return ProviderCapability(
provider_name=self.provider_name,
display_name=self.display_name,
authority_level=self.authority_level,
supported_layers=self.supported_layers,
supported_geometry_types=self.supported_geometry_types,
supported_query_modes=self.supported_query_modes,
fetch_signature=self.fetch_signature,
configured=self.is_configured,
status="configured" if self.is_configured else "not_configured",
limitation_message=self.limitation_message,
attribution=self.attribution,
license_note=self.license_note,
not_configured_reason=None if self.is_configured else "Provider integration is not configured yet",
)
@property
def is_configured(self) -> bool:
return self._configured
def fetch(self, project_id: str, area_id: str | None, layers: list[str]) -> dict[str, Any]:
del project_id, area_id, layers
return {
"provider": self.provider_name,
"status": "not_configured",
"message": "Provider integration is not configured yet",
}
backend/app/providers/fixture.py
+20
View File
@@ -0,0 +1,20 @@
from __future__ import annotations
from app.providers.base import BaseReferenceProvider
class FixtureProvider(BaseReferenceProvider):
def __init__(self) -> None:
super().__init__(
provider_name="fixture",
display_name="Fixture data",
authority_level="fixture",
supported_layers=["buildings", "roads", "water", "landuse", "custom"],
supported_geometry_types=["Point", "MultiPoint", "LineString", "MultiLineString", "Polygon", "MultiPolygon"],
supported_query_modes=["fixture"],
fetch_signature="tests/fixtures and demo fixture upload flow",
limitation_message="Fixture provider represents local test/demo fixtures only.",
attribution="GeoIntel local fixtures",
license_note="Fixtures are for local development and tests; do not present them as official data.",
configured=True,
)
backend/app/providers/grb.py
+20
View File
@@ -0,0 +1,20 @@
from __future__ import annotations
from app.providers.base import BaseReferenceProvider
class GRBProvider(BaseReferenceProvider):
def __init__(self) -> None:
super().__init__(
provider_name="grb",
display_name="GRB",
authority_level="authoritative",
supported_layers=["buildings", "roads", "parcels"],
supported_geometry_types=["Polygon", "MultiPolygon", "LineString", "MultiLineString"],
supported_query_modes=["area"],
fetch_signature="POST /api/v1/external/grb/fetch",
limitation_message="GRB live WFS/download integration is not configured in Sprint 7B.",
attribution="Digitaal Vlaanderen - Basiskaart Vlaanderen (GRB)",
license_note="Use must follow Digitaal Vlaanderen open data and attribution terms.",
configured=False,
)
backend/app/providers/manual.py
+20
View File
@@ -0,0 +1,20 @@
from __future__ import annotations
from app.providers.base import BaseReferenceProvider
class ManualProvider(BaseReferenceProvider):
def __init__(self) -> None:
super().__init__(
provider_name="manual",
display_name="Manual upload",
authority_level="manual",
supported_layers=["buildings", "roads", "water", "landuse", "custom"],
supported_geometry_types=["Point", "MultiPoint", "LineString", "MultiLineString", "Polygon", "MultiPolygon"],
supported_query_modes=["upload"],
fetch_signature="POST /api/v1/projects/{project_id}/datasets/upload",
limitation_message="Manual provider data is supplied through the existing dataset upload flow.",
attribution="User supplied",
license_note="License and attribution must be supplied by the uploader in source metadata.",
configured=True,
)
backend/app/providers/osm.py
+20
View File
@@ -0,0 +1,20 @@
from __future__ import annotations
from app.providers.base import BaseReferenceProvider
class OSMProvider(BaseReferenceProvider):
def __init__(self) -> None:
super().__init__(
provider_name="osm",
display_name="OpenStreetMap",
authority_level="contextual",
supported_layers=["buildings", "roads", "water", "landuse"],
supported_geometry_types=["Polygon", "MultiPolygon", "LineString", "MultiLineString"],
supported_query_modes=["area"],
fetch_signature="POST /api/v1/external/osm/fetch",
limitation_message="OSM live Overpass/download integration is not configured in Sprint 7B.",
attribution="OpenStreetMap contributors",
license_note="OpenStreetMap data is available under ODbL; attribution is required.",
configured=False,
)
backend/app/providers/registry.py
+141
View File
@@ -0,0 +1,141 @@
from __future__ import annotations
from pydantic import BaseModel
from app.core.errors import AppError
from app.providers.base import ProviderCapability
from app.providers.fixture import FixtureProvider
from app.providers.grb import GRBProvider
from app.providers.manual import ManualProvider
from app.providers.osm import OSMProvider
class ProviderDatasetMapping(BaseModel):
provider_name: str
dataset_role: str
source_name: str
reference_required: bool
write_path: str = "DatasetService"
class ProviderImportResult(BaseModel):
provider_name: str
status: str
message: str
requested_layers: list[str]
dataset_id: str | None = None
dataset_role: str | None = None
source_name: str | None = None
class ExternalProviderRegistry:
def __init__(self) -> None:
self.providers = {
"grb": GRBProvider(),
"osm": OSMProvider(),
"manual": ManualProvider(),
"fixture": FixtureProvider(),
}
def list_capabilities(self) -> list[ProviderCapability]:
return [provider.capability for provider in self.providers.values()]
def get(self, provider_name: str):
normalized = provider_name.strip().lower()
if normalized not in self.providers:
raise AppError(code="PROVIDER_NOT_FOUND", message="Provider not found", status_code=404)
return self.providers[normalized]
def fetch(self, provider_name: str, project_id: str, area_id: str | None, layers: list[str]) -> dict:
provider = self.get(provider_name)
return provider.fetch(project_id=project_id, area_id=area_id, layers=layers)
def dataset_mapping(self, provider_name: str, requested_dataset_role: str | None = None) -> ProviderDatasetMapping:
provider = self.get(provider_name)
if provider.provider_name == "osm":
dataset_role = "reference" if requested_dataset_role == "reference" else "source"
return ProviderDatasetMapping(
provider_name="osm",
dataset_role=dataset_role,
source_name="osm",
reference_required=requested_dataset_role == "reference",
)
return ProviderDatasetMapping(
provider_name=provider.provider_name,
dataset_role="reference",
source_name=provider.provider_name,
reference_required=True,
)
def import_contract(
self,
provider_name: str,
project_id: str,
area_id: str | None,
layers: list[str],
requested_dataset_role: str | None = None,
) -> ProviderImportResult:
del project_id, area_id
provider = self.get(provider_name)
mapping = self.dataset_mapping(provider.provider_name, requested_dataset_role=requested_dataset_role)
if provider.provider_name in {"grb", "osm"}:
return ProviderImportResult(
provider_name=provider.provider_name,
status="not_configured",
message=f"No live {provider.display_name} import is configured in Sprint 7B.",
requested_layers=layers,
dataset_role=mapping.dataset_role,
source_name=mapping.source_name,
)
if provider.provider_name == "manual":
return ProviderImportResult(
provider_name="manual",
status="upload_flow_required",
message="Manual provider data must use the existing dataset upload/reference flow.",
requested_layers=layers,
dataset_role=mapping.dataset_role,
source_name=mapping.source_name,
)
return ProviderImportResult(
provider_name="fixture",
status="fixture_flow_required",
message="Fixture provider data must use checked-in demo/test fixture flows.",
requested_layers=layers,
dataset_role=mapping.dataset_role,
source_name=mapping.source_name,
)
_registry = ExternalProviderRegistry()
def list_provider_capabilities() -> list[ProviderCapability]:
return _registry.list_capabilities()
def get_provider(provider_name: str):
return _registry.get(provider_name)
def fetch_provider_data(provider_name: str, project_id: str, area_id: str | None, layers: list[str]) -> dict:
return _registry.fetch(provider_name, project_id, area_id, layers)
def get_provider_dataset_mapping(provider_name: str, requested_dataset_role: str | None = None) -> ProviderDatasetMapping:
return _registry.dataset_mapping(provider_name, requested_dataset_role=requested_dataset_role)
def import_provider_dataset(
provider_name: str,
project_id: str,
area_id: str | None,
layers: list[str],
requested_dataset_role: str | None = None,
) -> ProviderImportResult:
return _registry.import_contract(
provider_name=provider_name,
project_id=project_id,
area_id=area_id,
layers=layers,
requested_dataset_role=requested_dataset_role,
)
backend/app/repositories/.gitkeep
View File
backend/app/schemas/.gitkeep
View File
backend/app/schemas/__init__.py
+156
View File
@@ -0,0 +1,156 @@
from __future__ import annotations
from .common import ApiErrorEnvelope, ApiErrorItem, Envelope, PaginationEnvelope
from .project import ProjectCreate, ProjectList, ProjectRead, ProjectUpdate
from .area import AreaCreate, AreaList, AreaRead, AreaUpdate
from .dataset import DatasetCreateResponse, DatasetList
from .detection import (
DetectionListResponse,
DetectionModelCapability,
DetectionModelsResponse,
DetectionQaRequest,
DetectionRead,
DetectionRunListResponse,
DetectionRunRead,
DetectionRunRequest,
DetectionRunResponse,
)
from .segmentation import (
SegmentationListResponse,
SegmentationModelCapability,
SegmentationModelsResponse,
SegmentationQaRequest,
SegmentationRead,
SegmentationRunListResponse,
SegmentationRunRead,
SegmentationRunRequest,
SegmentationRunResponse,
)
from .health import HealthResponse, SystemCapabilities
from .job import JobCreate, JobList, JobRead, JobStatus
from .external import (
ExternalFetchRequest,
ExternalFetchResponse,
ProviderCapabilitiesResponse,
ProviderCapabilityResponse,
ProviderImportRequest,
ProviderImportResponse,
ProviderLayersResponse,
ProviderStatusResponse,
)
from .export import (
ExportContentResponse,
ExportCreateResponse,
ExportListResponse,
ExportRead,
GeoJsonExportRequest,
MetadataExportRequest,
ReportExportRequest,
)
from .qa import QaProviderComparisonRequest, QaProviderComparisonResult
from .operations import (
RasterClipRequest,
RasterIndexBaseRequest,
RasterMetadataResponse,
RasterNdviRequest,
RasterNdwiRequest,
RasterNdbiRequest,
RasterOperationResult,
RasterPreviewResponse,
RasterReprojectRequest,
RasterReprojectResponse,
RasterStatsResponse,
RasterTileManifest,
RasterTileManifestTile,
RasterTileRequest,
RasterTileResponse,
VectorBBoxResponse,
VectorBufferRequest,
VectorClipRequest,
VectorIntersectRequest,
VectorOperationRequest,
VectorOperationResult,
VectorStatsRequest,
VectorStatsResponse,
)
__all__ = [
"Envelope",
"ApiErrorEnvelope",
"ApiErrorItem",
"PaginationEnvelope",
"ProjectCreate",
"ProjectRead",
"ProjectUpdate",
"ProjectList",
"AreaCreate",
"AreaRead",
"AreaUpdate",
"AreaList",
"DatasetCreateResponse",
"DatasetList",
"DetectionListResponse",
"DetectionModelCapability",
"DetectionModelsResponse",
"DetectionQaRequest",
"DetectionRead",
"DetectionRunListResponse",
"DetectionRunRead",
"DetectionRunRequest",
"DetectionRunResponse",
"SegmentationListResponse",
"SegmentationModelCapability",
"SegmentationModelsResponse",
"SegmentationQaRequest",
"SegmentationRead",
"SegmentationRunListResponse",
"SegmentationRunRead",
"SegmentationRunRequest",
"SegmentationRunResponse",
"HealthResponse",
"SystemCapabilities",
"JobCreate",
"JobList",
"JobRead",
"JobStatus",
"VectorBBoxResponse",
"VectorClipRequest",
"VectorBufferRequest",
"VectorIntersectRequest",
"VectorOperationRequest",
"VectorOperationResult",
"RasterClipRequest",
"RasterStatsResponse",
"RasterReprojectRequest",
"RasterReprojectResponse",
"RasterTileRequest",
"RasterMetadataResponse",
"RasterOperationResult",
"RasterPreviewResponse",
"RasterTileManifestTile",
"RasterTileManifest",
"RasterTileResponse",
"RasterIndexBaseRequest",
"RasterNdviRequest",
"RasterNdwiRequest",
"RasterNdbiRequest",
"VectorStatsRequest",
"VectorStatsResponse",
"ExternalFetchRequest",
"ExternalFetchResponse",
"ProviderCapabilitiesResponse",
"ProviderCapabilityResponse",
"ProviderImportRequest",
"ProviderImportResponse",
"ProviderLayersResponse",
"ProviderStatusResponse",
"GeoJsonExportRequest",
"MetadataExportRequest",
"ReportExportRequest",
"ExportRead",
"ExportCreateResponse",
"ExportListResponse",
"ExportContentResponse",
"QaProviderComparisonRequest",
"QaProviderComparisonResult",
]
backend/app/schemas/area.py
+40
View File
@@ -0,0 +1,40 @@
from __future__ import annotations
from datetime import datetime
from uuid import UUID
from pydantic import BaseModel
class AreaCreate(BaseModel):
name: str
geometry: dict
crs: str | None = "EPSG:4326"
class AreaUpdate(BaseModel):
name: str | None = None
crs: str | None = None
class AreaRead(BaseModel):
id: UUID
project_id: UUID
name: str
original_crs: str | None
area_m2: float | None
created_at: datetime | None = None
geometry_type: str | None = None
model_config = {"from_attributes": True}
class AreaListItem(AreaRead):
pass
class AreaList(BaseModel):
items: list[AreaRead]
total: int
limit: int
offset: int
backend/app/schemas/common.py
+31
View File
@@ -0,0 +1,31 @@
from __future__ import annotations
from pydantic import BaseModel, Field
class Envelope(BaseModel):
data: object
class PaginatedEnvelope(BaseModel):
items: list
total: int
limit: int
offset: int
class PaginationEnvelope(BaseModel):
items: list
total: int
limit: int = Field(default=50)
offset: int = Field(default=0)
class ApiErrorItem(BaseModel):
code: str
message: str
details: dict = Field(default_factory=dict)
class ApiErrorEnvelope(BaseModel):
error: ApiErrorItem
backend/app/schemas/dataset.py
+81
View File
@@ -0,0 +1,81 @@
from __future__ import annotations
from datetime import datetime
from uuid import UUID
from pydantic import BaseModel
class DatasetStorageResponse(BaseModel):
original_filename: str | None = None
stored_filename: str | None = None
content_type: str | None = None
size_bytes: int | None = None
checksum_sha256: str | None = None
class DatasetVectorSummary(BaseModel):
feature_count: int | None = None
geometry_types: list[str] | None = None
bounds_json: dict | None = None
approximate_area_m2: float | None = None
crs: str | None = None
feature_geometry_count: int | None = None
invalid_features: int | None = None
crs_assumed: bool | None = None
class DatasetCreateResponse(BaseModel):
id: UUID
name: str
dataset_type: str
source: str
dataset_role: str = "source"
source_name: str | None = None
reference_layer_name: str | None = None
source_metadata: dict | None = None
provenance_metadata: dict | None = None
imported_at: datetime | None = None
project_id: UUID
area_id: UUID | None = None
storage_path: str | None = None
original_filename: str | None = None
stored_filename: str | None = None
content_type: str | None = None
size_bytes: int | None = None
checksum_sha256: str | None = None
crs: str | None = None
bounds_json: dict | None = None
metadata_json: dict | None = None
vector_summary: DatasetVectorSummary | None = None
status: str
derived_from_dataset_id: UUID | None = None
created_at: datetime | None = None
feature_count: int | None = None
model_config = {"from_attributes": True}
class DatasetList(BaseModel):
items: list[DatasetCreateResponse]
total: int
limit: int
offset: int
class DatasetMetadataRefresh(BaseModel):
feature_count: int | None = None
geometry_types: list[str] | None = None
bounds_json: dict | None = None
crs: str | None = None
class ExportRequest(BaseModel):
dataset_id: UUID
name: str | None = None
class ExportRead(BaseModel):
export_id: UUID
path: str
status: str
backend/app/schemas/demo.py
+17
View File
@@ -0,0 +1,17 @@
from __future__ import annotations
from uuid import UUID
from pydantic import BaseModel
class DemoWorkflowResponse(BaseModel):
project_id: UUID
area_id: UUID
reference_dataset_id: UUID
candidate_dataset_id: UUID
quality_check_id: UUID
metric_count: int
status: str
message: str
created: bool
backend/app/schemas/detection.py
+98
View File
@@ -0,0 +1,98 @@
from __future__ import annotations
from datetime import datetime
from uuid import UUID
from pydantic import BaseModel, Field
class DetectionModelCapability(BaseModel):
model_id: str
display_name: str
framework: str
task_type: str
supported_classes: list[str]
configured: bool
status: str
limitation_message: str
version: str | None = None
class DetectionModelsResponse(BaseModel):
models: list[DetectionModelCapability]
class DetectionRunRequest(BaseModel):
project_id: UUID
dataset_id: UUID
model_id: str
confidence_threshold: float = Field(default=0.5, ge=0.0, le=1.0)
class_filter: list[str] | None = None
tile_manifest_path: str | None = None
parameters_json: dict = Field(default_factory=dict)
class DetectionQaRequest(BaseModel):
reference_dataset_id: UUID
iou_threshold: float = Field(default=0.5, ge=0.0, le=1.0)
class_name: str | None = None
min_confidence: float | None = Field(default=None, ge=0.0, le=1.0)
class DetectionRunResponse(BaseModel):
analysis_run_id: UUID
job_id: UUID
project_id: UUID
dataset_id: UUID
model_id: str
status: str
detection_count: int
error_code: str | None = None
message: str
class DetectionRunRead(BaseModel):
id: UUID
project_id: UUID
dataset_id: UUID | None = None
job_id: UUID | None = None
analysis_type: str
status: str
model_name: str | None = None
model_version: str | None = None
parameters_json: dict
result_json: dict | None = None
error_message: str | None = None
created_at: datetime | None = None
started_at: datetime | None = None
finished_at: datetime | None = None
model_config = {"from_attributes": True}
class DetectionRunListResponse(BaseModel):
items: list[DetectionRunRead]
total: int
class DetectionRead(BaseModel):
id: UUID
project_id: UUID
dataset_id: UUID | None = None
analysis_run_id: UUID | None = None
job_id: UUID | None = None
model_name: str
model_version: str | None = None
class_name: str
confidence: float
bbox_json: dict | None = None
source_tile_path: str | None = None
properties_json: dict | None = None
created_at: datetime | None = None
model_config = {"from_attributes": True}
class DetectionListResponse(BaseModel):
items: list[DetectionRead]
total: int
backend/app/schemas/export.py
+69
View File
@@ -0,0 +1,69 @@
from __future__ import annotations
from datetime import datetime
from typing import Literal
from uuid import UUID
from pydantic import BaseModel, model_validator
ExportKind = Literal["dataset", "detection_run", "segmentation_run"]
class GeoJsonExportRequest(BaseModel):
dataset_id: UUID | None = None
analysis_run_id: UUID | None = None
export_kind: ExportKind = "dataset"
name: str | None = None
@model_validator(mode="after")
def validate_target(self) -> "GeoJsonExportRequest":
if self.export_kind == "dataset" and self.dataset_id is None:
raise ValueError("dataset_id is required for dataset GeoJSON exports")
if self.export_kind in {"detection_run", "segmentation_run"} and self.analysis_run_id is None:
raise ValueError("analysis_run_id is required for run GeoJSON exports")
return self
class MetadataExportRequest(BaseModel):
project_id: UUID
name: str | None = None
class ReportExportRequest(BaseModel):
project_id: UUID
name: str | None = None
class ExportRead(BaseModel):
id: UUID
project_id: UUID
analysis_run_id: UUID | None = None
export_type: str
storage_path: str
metadata_json: dict | None = None
created_at: datetime | None = None
status: str = "ready"
model_config = {"from_attributes": True}
class ExportCreateResponse(BaseModel):
export_id: UUID
path: str
status: str
export_type: str
metadata_json: dict | None = None
class ExportListResponse(BaseModel):
items: list[ExportRead]
total: int
limit: int
offset: int
class ExportContentResponse(BaseModel):
export_id: UUID
export_type: str
content: dict
backend/app/schemas/external.py
+68
View File
@@ -0,0 +1,68 @@
from __future__ import annotations
from uuid import UUID
from pydantic import BaseModel
class ProviderCapabilityResponse(BaseModel):
provider_name: str
display_name: str
authority_level: str
supported_layers: list[str]
supported_geometry_types: list[str]
supported_query_modes: list[str]
fetch_signature: str
configured: bool
status: str
limitation_message: str
attribution: str
license_note: str
not_configured_reason: str | None = None
class ProviderCapabilitiesResponse(BaseModel):
providers: list[ProviderCapabilityResponse]
class ProviderLayersResponse(BaseModel):
provider_name: str
layers: list[str]
class ProviderStatusResponse(BaseModel):
provider_name: str
configured: bool
status: str
limitation_message: str
class ExternalFetchRequest(BaseModel):
project_id: UUID
area_id: UUID | None = None
layers: list[str] = []
class ExternalFetchResponse(BaseModel):
provider: str
status: str
message: str
requested_layers: list[str]
project_id: UUID
area_id: UUID | None = None
class ProviderImportRequest(BaseModel):
project_id: str
area_id: str | None = None
layers: list[str] = []
dataset_role: str | None = None
class ProviderImportResponse(BaseModel):
provider_name: str
status: str
message: str
requested_layers: list[str]
dataset_id: str | None = None
dataset_role: str | None = None
source_name: str | None = None

Some files were not shown because too many files have changed in this diff Show More