Initial public release

2026-08-31 21:56:53 +02:00
commit faeb58ef6d
1386 changed files with 263203 additions and 0 deletions
@@ -0,0 +1,41 @@
+# BACKLOG PRIORITIES — MoSCoW
+
+## Must have for Sprint 1
+
+- Backend app foundation.
+- Database/PostGIS foundation.
+- Project Manager.
+- Area Manager.
+- Dataset Manager for GeoJSON.
+- Metadata extraction.
+- Frontend shell.
+- MapLibre map.
+- Demo fixture display.
+- Readiness checks.
+
+## Should have for Sprint 1 if musts are complete
+
+- GeoJSON export baseline.
+- Area import from fixture.
+- Dataset detail metadata panel.
+- Simple layer tree.
+- Basic smoke tests for frontend build.
+
+## Could have later in same early phase
+
+- Reference/prediction fixture import.
+- QA/QC schema validation.
+- Simple feature statistics.
+- Layer opacity controls.
+
+## Won't have in Sprint 1
+
+- Live YOLO inference.
+- Live SAM segmentation.
+- GRB live WFS.
+- Sentinel automation.
+- LiDAR processing.
+- Training Studio.
+- AI Copilot.
+- Full PDF report generator.
+- Multi-user auth.
@@ -0,0 +1,78 @@
+# BUILD ORDER GRAPH — M14 Launch
+
+Codex must follow this dependency graph for implementation.
+
+## Graph
+
+```text
+Repository readiness
+  -> Environment/config
+  -> Backend app foundation
+  -> Database/PostGIS foundation
+  -> Domain models/schemas
+  -> Migrations
+  -> Project API
+  -> Area API
+  -> Dataset API
+  -> Storage service
+  -> Metadata extraction
+  -> Frontend shell
+  -> Frontend API client
+  -> Map workbench shell
+  -> Dataset UI
+  -> Fixture layer display
+  -> Export baseline
+  -> Tests and readiness
+```
+
+## Blocked until foundation is stable
+
+The following are blocked until the above graph is green:
+
+- Detection Lab.
+- Segmentation Lab.
+- QA/QC engine beyond fixture/schema validation.
+- Remote Sensing Lab.
+- GRB live adapter.
+- OSM live adapter.
+- LiDAR Workbench.
+- Training Studio.
+
+## Rule
+
+If a later module requires missing foundation work, Codex must complete the foundation work first instead of building around it.
+
+## Preferred first implementation passes
+
+### Pass 1 — Backend and config
+
+- Create app structure.
+- Add config/settings.
+- Add health endpoint.
+- Add response envelope and error handling.
+
+### Pass 2 — Database and domain
+
+- Add database connection.
+- Add migrations.
+- Add core models.
+- Enable PostGIS.
+
+### Pass 3 — Project/Area/Dataset API
+
+- Implement CRUD.
+- Validate geometry.
+- Extract metadata.
+
+### Pass 4 — Frontend shell and map
+
+- React app.
+- Layout.
+- MapLibre.
+- API client.
+
+### Pass 5 — Fixture-driven integration
+
+- Load demo area/reference layer.
+- Display layers.
+- Run smoke tests.
@@ -0,0 +1,106 @@
+# BUILD SUCCESS DEFINITION — Sprint 1 / First Codex Run
+
+This document defines the exact point at which Codex must stop expanding scope and consider the first implementation run successful.
+
+## Purpose
+
+The first build is not successful because many features exist. It is successful when the foundation is stable, testable and ready for the next module.
+
+## Sprint 1 success statement
+
+Sprint 1 is successful when GeoIntel can run locally with a backend, frontend, database and a minimal geospatial dataset workflow.
+
+The vertical slice is:
+
+```text
+Project
+  -> Area
+  -> Dataset upload/registration
+  -> Metadata extraction
+  -> PostGIS persistence
+  -> Minimal map display
+  -> GeoJSON export
+```
+
+## Required backend success criteria
+
+- FastAPI application starts without import errors.
+- `/health` returns a successful response.
+- `/docs` or OpenAPI schema is available in development.
+- Database configuration is loaded from environment variables.
+- PostgreSQL connection is verified by a health or readiness check.
+- PostGIS extension is created or verified by migration/bootstrap logic.
+- Project CRUD works through API endpoints.
+- Area CRUD works with valid GeoJSON polygon input.
+- Dataset registration or upload works for at least GeoJSON.
+- Dataset metadata is extracted and persisted.
+- API responses use the canonical response envelope.
+- Errors use the canonical error format.
+
+## Required database success criteria
+
+- Alembic or equivalent migration path exists.
+- Core tables exist:
+  - `projects`
+  - `areas`
+  - `datasets`
+  - `dataset_versions` or documented equivalent
+  - `analysis_runs` placeholder/table if needed for future compatibility
+  - `exports` placeholder/table if needed for future compatibility
+- Geometry columns use PostGIS types.
+- CRS/SRID rules follow `docs/specs/GIS_STANDARDS.md`.
+- Demo seed data can be loaded or fixtures can be used in tests.
+
+## Required frontend success criteria
+
+- React/Vite application starts.
+- Main layout is visible.
+- MapLibre map renders.
+- Project list/detail state exists.
+- Dataset upload/registration UI exists for the Sprint 1 dataset type.
+- Area display or drawing/import flow exists in minimal form.
+- API client uses configured backend base URL.
+- Loading, empty and error states exist for the implemented pages.
+
+## Required storage success criteria
+
+- Uploaded/registered dataset files are stored under a controlled storage path.
+- Storage paths are not hardcoded to a developer machine.
+- Metadata in the database references stored files where applicable.
+- Generated exports go to a controlled exports folder.
+
+## Required testing success criteria
+
+At minimum:
+
+- Backend import smoke test passes.
+- Health endpoint test passes.
+- Database connection/migration smoke test passes or has a documented fallback if no DB is available in CI.
+- GeoJSON fixture validation passes.
+- Frontend build or typecheck passes.
+- `make readiness` passes.
+
+## Explicitly not required for Sprint 1
+
+Do not block Sprint 1 on:
+
+- YOLO inference.
+- SAM inference.
+- Sentinel download automation.
+- GRB live WFS integration.
+- LiDAR processing.
+- Training Studio.
+- AI Copilot.
+- Advanced reporting.
+- Multi-user authentication.
+- Production deployment.
+
+## Stop condition
+
+When all Sprint 1 success criteria are met, Codex must stop feature expansion and produce:
+
+- changelog entry;
+- commands run;
+- test results;
+- known limitations;
+- next pass recommendation.
@@ -0,0 +1,45 @@
+# CODEX STOP RULES
+
+Codex must stop or pause expansion when these conditions occur.
+
+## Hard stop conditions
+
+Stop implementation and report if:
+
+- backend cannot import;
+- frontend cannot build due to own changes;
+- migrations cannot be generated or applied due to unclear schema conflict;
+- API contract conflict is found;
+- architecture invariant would need to be broken;
+- dependency choice conflicts with ADRs;
+- secrets or API keys are accidentally introduced;
+- external service is required for a Sprint 1 must-have.
+
+## Soft stop conditions
+
+Pause expansion and finish cleanup if:
+
+- tests fail after the intended module is implemented;
+- a feature starts requiring a non-Sprint-1 module;
+- implementation requires more than one new abstraction not already documented;
+- generated code duplicates existing logic;
+- TODO comments are being used to hide incomplete logic.
+
+## What Codex must do at stop
+
+Report:
+
+- exact blocker;
+- files affected;
+- commands run;
+- failing output summary;
+- recommended fix;
+- whether rollback is needed.
+
+## What Codex must not do
+
+- Do not continue building unrelated modules while the build is broken.
+- Do not silence errors by weakening tests.
+- Do not replace real functionality with permanent mock logic.
+- Do not change the architecture to make one test pass.
+- Do not introduce a new dependency without ADR-compatible justification.
@@ -0,0 +1,116 @@
+# DATA ACQUISITION PLAYBOOK
+
+This playbook tells Codex how to think about data acquisition without spending the first build researching or overengineering live integrations.
+
+## Principle
+
+Sprint 1 uses local demo fixtures first. Live external data integrations are later adapters.
+
+The data hierarchy is:
+
+1. Golden fixtures for tests and UI development.
+2. Local user-uploaded files.
+3. Cached reference extracts.
+4. Live external data services.
+
+## Sprint 1 data sources
+
+### Golden GeoJSON fixtures
+
+Use existing files in `demo/geel/` as the first source of truth:
+
+- `area_geel_center.geojson`
+- `reference_buildings.geojson`
+- `demo_detections.geojson`
+- `expected_qaqc_metrics.json`
+
+These are not meant to be geographically complete. They are contract fixtures.
+
+### User upload/register
+
+Support GeoJSON first. Validate:
+
+- file extension;
+- JSON parse;
+- FeatureCollection shape;
+- geometry existence;
+- CRS handling fallback;
+- bounds calculation.
+
+## GRB strategy
+
+GRB is the professional reference layer for Vlaanderen. It is the preferred future reference for buildings and related vector features.
+
+### Sprint 1
+
+- Do not implement live GRB WFS yet.
+- Use GRB-like local fixture data.
+- Design a `ReferenceDataAdapter` interface so GRB can be added without rewriting QA/QC.
+
+### Sprint 2+
+
+- Add GRB adapter.
+- Prefer bbox/area-scoped retrieval.
+- Cache retrieved features in PostGIS.
+- Track source, retrieval timestamp and layer name.
+
+### Fallback
+
+If GRB is unavailable:
+
+- use cached extract;
+- show external-source-unavailable status;
+- do not fake live data.
+
+## OSM strategy
+
+OSM can be useful as a broad fallback/reference but must not replace GRB for professional building QA where GRB is available.
+
+### Sprint 1
+
+- No live OSM required.
+- Keep adapter boundary ready.
+
+### Later
+
+- Use Overpass or local extracts for bounded areas.
+- Cache in PostGIS.
+
+## Sentinel strategy
+
+Sentinel belongs to the Remote Sensing Lab, not Sprint 1.
+
+### Sprint 1
+
+- No Sentinel automation.
+- Do not add Copernicus dependencies.
+
+### Later
+
+- Prefer STAC-based lookup where possible.
+- Support NDVI/NDWI/NDBI through raster pipeline.
+- Cache downloaded scenes/derived rasters.
+
+## DHMV / height data strategy
+
+DHMV/DEM/DSM belongs after raster/vector foundations are stable.
+
+### Sprint 1
+
+- No height integration.
+
+### Later
+
+- Add DEM/DSM products as raster datasets.
+- Reuse raster metadata, clipping and tiling pipelines.
+
+## Data acquisition acceptance criteria
+
+A new data source is accepted only when it has:
+
+- adapter boundary;
+- source metadata;
+- cache strategy;
+- error handling;
+- tests or fixture equivalent;
+- documentation update.
@@ -0,0 +1,75 @@
+# FOLDER OWNERSHIP AND RESPONSIBILITIES
+
+Codex must keep responsibilities separated.
+
+## `backend/app/api/`
+
+HTTP routes only. No heavy business logic.
+
+## `backend/app/schemas/`
+
+Pydantic request/response models and API DTOs.
+
+## `backend/app/models/`
+
+Database ORM models.
+
+## `backend/app/services/`
+
+Business logic and orchestration.
+
+## `backend/app/repositories/`
+
+Database access patterns if repository layer is used.
+
+## `backend/app/workers/`
+
+RQ/Celery/background job entry points.
+
+## `backend/app/geo/`
+
+GIS-specific helpers: CRS, geometry validation, bounds, area calculations.
+
+## `backend/app/storage/`
+
+File storage logic.
+
+## `backend/tests/`
+
+Backend tests. Tests should not live beside production modules unless project conventions are changed deliberately.
+
+## `frontend/src/pages/`
+
+Route-level pages.
+
+## `frontend/src/components/`
+
+Reusable UI components.
+
+## `frontend/src/features/`
+
+Feature-oriented frontend modules, e.g. projects, datasets, map, exports.
+
+## `frontend/src/services/`
+
+API client/services.
+
+## `frontend/src/stores/`
+
+Client state only if needed.
+
+## `docs/`
+
+Architecture, specs and governance. Update docs when contracts or build rules change.
+
+## `demo/`
+
+Small golden fixtures and demo manifests. Not production storage.
+
+## `datasets/`
+
+Local development data folders. Do not commit large real datasets.
+
+## `storage/`
+
+Runtime storage. Keep `.gitkeep`, do not commit generated artifacts.
@@ -0,0 +1,67 @@
+# GOLDEN DATASET PACKAGE
+
+The golden dataset package defines the stable demo/test inputs that Codex must protect.
+
+## Purpose
+
+Golden data allows Codex to build and test without depending on live external services.
+
+## Golden area
+
+`demo/geel/area_geel_center.geojson`
+
+Represents the canonical Sprint 1 area fixture. It must remain valid GeoJSON.
+
+## Golden reference layer
+
+`demo/geel/reference_buildings.geojson`
+
+Represents reference building polygons. In future this will map to GRB-like reference data.
+
+Required properties:
+
+- stable feature IDs where possible;
+- polygon or multipolygon geometry;
+- deterministic feature count;
+- valid geometries.
+
+## Golden prediction layer
+
+`demo/geel/demo_detections.geojson`
+
+Represents predicted building detections or imported detection outputs.
+
+Required properties:
+
+- class label;
+- confidence where available;
+- polygon or bbox-derived polygon geometry;
+- stable enough for QA/QC fixture checks.
+
+## Golden expected metrics
+
+`demo/geel/expected_qaqc_metrics.json`
+
+Represents expected QA/QC output for the demo fixture. Codex may update this only if:
+
+- the fixture geometry intentionally changes;
+- the QA/QC algorithm version changes;
+- changelog explains the reason.
+
+## Golden data rules
+
+- Do not delete golden fixtures.
+- Do not replace golden fixtures with random generated data.
+- Do not make tests depend on external services when golden fixtures are sufficient.
+- Keep fixtures small enough for fast CI/smoke runs.
+- If new fixture files are added, update `docs/DEMO_FIXTURE_MANIFEST.md`.
+
+## Sprint 1 usage
+
+Codex should use golden data for:
+
+- frontend map layer smoke display;
+- backend fixture validation;
+- dataset metadata extraction tests;
+- future QA/QC regression tests;
+- export contract tests.
@@ -0,0 +1,83 @@
+# MODULE ACCEPTANCE CRITERIA
+
+This document defines when a module is considered done enough to move forward.
+
+## Backend foundation
+
+Done when:
+
+- app imports without error;
+- health endpoint works;
+- config is environment-driven;
+- response envelope is used;
+- error shape is canonical;
+- backend smoke test exists.
+
+## Database foundation
+
+Done when:
+
+- database connection is configured;
+- PostGIS is enabled or checked;
+- migrations exist;
+- core tables can be created;
+- geometry storage works;
+- tests or smoke checks exist.
+
+## Project Manager
+
+Done when:
+
+- project create/list/read works;
+- invalid payloads return canonical errors;
+- frontend can display projects;
+- tests cover at least one create/list/read path.
+
+## Area Manager
+
+Done when:
+
+- valid GeoJSON polygon can be stored;
+- invalid geometry is rejected;
+- bounds or area metadata is returned where practical;
+- geometry is stored in PostGIS;
+- frontend can display at least one area on the map.
+
+## Dataset Manager
+
+Done when:
+
+- GeoJSON dataset can be registered/uploaded;
+- metadata is extracted;
+- metadata is persisted;
+- original file or reference is stored;
+- dataset state follows the state machine;
+- frontend can list datasets.
+
+## Map Workbench Foundation
+
+Done when:
+
+- MapLibre renders;
+- fixture layer can be displayed;
+- layer loading state exists;
+- error state exists;
+- map controls do not break layout.
+
+## Export baseline
+
+Done when:
+
+- at least one valid GeoJSON export path exists;
+- export file is stored or streamed consistently;
+- export metadata is recorded or returned;
+- invalid export requests fail gracefully.
+
+## QA/QC skeleton
+
+Done when:
+
+- schema for reference and prediction layers is documented/implemented;
+- no live AI inference is required;
+- future QA engine can plug into analysis run structure;
+- fixture metrics can be validated if implemented.
@@ -0,0 +1,68 @@
+# RELEASE STRATEGY
+
+GeoIntel releases must be small, testable and aligned with the build order graph.
+
+## Version targets
+
+### v0.1 — Foundation
+
+- Backend starts.
+- Frontend starts.
+- Database/PostGIS works.
+- Health/readiness works.
+
+### v0.2 — Project and Area Manager
+
+- Project CRUD.
+- Area CRUD.
+- Geometry persistence.
+- Map displays area fixtures.
+
+### v0.3 — Dataset Manager
+
+- GeoJSON upload/register.
+- Metadata extraction.
+- Dataset list/detail UI.
+
+### v0.4 — Raster/Vector Foundation
+
+- Raster metadata skeleton if dependencies are available.
+- Vector operations baseline.
+- Layer display improvements.
+
+### v0.5 — Detection Import / Detection Lab Skeleton
+
+- Detection result import.
+- Detection layer display.
+- Model-adapter boundary, no heavy inference required yet.
+
+### v0.6 — QA/QC Foundation
+
+- Reference vs prediction matching.
+- Metrics.
+- False positive/false negative outputs.
+
+### v0.7 — GeoJSON Export and Review
+
+- Stable export workflow.
+- Export validation.
+- Basic review UI.
+
+### v1.0 — GeoAI Workbench MVP
+
+- Stable dataset workflow.
+- Raster/vector foundations.
+- Detection/segmentation architecture.
+- QA/QC workflow.
+- Export workflow.
+- Portfolio-ready demo.
+
+## Release rule
+
+A release cannot be cut if:
+
+- readiness checks fail;
+- golden paths regress;
+- docs are stale;
+- API contracts drift without documentation;
+- known limitations are hidden.
@@ -0,0 +1,82 @@
+# RISK REGISTER
+
+This register captures known build and product risks before the first implementation run.
+
+## R1 — Scope creep during first Codex run
+
+Impact: high.
+
+Mitigation:
+
+- Follow `SPRINT_1_SCOPE_FREEZE.md`.
+- Use `CODEX_STOP_RULES.md`.
+- Add out-of-scope items to backlog.
+
+## R2 — External data source instability
+
+Impact: medium/high.
+
+Mitigation:
+
+- Use golden fixtures in Sprint 1.
+- Add live adapters later.
+- Cache retrieved external data.
+
+## R3 — GIS CRS mistakes
+
+Impact: high.
+
+Mitigation:
+
+- Follow `docs/specs/GIS_STANDARDS.md`.
+- Validate geometries.
+- Store SRID explicitly.
+- Test fixture bounds.
+
+## R4 — Raster files too large
+
+Impact: medium.
+
+Mitigation:
+
+- No heavy raster processing in Sprint 1.
+- Use tiling strategy later.
+- Enforce file size limits when upload is implemented.
+
+## R5 — AI modules introduced too early
+
+Impact: high.
+
+Mitigation:
+
+- Keep live inference out of Sprint 1.
+- Implement adapter boundaries only when needed.
+- Use imported demo detections before live models.
+
+## R6 — Frontend becomes dashboard-first
+
+Impact: medium.
+
+Mitigation:
+
+- Follow Constitution: data -> processing -> QA/QC -> export.
+- Map supports analysis; it is not the product by itself.
+
+## R7 — Mock data becomes permanent
+
+Impact: high.
+
+Mitigation:
+
+- Golden fixtures are contract data, not fake product behavior.
+- Mark demo/fixture paths clearly.
+- Production endpoints must use persisted data.
+
+## R8 — Too many competing docs
+
+Impact: medium.
+
+Mitigation:
+
+- Follow `CODEX_START.md` and `docs/00-start/START_HERE.md`.
+- M14 launch docs supersede older first-run plans where conflicts exist.
@@ -0,0 +1,97 @@
+# SPRINT 1 SCOPE FREEZE
+
+This document freezes the first implementation scope. Codex must not expand Sprint 1 beyond this boundary.
+
+## Sprint 1 goal
+
+Build the GeoIntel foundation vertical slice: a locally runnable app that proves project, area, dataset, PostGIS and map foundations.
+
+## Must build
+
+### Backend foundation
+
+- FastAPI app skeleton.
+- Settings/config module.
+- Structured logging baseline.
+- Health/readiness endpoint.
+- API response envelope.
+- Error handling middleware or equivalent.
+- CORS configuration for local frontend.
+
+### Database foundation
+
+- PostgreSQL/PostGIS connection.
+- Migration tooling.
+- Core models/schemas for projects, areas and datasets.
+- Geometry persistence.
+- Minimal seed/fixture support.
+
+### Project and area workflow
+
+- Create/list/read project.
+- Create/list/read area.
+- Store area geometry.
+- Validate geometry.
+- Return area metadata such as area size where practical.
+
+### Dataset workflow
+
+- Register/upload GeoJSON dataset.
+- Store original file or registered reference.
+- Extract metadata:
+  - dataset type;
+  - feature count;
+  - geometry type;
+  - CRS if available;
+  - bounds.
+- Persist metadata.
+
+### Frontend foundation
+
+- Vite/React/TypeScript app.
+- Main shell/layout.
+- MapLibre map.
+- Project workspace page.
+- Dataset manager page/panel.
+- Minimal layer display using demo GeoJSON.
+- API client.
+
+### Tooling
+
+- Docker Compose for DB and services.
+- README quickstart updated if implementation changes commands.
+- `make readiness` remains green.
+- Basic tests/smoke scripts.
+
+## Should build if Sprint 1 musts are complete
+
+- Area import from fixture.
+- GeoJSON export endpoint for area or dataset features.
+- Minimal QA placeholder that validates reference vs prediction fixture schema, but does not run full QA engine yet.
+- Frontend layer opacity toggle.
+
+## May build only if zero risk
+
+- Demo fixture loader.
+- Simple statistics card for feature count and area.
+- Simple project status panel.
+
+## Must not build in Sprint 1
+
+- YOLO live inference.
+- SAM live segmentation.
+- Sentinel/STAC integration.
+- GRB live WFS client.
+- DHMV integration.
+- LiDAR LAS/LAZ processing.
+- Training Studio.
+- Model registry UI.
+- AI Copilot.
+- Advanced PDF reports.
+- Multi-user authentication.
+- Role-based permissions.
+- Production Kubernetes/deployment stack.
+
+## Scope conflict rule
+
+If a task seems useful but is not listed under `Must`, `Should` or `May`, Codex must not implement it in Sprint 1. Add it to backlog/open issues instead.