geointel/CHANGELOG.md

# M13 — Codex Optimization Pack

- Added reusable Codex skills under `skills/`.
- Added prompt discipline, token/context budget policy, secrets policy and parallel-agent strategy.
- Added M13 day-one optimized master prompt and pass completion report prompt.
- Added M13 validation script and included it in readiness checks.

# Changelog

## Sprint 26 frontend workflow hook hardening (2026-06-17)

- Moved Detection Lab orchestration state and API calls from `App.tsx` into `useDetectionWorkflow`.
- Moved Segmentation Lab orchestration state and API calls from `App.tsx` into `useSegmentationWorkflow`.
- Added shared frontend `formatError` helper.
- Added regression coverage to verify `App.tsx` wires the workflow hooks and panels without direct detection/segmentation API ownership.
- No API contracts, backend behavior, migrations, product features, provider fetching, AI behavior or UI redesign were introduced.

## Sprint 25 YOLO compatibility smoke hardening (2026-06-17)

- Added an explicit `--check-model-load` mode to `scripts/yolo_preflight.py`.
- Added the same YOLO preflight entrypoint under `backend/scripts/` so it can run inside the backend Docker container.
- The smoke loads only a configured local model file through the YOLO adapter, runs no inference and does not download weights.
- The CLI rejects `--check-model-load` together with `--assume-dependencies` to avoid false-positive AI readiness.
- Added tests for successful mocked model-load smoke, load failure reporting and CLI guard behavior.
- Added the YOLO preflight script to the main readiness gate via Python compile validation.
- No base dependencies, API contracts, migrations, product features, provider fetching or detection persistence behavior were changed.

## Sprint 24 demo/export artifact cleanup tooling (2026-06-17)

- Added `scripts/cleanup_demo_artifacts.py`, a dry-run-first maintenance script for old offline demo export artifacts.
- Added the same cleanup entrypoint under `backend/scripts/` so it can run inside the backend Docker container.
- The cleanup keeps the newest exports per matching demo project, deletes only explicit `exports` rows/files when `--apply` is set and refuses file deletion outside `STORAGE_ROOT`.
- Added tests for cleanup candidate selection, storage-root path safety and readiness gate coverage.
- Added the cleanup script to the main readiness gate via Python compile validation.
- No API contracts, migrations, product features, provider fetching, AI inference or source dataset cleanup behavior were changed.

## Sprint 23 V1 report handoff summary (2026-06-17)

- Added V1 readiness summary data to project metadata exports.
- Added a V1 Readiness Summary and Known Limitations section to lightweight HTML project reports.
- The summary covers project, AOI, dataset readiness, QA/QC and export history using persisted state.
- No new report designer, PDF generation, provider fetching, AI inference, migrations or API route changes were introduced.

## Sprint 22 V1 workbench status strip (2026-06-17)

- Added a compact frontend status strip for project, AOI, datasets, active map layer, QA/QC and exports.
- The strip is driven by existing App state and suggests the next operator action in the V1 loop.
- Hardened the MapLibre component so GeoJSON sources/layers wait for the map style to finish loading before updates run.
- Hardened the offline demo workflow so duplicate historical demo projects prefer complete fixture state before repairing incomplete state.
- Added regression coverage to ensure the strip remains wired without introducing new API calls.
- No backend behavior, migrations, API contracts, provider downloads, AI inference or new dependencies were introduced.

## Sprint 21 V1 demo workflow smoke hardening (2026-06-17)

- Hardened the browser-facing demo/export smoke to verify connected V1 state: project area GeoJSON, fixture datasets, vector FeatureCollection content, vector feature summary, persisted QA/QC metrics and export downloads.
- Loading the offline demo workflow in the frontend now opens the candidate vector fixture dataset directly, so the map workbench is populated after the demo action.
- Added regression tests for the strengthened demo smoke and frontend demo dataset loading contract.
- No migrations, provider downloads, AI inference, new dependencies or API route renames were introduced.

## Sprint 20 selected area map overlay (2026-06-17)

- Added persisted AOI GeoJSON to area API responses without changing the database schema.
- Added a dedicated MapLibre area overlay layer with visibility and opacity controls.
- Added area list actions to choose which AOI is shown on the map.
- Added regression tests for area GeoJSON serialization and frontend map overlay wiring.
- No migrations, provider downloads, AI inference, new dependencies or API route renames were introduced.

## Sprint 19 V1 map workbench controls (2026-06-17)

- Added MapLibre layer visibility and opacity controls for the active GeoJSON workbench layer.
- Added click-to-inspect feature properties for the active map layer.
- Updated the visible app identity from the stale Sprint 9 label to GeoIntel Kempen V1 Workbench.
- Added regression tests that lock the map control and feature inspection wiring.
- No API contracts, migrations, backend behavior, provider fetching, AI inference or new dependencies were introduced.

## Sprint 18 vector change detection foundation (2026-06-16)

- Added `POST /api/v1/analysis/change-detection` for synchronous comparison of two vector datasets in the same project.
- Added a `ChangeDetectionService` that prefers persisted `vector_features`, falls back to stored GeoJSON with an explicit warning, and returns added/removed/unchanged GeoJSON features.
- Added a frontend Change Detection panel and MapLibre change overlay styling for added, removed and unchanged geometries.
- Added nginx no-cache headers for frontend HTML/assets so LAN Docker rebuilds are visible without stale browser modules.
- Added backend tests for persisted vector feature comparison and canonical API envelope behavior.
- No migrations, live provider fetching, AI inference, new dependencies, LiDAR, Copilot, Training Studio or separate Reports module were introduced.

## Release hardening audit pass (2026-06-15)

- Replaced remaining backend `datetime.utcnow()` usage with timezone-aware UTC timestamps.
- Verified the affected backend tests with `DeprecationWarning` promoted to errors.
- Split the frontend production bundle into explicit app, React vendor and MapLibre vendor chunks.
- Raised the Vite chunk warning threshold to match the isolated MapLibre GIS dependency rather than masking app-code growth.
- Hardened the main readiness gate so backend deprecation warnings fail release readiness.
- Added API contract smoke validation to the main readiness gate.
- Hardened the pass-end placeholder scan to skip dependency, build-output and bytecode-cache folders.
- Added backend tests for the release/readiness script expectations.
- Updated `docs/TODO.md` with current implementation status while preserving older planning context.
- No API contracts, migrations, product features, AI dependencies or provider behavior were changed.

## Docker runtime hardening (2026-06-15)

- Fixed the backend Docker build by copying `README.md` and `app/` before `pip install .`.
- Removed mandatory Compose `.env` references so `docker compose up` works with checked-in local defaults.
- Published the Docker Compose frontend on host port `1202`.
- Added backend CORS defaults for `http://localhost:1202` and `http://127.0.0.1:1202`.
- Stopped publishing PostGIS on host port `5432`; backend uses Docker-internal `db:5432`.
- Added a PostGIS healthcheck and made the backend wait for a healthy database.
- Added a backend Docker start script that retries a real SQL connection before running migrations, avoiding first-start database race conditions.
- Made the backend container run `alembic upgrade head` before starting Uvicorn.
- Added backend/frontend `.dockerignore` files to keep dependency folders, build outputs and caches out of Docker build contexts.
- Added Docker runtime configuration regression tests.
- Fixed Alembic logging format so Docker migration logs no longer print literal `%(levelname)` formatter strings.
- Changed the frontend API default to same-origin requests and added a Vite proxy for `/api` and `/health`, with Docker routing to `http://backend:8000`.
- Added a browser runtime verification script that fails when the frontend `/api` proxy returns Vite HTML instead of the backend JSON envelope.
- Updated environment and local runbook documentation so Docker/LAN browser clients use same-origin API calls through the frontend proxy by default.
- Corrected example YOLO environment variables to the names the backend actually reads: `YOLO_ENABLED`, `YOLO_MODEL_PATH` and `YOLO_MAX_TILES`.
- Replaced the Docker frontend runtime with an nginx-served production build and explicit `/api` plus `/health` reverse proxy to the backend service, avoiding Vite HTML fallback for API requests.

## M6 — Codex Autonomy Pack

Added:

- M6 autonomy boundaries.
- M6 quality gates.
- Codex self-review checklist.
- Failure recovery playbook.
- Gap registry.
- Next-day execution checklist.
- Final handoff template.
- Codex pass prompts PASS 00 through PASS 12.
- GitHub issue templates and PR template.
- GitHub Actions docs/contract smoke workflow.
- Codex preflight and pass-end scripts.

Purpose:

- Prepare the repository so Codex can build with strict guidance and bounded improvement freedom.

## M7 - Implementation Control Layer

Added:

- M7 implementation control layer.
- Locked build sequence.
- Regression trap catalogue.
- Codex self-review checklist.
- Geospatial calculation rules.
- Frontend state rules.
- API response rules.
- Module completion matrix.
- Codex decision boundaries.
- Proposed improvements backlog.
- End-of-pass review prompts.
- Regression and contract drift audit prompts.
- Module contracts for project/area/dataset, detection boundary and QA/QC.
- M7 self-review scripts.

## M8 - Tomorrow Execution Pack

- Added Day 1 Codex execution pack.
- Added pass-by-pass Day 1 prompts.
- Added autonomy boundaries, failure recovery and quality gate matrix.
- Added operator checklist and smoke script scaffold.
- Added next-pass guidance for Day 2 GeoAI loop.

## v0.9 — M9 Max Preparation

- Added Codex day-one master prompt.
- Added autonomous build doctrine and pass scorecards.
- Added real-vs-demo data policy and detailed data contracts.
- Added geospatial edge cases, UI state spec and API validation examples.
- Added implementation review script, regression map and gap-to-task conversion rules.
- Added final pre-code checklist and long-form prompt variants.

## Sprint 1 readiness hardening (2026-06-11)

- Added cross-platform backend/runtime scripts with `python`/`python3` fallback in readiness tooling.
- Fixed backend packaging metadata so editable install works in current flat repo layout.
- Added dependency and smoke test script updates for Sprint 1 services.
- Added minimal Sprint 1 tests for health endpoint, GeoJSON metadata extraction, invalid payload rejection, and dataset content reads.
- Fixed frontend README doc reference typo for repository conventions.
- Updated Sprint 1 docs to include backend import smoke and concrete local setup commands.

## Sprint 2 foundation (2026-06-11)

- Added canonical vector/raster dataset handling and lifecycle status transitions (`uploaded`, `validating`, `ready`, `failed`).
- Added vector metadata extraction (feature count, geometry types, bounds, approximate area, CRS assumptions).
- Added raster metadata extraction service with dependency-aware fallback (`RASTER_PROCESSING_UNAVAILABLE`).
- Added deterministic storage metadata capture (`original_filename`, `stored_filename`, `content_type`, `size_bytes`, `checksum_sha256`) and upload folder layout.
- Added dataset inspection/vector summary/raster metadata API endpoints and frontend detail panel support.
- Added Sprint 2 tests for vector metadata, invalid GeoJSON handling, legacy geojson compatibility and raster dependency fallback.

## Sprint 3 foundation (2026-06-11)

- Added job model and database migration for queued/running/success/failed operations.
- Added job APIs for create/list/read/status under project scope.
- Added vector operation services and route wiring for inspect/bbox/stats/clip/buffer/intersect.
- Added raster operation scaffolding for inspect/metadata/preview, with dependency-aware clip/tile unavailability.
- Added frontend operation controls, job status display, and derived dataset link-through in dataset detail panel.
- Updated API contracts and execution log for Sprint 3 foundations.

## Sprint 5 raster analytics hardening (2026-06-11)

- Added raster band statistics operation:
  - min, max, mean, std, nodata count, nodata ratio, valid pixel count, dtype, band index and optional histogram.
  - chunked raster reads to reduce memory pressure and explicit dependency-aware unavailable mode when raster libs are missing.
- Added raster reproject operation foundation with CRS validation:
  - supports target CRS selection via explicit parameter,
  - persists derived output dataset with operation provenance,
  - records operation parameters and error details when invalid.
- Hardened raster clip and tile manifest flow:
  - explicit empty clip failure behavior,
  - bounds/metadata refresh and improved tile manifest fields.
- Added raster operation job persistence tests:
  - result_json and error_message persistence,
  - dependency-aware statistics failure behavior,
  - invalid CRS handling,
  - output linkage for reprojected datasets.
- Extended dataset UI dataset detail job panel:
  - raster metadata visibility (CRS, bounds, resolution),
  - raster band statistics rendering,
  - reproject form and job result visibility.

## Sprint 6 local spectral indices (2026-06-12)

- Added local spectral index operations:
  - NDVI endpoint
  - NDWI endpoint
  - NDBI endpoint
- Added explicit spectral index input validation:
  - positive integer checks
  - source raster band-count bounds checks
- Implemented dependency-aware index execution for missing raster dependencies.
- Added local spectral raster output generation using float32 and `NaN` invalid handling.
- Stored index-derived dataset provenance metadata:
  - `source_dataset_id`
  - `operation` (`raster.ndvi`, `raster.ndwi`, `raster.ndbi`)
  - `band_mapping`
  - `formula`
  - `output_dtype`
  - `nodata_strategy`
  - `value_range_note`
  - `created_at`
  - `output_dataset_id`
  - `path`
- Extended dataset detail UI with spectral index controls and result dataset actions.
- Updated:
  - `docs/API_CONTRACTS.md`
  - `docs/RASTER_OPERATIONS_SPEC.md`
  - `backend/README.md`
  - `frontend/README.md`
  - `docs/CODEX_EXECUTION_LOG.md`

## Sprint 7A persistence and QA foundation (2026-06-12)

- Added `vector_features` as first-class queryable vector state while preserving original uploaded files as source artifacts.
- Added `quality_checks` and `metrics` as persisted QA/QC domain records.
- Added Alembic migration `202606120700_sprint7a_persistence_foundation.py` for vector features, quality checks, metrics and required indexes.
- Persisted uploaded vector GeoJSON feature properties and geometries into PostGIS-backed feature rows.
- Updated QA candidate-vs-reference jobs to persist quality checks and metric rows and return `quality_check_id`.
- Hardened GRB/OSM provider contracts as honest `not_configured` capability stubs only.
- Added tests for Sprint 7A persistence, dataset role validation, provider contracts, migration integrity and QA route persistence.

## Sprint 7B provider integration skeleton (2026-06-12)

- Added central provider registry entries for `grb`, `osm`, `manual` and `fixture`.
- Added provider capability, layer, status and future import-contract endpoints using the existing API envelope style.
- Kept GRB and OSM as explicit `not_configured` providers with no live WFS, Overpass or download behavior.
- Added provider-to-dataset mapping rules for future imports through `DatasetService` and `VectorFeatureService`.
- Added lightweight frontend Provider Capabilities panel with status, authority, layers, query modes and limitations.
- Added live PostGIS migration smoke script for opt-in local database verification.
- Added Sprint 7B provider registry/API tests.

## Sprint 8 Detection Lab foundation (2026-06-12)

- Added `detections` as first-class persisted PostGIS records linked to project, dataset, job and analysis run.
- Hardened `analysis_runs` with dataset/job/model/result metadata for future detection and segmentation workflows.
- Added model registry capabilities for `yolo-placeholder` and `manual-fixture-detector`.
- Added Detection Lab service and API foundation with dependency-aware `DETECTION_MODEL_UNAVAILABLE` responses.
- Added explicit fixture detector mode for tests/demo fixtures only; no fake production inference was introduced.
- Added minimal frontend Detection Lab panel for model status, raster dataset selection, confidence threshold and run status.
- Added Sprint 8 tests for persistence, model capabilities, unavailable model behavior, invalid dataset validation, explicit fixture persistence and API envelope shape.

## Sprint 8B configured YOLO foundation (2026-06-12)

- Added optional `ai` backend dependency group for Ultralytics/Torch without making AI dependencies mandatory for normal startup.
- Added `yolo-configured` model registry capability with `not_configured`, `dependency_unavailable` and `configured` status behavior.
- Added import-safe YOLO adapter that loads only an existing local model path and does not auto-download weights.
- Added raster tile manifest validation, configured tile limits and pixel bbox to EPSG:4326 detection polygon conversion.
- Added mocked YOLO persistence tests that verify first-class detection records without requiring YOLO dependencies.
- Added Detection Lab tile manifest path input for configured YOLO runs.
- Updated AI/API/backend/frontend docs for Sprint 8B configuration and limitations.

## Sprint 8C detection visualization and QA integration (2026-06-12)

- Added detection result review endpoints for run lists, filtered detections, detection detail and GeoJSON FeatureCollection output.
- Added detection QA against persisted reference `vector_features` using existing `quality_checks` and `metrics`.
- Added frontend Detection Lab run selection, detection table, class/confidence filters and MapLibre detection GeoJSON overlay.
- Added frontend detection QA controls and metric summary display.
- Added tests for detection GeoJSON shape, filters, detail, QA persistence, no-match QA and Sprint 8B manifest edge cases.
- Segmentation, LiDAR, Copilot, Training Studio and Reports remain out of scope.

## Sprint 9 Segmentation Lab foundation (2026-06-12)

- Added `segmentations` as first-class persisted PostGIS MultiPolygon records linked to project, dataset, job and analysis run.
- Added deterministic segmentation mask path convention under `storage/masks/{project_id}/{analysis_run_id}/tile_{tile_index}/`.
- Added segmentation model registry capabilities:
  - `segmentation-placeholder`
  - `fixture-segmenter`
  - `yolo-seg-configured`
  - `sam-configured`
- Added Segmentation Lab service and API foundation for model listing, run creation, run/result listing, detail, GeoJSON output and reference QA.
- Added segmentation QA against persisted reference `vector_features` using existing `quality_checks` and `metrics`.
- Added minimal frontend Segmentation Lab UI with model status, raster selection, run/result table, map overlay and QA metric display.
- Real SAM, real YOLO-seg, model downloads, new AI dependencies, LiDAR, Copilot, Training Studio and Reports remain out of scope.

## Sprint 10 release hardening and modularization (2026-06-13)

- Extracted project, area, provider capabilities, Detection Lab and Segmentation Lab UI sections from `frontend/src/App.tsx` into focused components.
- Preserved existing API client usage, state ownership, MapLibre overlay behavior and workbench UX.
- Hardened readiness checks to include Alembic head verification and `scripts/live_migration_smoke.sh` syntax validation.
- No new product features, migrations, AI dependencies or live external provider fetching were introduced.

## Sprint 11 live Docker/PostGIS runtime validation (2026-06-13)

- Hardened `scripts/live_migration_smoke.sh` so fresh databases run Alembic migrations before checking `PostGIS_Version()`.
- Added live runtime schema-object checks for core migrated tables and geometry indexes.
- Added backend tests that lock the live migration smoke ordering and schema-check contract.
- Documented exact Docker/PostGIS validation commands, expected `DATABASE_URL` and local cleanup commands.
- Docker was unavailable in the current shell, so live container execution remains pending on a Docker-enabled machine.

## Sprint 12 QA/QC golden dataset and benchmarking (2026-06-15)

- Added deterministic golden building QA/QC fixtures and expected metric baseline.
- Added `scripts/run_golden_qa_benchmark.py` to run existing QA/QC logic against the golden fixtures and fail on metric drift.
- Added backend tests covering expected golden metrics and `QualityCheck`/`Metric` persistence verification.
- Documented benchmark purpose, command, expected outputs, tolerance and limitations.
- No product features, API contracts, migrations, live providers, AI models or new dependencies were introduced.

## Sprint 13 real YOLO operational hardening (2026-06-15)

- Added `YoloPreflightService` for local configured-YOLO readiness checks without loading models or running inference.
- Added `scripts/yolo_preflight.py` for checking enabled state, optional dependency availability, local model path, tile manifest validity, tile limit and referenced tile paths.
- Added backend tests for disabled, dependency-unavailable and ready preflight states plus CLI JSON output.
- Documented preflight usage in backend and AI pipeline docs.
- No model downloads, API contracts, migrations, new dependencies, segmentation behavior or provider fetching were introduced.

## Sprint 14 Docker GIS runtime enablement (2026-06-16)

- Added a backend `gis` optional dependency group for the approved raster/vector runtime stack.
- Updated the backend Docker image to install the `gis` extra plus GDAL/GEOS/PROJ system packages.
- Added `scripts/verify_gis_runtime.sh` to verify browser-facing PostGIS, Rasterio and GeoPandas capabilities through the frontend proxy.
- Added `scripts/gis_import_smoke.py` and made the backend Docker build fail if Rasterio, GeoPandas or pyogrio cannot be imported.
- Moved the Docker build-time GIS import smoke into the backend build context and kept the root script as a local wrapper.
- Included the GIS runtime script syntax check in the main readiness gate.
- Added backend and frontend Docker Compose healthchecks and made the frontend wait for a healthy backend.
- Added regression tests for Docker GIS dependency installation and capability verification script coverage.
- Documented Docker GIS runtime verification commands for local and LAN deployments.
- No API contracts, migrations, AI dependencies, provider fetching or product features were changed.

## Sprint 15 explicit demo workflow seed (2026-06-16)

- Added `POST /api/v1/demo/workflow` to seed or return an explicit offline demo workflow.
- The demo workflow creates a project, AOI, fixture reference building dataset, fixture candidate building dataset and persisted QA/QC metrics.
- Added `scripts/seed_demo_workflow.py` for CLI-based demo seeding.
- Added frontend "Load demo workflow" action in the Projects panel.
- Added tests for the demo endpoint envelope and fixture contract.
- No live GRB/OSM fetching, AI inference, migrations or new dependencies were introduced.

## Sprint 16 QA/QC result visibility (2026-06-16)

- Added `GET /api/v1/projects/{project_id}/quality-checks` to list persisted quality checks and metric rows.
- Added a frontend QA/QC Results panel for project-level persisted QA output.
- Demo workflow loading and QA actions now refresh visible QA/QC results.
- Added backend tests for quality check listing and canonical response envelopes.
- No migrations, new dependencies, live provider fetching or AI inference were introduced.

## Sprint 17 export foundation (2026-06-16)

- Hardened `POST /api/v1/exports/geojson` so exports persist `Export` rows instead of returning dataset ids as export ids.
- Added GeoJSON export support for vector datasets, detection runs and segmentation runs using existing persisted geometry services.
- Added project metadata JSON export, lightweight project report HTML export and export list/read/content/download endpoints.
- Added a frontend Export Center panel for creating exports, listing export records, previewing JSON artifact content and downloading artifacts.
- Added export history to project metadata/report artifacts.
- Added `scripts/verify_demo_export_workflow.sh` to smoke test demo seeding, QA/QC visibility, metadata/report/vector exports, export listing and artifact downloads through the browser-facing URL.
- Added backend tests for export persistence, artifact writing, raster rejection, HTML report creation, canonical export envelopes and raw file downloads.
- No migrations, new dependencies, live provider fetching, AI inference, LiDAR, Copilot, Training Studio or separate Reports module were introduced.