1533 lines
82 KiB
Markdown
1533 lines
82 KiB
Markdown
## Sprint 32 Unraid all-in-one runtime (2026-06-17)
|
|
|
|
Changed:
|
|
- Added `docker-compose.unraid.yml` for a single editable `geointel` container on Unraid.
|
|
- Added `deploy/unraid/Dockerfile.all-in-one` to build one image containing embedded PostGIS, backend GIS runtime, nginx and frontend static assets.
|
|
- Added `deploy/unraid/all-in-one-start.sh` to start embedded PostGIS, apply Alembic migrations, start FastAPI on internal localhost and serve nginx on container port 80.
|
|
- Added `deploy/unraid/nginx-all-in-one.conf` so `/api` and `/health` proxy to `127.0.0.1:8000` inside the same container.
|
|
- Updated `deploy/unraid/geointel.env.example`, Unraid XML template and README for one-container operation and editable web/storage/database paths.
|
|
- Updated Tower deploy scripts to stop the old multi-container stack without deleting volumes and start `docker-compose.unraid.yml`.
|
|
- Updated `scripts/live_migration_smoke.sh` to support both all-in-one `geointel` and legacy `backend` Compose services.
|
|
- Added root `.dockerignore` for all-in-one builds.
|
|
- Updated Sprint 31 tests to cover the all-in-one Dockerfile, startscript, nginx config, deploy scripts and template metadata.
|
|
|
|
Validation:
|
|
- `python -m pytest backend/tests/test_sprint31_unraid_template.py backend/tests/test_live_migration_smoke_script.py backend/tests/test_docker_runtime_config.py` passed: 26 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest` passed: 179 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed: 179 backend tests, frontend typecheck/build, Alembic head check and script syntax checks.
|
|
- `cd backend && python -m alembic heads` passed: single head `202606120900`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `bash -n deploy/unraid/all-in-one-start.sh` passed.
|
|
|
|
Notes:
|
|
- No API contracts, migrations, product features, provider fetching or AI behavior changed.
|
|
- Local Windows environment does not have `docker` in PATH; Tower deployment should provide the live all-in-one Docker validation.
|
|
|
|
## Sprint 31 Unraid deployment template (2026-06-17)
|
|
|
|
Changed:
|
|
- Made `docker-compose.yml` configurable through `.env` defaults for frontend port, backend port, storage path, PostGIS database/user/password, CORS origins and upload limit.
|
|
- Added `deploy/unraid/geointel.env.example` for Unraid/Tower runtime configuration.
|
|
- Added `deploy/unraid/geointel-unraid-template.xml` as Unraid/DockerMan-style metadata for the editable Compose stack settings.
|
|
- Added `deploy/unraid/geointel-icon.svg` and `deploy/unraid/geointel-icon.png`, served through matching `frontend/public` assets.
|
|
- Added the frontend favicon link for the GeoIntel icon.
|
|
- Added Sprint 31 tests for Unraid template coverage, compose variable coverage, docs and icon availability.
|
|
- Updated root README, TODO and changelog docs.
|
|
|
|
Validation:
|
|
- `python -m pytest backend/tests/test_sprint31_unraid_template.py backend/tests/test_docker_runtime_config.py` passed: 22 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest` passed: 177 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed: 177 backend tests, frontend typecheck/build, Alembic head check and script syntax checks.
|
|
- `cd backend && python -m alembic heads` passed: single head `202606120900`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- Tower deploy via `scripts/deploy_tower.ps1` passed after commit `3a8b82f`.
|
|
- Tower Docker Compose config/build/up passed with the env-default compose file.
|
|
- Tower live migration smoke passed with PostGIS 3.4 and Alembic head `202606120900`.
|
|
- Tower browser runtime verification passed on `http://192.168.10.150:1202`.
|
|
- Additional HTTP smoke passed for frontend HTML, `GET /api/v1/projects`, `/geointel-icon.svg` and `/geointel-icon.png`.
|
|
|
|
Notes:
|
|
- Local Windows environment does not have `docker` in PATH, so local `docker compose config` could not be run from this machine.
|
|
- Tower deployment validated Docker Compose config and live runtime after commit.
|
|
- No API contracts, backend behavior, migrations, product features, provider fetching or AI behavior changed.
|
|
|
|
## Sprint 30 workbench component decomposition (2026-06-17)
|
|
|
|
Changed:
|
|
- Moved persisted QA/QC result rendering from `frontend/src/App.tsx` into `frontend/src/components/quality/QualityResultsPanel.tsx`.
|
|
- Moved map layer controls, MapLibre composition and feature inspector rendering into `frontend/src/components/map/MapWorkspace.tsx`.
|
|
- Updated map/workbench and QA regression tests for the new component boundaries.
|
|
- Added Sprint 30 component wiring tests to keep QA and map markup out of `App.tsx`.
|
|
- Updated frontend README, TODO and changelog docs.
|
|
|
|
Validation:
|
|
- `python -m pytest backend/tests/test_sprint30_workbench_components.py backend/tests/test_sprint27_frontend_workflow_hooks.py backend/tests/test_sprint19_map_workbench.py` passed: 9 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest` passed: 173 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed: 173 backend tests, frontend typecheck/build, Alembic head check and script syntax checks.
|
|
- `cd backend && python -m alembic heads` passed: single head `202606120900`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- Tower deploy via `scripts/deploy_tower.ps1` passed after commit `acf9590`.
|
|
- Tower live migration smoke passed with PostGIS 3.4 and Alembic head `202606120900`.
|
|
- Tower browser runtime verification passed on `http://192.168.10.150:1202`.
|
|
- Additional HTTP smoke passed for frontend HTML and `GET /api/v1/projects` through the frontend proxy.
|
|
|
|
Notes:
|
|
- No API contracts, backend behavior, migrations, product features, provider fetching, AI behavior or UI redesign changed.
|
|
- Next maintainability pass should split export preview and remaining shared workbench orchestration into focused components/hooks.
|
|
|
|
## Sprint 29 dataset component decomposition (2026-06-17)
|
|
|
|
Changed:
|
|
- Moved dataset upload/list rendering from `frontend/src/App.tsx` into `frontend/src/components/datasets/DatasetPanel.tsx`.
|
|
- Moved dataset detail and job-list rendering into `frontend/src/components/datasets/DatasetDetailPanel.tsx`.
|
|
- Split raster controls and vector controls into `frontend/src/components/datasets/RasterControls.tsx` and `frontend/src/components/datasets/VectorControls.tsx`.
|
|
- Updated Sprint 28 regression tests for the new component boundary and added Sprint 29 component wiring tests.
|
|
- Updated frontend README, TODO and changelog docs.
|
|
|
|
Validation:
|
|
- `python -m pytest backend/tests/test_sprint29_dataset_components.py backend/tests/test_sprint28_dataset_workflow_hook.py backend/tests/test_sprint27_frontend_workflow_hooks.py` passed: 10 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest` passed: 170 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed: 170 backend tests, frontend typecheck/build, Alembic head check and script syntax checks.
|
|
- `cd backend && python -m alembic heads` passed: single head `202606120900`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- Tower deploy via `scripts/deploy_tower.ps1` passed after commit `1cbb356`.
|
|
- Tower live migration smoke passed with PostGIS 3.4 and Alembic head `202606120900`.
|
|
- Tower browser runtime verification passed on `http://192.168.10.150:1202`.
|
|
- Additional HTTP smoke passed for frontend HTML and `GET /api/v1/projects` through the frontend proxy.
|
|
|
|
Notes:
|
|
- No API contracts, backend behavior, migrations, product features, provider fetching, AI behavior or UI redesign changed.
|
|
- Next maintainability pass should split change detection, QA/QC results and map workspace controls into focused presentational components.
|
|
|
|
## Sprint 28 dataset workflow hook hardening (2026-06-17)
|
|
|
|
Changed:
|
|
- Moved dataset selection, upload form state, dataset detail loading, dataset jobs and raster/vector operation orchestration from `frontend/src/App.tsx` into `frontend/src/hooks/useDatasetWorkflow.ts`.
|
|
- Kept project dataset listing in `App.tsx` so project/area loading remains the shared workbench boundary.
|
|
- Added regression tests to verify App uses the dataset workflow hook and still wires dataset, raster and vector callbacks.
|
|
- Updated frontend README, TODO and changelog docs.
|
|
|
|
Validation:
|
|
- `python -m pytest backend/tests/test_sprint28_dataset_workflow_hook.py backend/tests/test_sprint27_frontend_workflow_hooks.py backend/tests/test_sprint26_frontend_workflow_hooks.py` passed: 11 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest` passed: 167 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed: 167 backend tests, frontend typecheck/build, Alembic head check and script syntax checks.
|
|
- `cd backend && python -m alembic heads` passed: single head `202606120900`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- Initial Tower rebuild hit Docker btrfs storage exhaustion; safely reclaimed build cache only with `docker builder prune -af` without pruning volumes.
|
|
- Tower Docker storage recovered from `/var/lib/docker` 94% used to 58% used after cleanup and rebuild.
|
|
- Tower deploy via `scripts/deploy_tower.ps1` passed after commit `361776c`.
|
|
- Tower live migration smoke passed with PostGIS 3.4 and Alembic head `202606120900`.
|
|
- Tower browser runtime verification passed on `http://192.168.10.150:1202`.
|
|
- Additional HTTP smoke passed for frontend HTML and `GET /api/v1/projects` through the frontend proxy.
|
|
|
|
Notes:
|
|
- No API contracts, backend behavior, migrations, product features, provider fetching, AI behavior or UI redesign changed.
|
|
- Next maintainability pass should split dataset detail, raster controls and vector controls into presentational components fed by the hook state.
|
|
|
|
## Sprint 27 export and QA workflow hook hardening (2026-06-17)
|
|
|
|
Changed:
|
|
- Moved Export Center orchestration state and API calls from `frontend/src/App.tsx` into `frontend/src/hooks/useExportWorkflow.ts`.
|
|
- Moved QA/QC comparison state and persisted quality-check listing from `frontend/src/App.tsx` into `frontend/src/hooks/useQualityWorkflow.ts`.
|
|
- Added regression tests to verify App uses export/quality hooks and still wires QA/QC results plus ExportCenter callbacks.
|
|
- Updated frontend README, TODO and changelog docs.
|
|
|
|
Validation:
|
|
- `python -m pytest backend/tests/test_sprint27_frontend_workflow_hooks.py backend/tests/test_sprint26_frontend_workflow_hooks.py` passed: 8 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest` passed: 164 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed: 164 backend tests, frontend typecheck/build, Alembic head check and script syntax checks.
|
|
- `cd backend && python -m alembic heads` passed: single head `202606120900`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- Tower deploy via `scripts/deploy_tower.ps1` passed after commit `fd0000f`.
|
|
- Tower live migration smoke passed with PostGIS 3.4 and Alembic head `202606120900`.
|
|
- Tower browser runtime verification passed on `http://192.168.10.150:1202`.
|
|
- Additional HTTP smoke passed for frontend HTML and `GET /api/v1/projects` through the frontend proxy.
|
|
|
|
Notes:
|
|
- No API contracts, backend behavior, migrations, product features, provider fetching, AI behavior or UI redesign changed.
|
|
- Local Windows Docker CLI was unavailable (`docker` command not found); Tower deployment remains handled through `scripts/deploy_tower.ps1`.
|
|
- Next maintainability pass should extract dataset/raster/vector operation workflows from `App.tsx`.
|
|
|
|
## Sprint 26 frontend workflow hook hardening (2026-06-17)
|
|
|
|
Changed:
|
|
- Moved Detection Lab orchestration state and API calls from `frontend/src/App.tsx` into `frontend/src/hooks/useDetectionWorkflow.ts`.
|
|
- Moved Segmentation Lab orchestration state and API calls from `frontend/src/App.tsx` into `frontend/src/hooks/useSegmentationWorkflow.ts`.
|
|
- Added shared frontend `formatError` helper under `frontend/src/lib/formatError.ts`.
|
|
- Added regression tests to verify App uses workflow hooks and still wires DetectionLab/SegmentationLab callbacks.
|
|
- Updated frontend README, TODO and changelog docs.
|
|
|
|
Validation:
|
|
- `python -m pytest backend/tests/test_sprint26_frontend_workflow_hooks.py backend/tests/test_sprint22_workbench_status_strip.py` passed: 6 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed: 160 backend tests, frontend typecheck/build, Alembic head check and script syntax checks.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- Tower deploy/live migration/browser runtime passed after commit `6c32f29`.
|
|
- In-app browser check passed on `http://192.168.10.150:1202`: workbench, status strip, Detection Lab, Segmentation Lab and Export Center visible with no console error logs.
|
|
|
|
Notes:
|
|
- No API contracts, backend behavior, migrations, product features, provider fetching, AI behavior or UI redesign changed.
|
|
## Sprint 25 YOLO compatibility smoke hardening (2026-06-17)
|
|
|
|
Changed:
|
|
- Added explicit `--check-model-load` support to `scripts/yolo_preflight.py`, `backend/scripts/yolo_preflight.py` and `YoloPreflightService`.
|
|
- The model-load smoke requires real optional AI dependencies, loads only an existing local model file, runs no inference and does not download weights.
|
|
- The CLI rejects `--check-model-load` with `--assume-dependencies` to avoid false-positive AI readiness.
|
|
- Added regression tests for mocked successful load, load failure reporting and CLI guard behavior.
|
|
- Added Python compile validation for both YOLO preflight entrypoints to the readiness gate.
|
|
- Updated AI pipeline, scripts, backend, TODO and changelog docs, including Docker runtime usage.
|
|
|
|
Validation:
|
|
- `python -m py_compile scripts/yolo_preflight.py` passed.
|
|
- `python -m pytest backend/tests/test_sprint13_yolo_preflight.py backend/tests/test_readiness_gate.py` passed: 14 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed: 156 backend tests, frontend typecheck/build, Alembic head check and script syntax checks.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- Tower deploy/live migration/browser runtime passed after commit `6c32f29`.
|
|
- In-app browser check passed on `http://192.168.10.150:1202`: workbench, status strip, Detection Lab, Segmentation Lab and Export Center visible with no console error logs.
|
|
- Tower deploy/live migration/browser runtime passed after commit `382dcad`.
|
|
- Backend-container YOLO preflight sanity passed: default status `not_configured`, `will_download_models=false`, `will_run_inference=false`.
|
|
- Backend-container CLI guard passed: `--check-model-load` with `--assume-dependencies` is rejected.
|
|
|
|
Notes:
|
|
- No base dependencies, API contracts, migrations, product features, provider fetching or detection persistence behavior changed.
|
|
## Sprint 24 demo/export artifact cleanup tooling (2026-06-17)
|
|
|
|
Changed:
|
|
- Added `scripts/cleanup_demo_artifacts.py` for dry-run-first cleanup of old offline demo export artifacts.
|
|
- Added `backend/scripts/cleanup_demo_artifacts.py` so the same cleanup can run inside the backend Docker container.
|
|
- Cleanup is constrained to an exact demo project name by default, keeps the newest exports per project and refuses file deletion outside `STORAGE_ROOT`.
|
|
- Added regression tests for cleanup selection, path safety, dry-run candidate reporting and readiness gate coverage.
|
|
- Added Python compile validation for the cleanup script to `scripts/run_readiness_check.sh`.
|
|
- Documented cleanup usage in `scripts/README.md`, `docs/STORAGE_ARCHITECTURE.md`, `backend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Validation:
|
|
- `python -m py_compile scripts/cleanup_demo_artifacts.py` passed.
|
|
- `python -m pytest backend/tests/test_sprint24_cleanup_demo_artifacts.py backend/tests/test_readiness_gate.py` passed: 10 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed twice after adding the backend container entrypoint: 152 backend tests, frontend typecheck/build, Alembic head check and script syntax checks.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` and `bash -n scripts/verify_demo_export_workflow.sh` passed.
|
|
- Local `docker compose config` could not run because the Windows Docker CLI is not installed in this Codex environment.
|
|
- Tower deploy/live migration/browser runtime passed after final commit `2d4e6bd`.
|
|
- Live backend-container cleanup dry-run exposed and then fixed confusing dry-run labeling so candidates are reported as `candidate_files`, not `deleted_files`; final dry-run reported 2 candidate files, 0 deleted files and 0 deleted export rows.
|
|
|
|
Notes:
|
|
- No API contracts, migrations, product features, provider fetching, AI inference or source dataset cleanup behavior changed.
|
|
## Sprint 23 V1 report handoff summary (2026-06-17)
|
|
|
|
Changed:
|
|
- Added V1 readiness summary data to project metadata exports.
|
|
- Added V1 Readiness Summary and Known Limitations sections to lightweight HTML project report exports.
|
|
- Included persisted AOI, dataset readiness, QA/QC and export-history counts in the handoff summary.
|
|
- Updated export tests, API contract docs, frontend README, TODO and changelog.
|
|
|
|
Tested:
|
|
- Passed: backend compile, export tests through full readiness, full backend pytest, frontend typecheck/build, Alembic heads, Alembic SQL upgrade, live smoke syntax check, Tower deploy, live migration smoke and live project report smoke.
|
|
|
|
Known limitations:
|
|
- Report export remains a lightweight HTML artifact, not a PDF designer or custom report builder.
|
|
|
|
Next recommended pass:
|
|
- Add optional cleanup tooling for stale demo/export artifacts if repeated smoke runs keep accumulating local artifacts.
|
|
## Sprint 22 V1 workbench status strip (2026-06-17)
|
|
|
|
Changed:
|
|
- Added `frontend/src/components/WorkbenchStatusStrip.tsx` to summarize existing V1 state for project, AOI, datasets, active map layer, QA/QC and exports.
|
|
- Wired the status strip into `frontend/src/App.tsx` using existing orchestration state only.
|
|
- Added compact status-strip styling and regression tests for the frontend wiring contract.`n- Hardened `frontend/src/components/GeoMap.tsx` so MapLibre source/layer updates wait for style readiness before adding sources.`n- Hardened demo project lookup so duplicate historical demo projects prefer complete fixture state before repairing incomplete state.
|
|
- Updated frontend README, TODO and changelog.
|
|
|
|
Tested:
|
|
- Passed: backend compile, focused pytest, full backend pytest with DeprecationWarning as error, frontend typecheck/build, Alembic heads, Alembic SQL upgrade, readiness via Git Bash and live smoke syntax check via Git Bash.
|
|
|
|
Known limitations:
|
|
- The strip is a read-only operator summary; it intentionally does not add new backend status APIs or product workflows.
|
|
|
|
Next recommended pass:
|
|
- Add a compact project handoff summary in exports/report output if the browser-facing V1 workflow remains green.
|
|
|
|
## Sprint 21 V1 demo workflow smoke hardening (2026-06-17)
|
|
|
|
Changed:
|
|
- Hardened `scripts/verify_demo_export_workflow.sh` so the explicit offline demo smoke validates area GeoJSON, fixture datasets, vector FeatureCollection content, vector feature summaries, persisted QA/QC metrics and export downloads through the frontend proxy.
|
|
- Updated the frontend demo workflow action to open the candidate vector fixture dataset after seeding/loading the demo, so the Map Workbench is populated without a manual dataset click.
|
|
- Added regression tests for the strengthened smoke script and frontend demo loading contract.
|
|
- Updated scripts/frontend documentation, changelog and TODO status.
|
|
|
|
Tested:
|
|
- Passed: backend compile, backend pytest with DeprecationWarning as error, readiness, frontend typecheck/build, Alembic heads, Alembic SQL upgrade, Tower deploy, live migration smoke, browser-runtime smoke and expanded demo/export workflow smoke.
|
|
|
|
Known limitations:
|
|
- The demo smoke intentionally seeds fixture demo data when run; it should be used as an explicit verification command, not as an implicit healthcheck.
|
|
|
|
Next recommended pass:
|
|
- Add a compact V1 dashboard/status strip for project, AOI, datasets, QA and exports so operators can see readiness at a glance after opening a project.
|
|
## Sprint 20 V1 selected area map overlay (2026-06-17)
|
|
|
|
Changed:
|
|
- Added GeoJSON geometry serialization for project areas so persisted AOIs can be displayed by the map workbench.
|
|
- Added a dedicated MapLibre area overlay layer with separate visibility and opacity controls.
|
|
- Added area list actions and map workspace controls to select the active AOI.
|
|
- Updated API/frontend docs, changelog and TODO status for selected area display.
|
|
|
|
Tested:
|
|
- Passed: backend compile, backend pytest with DeprecationWarning as error, readiness, frontend typecheck/build, Alembic heads, Alembic SQL upgrade, Tower deploy, live migration smoke, browser-runtime smoke and browser UI audit.
|
|
|
|
Known limitations:
|
|
- Area geometry is displayed as a simple filled/outlined GeoJSON overlay; no drawing/editing workflow is introduced in this pass.
|
|
|
|
Next recommended pass:
|
|
- Add a small V1 workflow polish pass for richer dataset/area empty states and a fixture-driven end-to-end browser smoke once the new build is deployed.
|
|
## Sprint 19 V1 map workbench controls (2026-06-17)
|
|
|
|
Changed:
|
|
- Added active MapLibre layer visibility and opacity controls.
|
|
- Added click-to-inspect feature property display for the active GeoJSON workbench layer.
|
|
- Added active layer label and feature count to the Map workspace panel.
|
|
- Updated the app header from the stale Sprint 9 label to the GeoIntel Kempen V1 Workbench identity.
|
|
- Added regression tests for the frontend map control and feature inspection wiring.
|
|
|
|
Limitations:
|
|
- The current workbench still shows one active GeoJSON overlay at a time; multi-layer stack ordering remains a later UI enhancement.
|
|
- Raster preview display still remains metadata/path-oriented unless the backend exposes a browser-safe raster image/tile URL.
|
|
- No API contracts, migrations, backend behavior, provider fetching, AI inference or new dependencies were introduced.
|
|
|
|
Validation planned:
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` via Git Bash on Windows
|
|
- Tower redeploy through `scripts/deploy_tower.ps1`
|
|
## Sprint 18 vector change detection foundation (2026-06-16)
|
|
|
|
Changed:
|
|
- Added `POST /api/v1/analysis/change-detection` for comparing two vector datasets in the same project through the existing synchronous job envelope.
|
|
- Added `ChangeDetectionService` with persisted `vector_features` as the primary source of comparable geometries and explicit stored-GeoJSON fallback warnings for older datasets.
|
|
- Added frontend Change Detection controls, summary counts and MapLibre overlay styling for `added`, `removed` and `unchanged` feature properties.
|
|
- Added nginx no-cache headers for frontend HTML/assets after browser verification exposed stale cached modules on the LAN deployment.
|
|
- Added backend tests for persisted-vector comparison and canonical API envelope behavior.
|
|
|
|
Limitations:
|
|
- The foundation classifies `added`, `removed` and `unchanged` only. It does not emit fake `changed` objects without durable object ids/versioning.
|
|
- No migrations, live GRB/OSM/Sentinel fetching, AI inference, new dependencies, LiDAR, Copilot, Training Studio or separate Reports module were introduced.
|
|
|
|
Validation:
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` via Git Bash on Windows
|
|
- `cd backend && python -m alembic heads && python -m alembic upgrade head --sql`
|
|
# Codex Execution Log
|
|
|
|
This file must be updated by Codex after each implementation pass.
|
|
|
|
## Format
|
|
|
|
```md
|
|
## Pass X — Title
|
|
Date:
|
|
|
|
### Completed
|
|
- ...
|
|
|
|
### Files changed
|
|
- ...
|
|
|
|
### Tests run
|
|
- ...
|
|
|
|
### Known limitations
|
|
- ...
|
|
|
|
### Next recommended pass
|
|
- ...
|
|
```
|
|
|
|
## Initial status
|
|
Specification phase completed through M0. No code foundation has been implemented yet.
|
|
|
|
## M2 Engineering Package
|
|
|
|
- [x] Add ADR decision records.
|
|
- [x] Add RFC placeholders for future modules.
|
|
- [x] Add API/database/event contracts.
|
|
- [x] Add model registry and class catalogs.
|
|
- [x] Add queue architecture.
|
|
- [x] Add acceptance matrix and test catalog.
|
|
- [x] Add Codex M2 build prompts.
|
|
- [ ] Start Codex Pass 01 backend foundation.
|
|
|
|
## M8 preparation
|
|
|
|
Added the tomorrow execution layer for Codex autonomy: Day 1 master prompt, pass prompts, failure recovery, quality gates, operator checklist and smoke script scaffold.
|
|
|
|
## Sprint 1 readiness hardening (2026-06-11)
|
|
|
|
Date: 2026-06-11
|
|
|
|
### Completed
|
|
- Hardened backend dependency declarations for Sprint 1 runtime modules in `backend/pyproject.toml`.
|
|
- Normalized helper scripts for backend/frontend install/test/dev/readiness across `python`/`python3` interpreter availability.
|
|
- Fixed PostGIS/geospatial stack compatibility issues (`geojson_service` import path, package config).
|
|
- Updated backend/frontend/docs/readme commands for import smoke, setup, and readiness verification.
|
|
- Fixed frontend shell script line-ending parsing failures and added `npm` availability checks.
|
|
- Added missing frontend type path fixes and TypeScript typing corrections.
|
|
- Added/verified minimal Sprint 1 tests for health and GeoJSON parsing/rejection behaviors.
|
|
|
|
### Files changed
|
|
- `backend/pyproject.toml`
|
|
- `backend/app/core/config.py`
|
|
- `backend/app/services/geojson_service.py`
|
|
- `backend/tests/test_health.py`
|
|
- `backend/tests/test_geojson_dataset_service.py`
|
|
- `backend/README.md`
|
|
- `frontend/package.json`
|
|
- `frontend/tsconfig.json`
|
|
- `frontend/src/components/GeoMap.tsx`
|
|
- `frontend/src/services/api/areas.ts`
|
|
- `frontend/src/services/api/projects.ts`
|
|
- `frontend/src/services/api/datasets.ts`
|
|
- `frontend/src/App.tsx`
|
|
- `frontend/README.md`
|
|
- `scripts/run_readiness_check.sh`
|
|
- `scripts/backend_install.sh`
|
|
- `scripts/backend_test.sh`
|
|
- `scripts/backend_dev.sh`
|
|
- `scripts/codex_pass_end_check.sh`
|
|
- `scripts/smoke_backend_import.sh`
|
|
- `scripts/frontend_install.sh`
|
|
- `scripts/frontend_typecheck.sh`
|
|
- `scripts/frontend_build.sh`
|
|
- `scripts/frontend_dev.sh`
|
|
- `README.md`
|
|
- `.env.example`
|
|
- `CHANGELOG.md`
|
|
- `Makefile`
|
|
|
|
### Tests run
|
|
- `python -m compileall backend/app` (pass)
|
|
- `cd backend; python -m pytest` (pass, 5 tests)
|
|
- `cd backend; python -c "from app.main import app; print(app.title)"` (pass)
|
|
- `bash scripts/run_readiness_check.sh` (pass)
|
|
- `bash scripts/smoke_backend_import.sh` (pass)
|
|
- `bash scripts/frontend_install.sh` (pass)
|
|
- `bash scripts/frontend_typecheck.sh` (pass)
|
|
- `bash scripts/frontend_build.sh` (pass)
|
|
- `make`-based targets not runnable in this environment (`make` command missing)
|
|
- `docker compose config` not runnable in this environment (`docker` command missing)
|
|
- `python3` commands fail in this environment because `python3` maps to Microsoft Store stub; use `python` instead.
|
|
|
|
### Known limitations
|
|
- `docker` and `make` are not installed in the current local shell environment.
|
|
- `python3` is not a usable interpreter in this environment; `python` must be used for all backend runtime/tests.
|
|
|
|
### Next recommended pass
|
|
- Run `docker compose` validation and DB-backed migration checks in an environment with Docker + PostGIS service available.
|
|
|
|
## Pass 12 — Sprint 2 Foundation
|
|
Date: 2026-06-11
|
|
|
|
### Completed
|
|
- Added vector/raster dataset typing and lifecycle states (`uploaded`, `validating`, `ready`, `failed`) in dataset service.
|
|
- Added vector metadata extraction details (feature counts, geometry types, bounds, area, CRS assumptions).
|
|
- Added raster metadata service with dependency-aware unavailable behavior and explicit `RASTER_PROCESSING_UNAVAILABLE` handling.
|
|
- Added dataset vector inspect/summary and raster metadata endpoints for project-scoped datasets.
|
|
- Persisted deterministic storage metadata for uploads (original/stored filename, MIME, size, checksum).
|
|
- Extended frontend dataset manager details panel with type/status/file metadata/feature counts and raster summary readiness.
|
|
- Added minimal Sprint 2 tests for vector metadata, legacy `geojson` compatibility, storage metadata persistence, and raster dependency fallback.
|
|
|
|
### Files changed
|
|
- `backend/app/services/dataset_service.py`
|
|
- `backend/app/services/raster_service.py`
|
|
- `backend/app/services/geojson_service.py`
|
|
- `backend/app/schemas/dataset.py`
|
|
- `backend/app/api/routes/datasets.py`
|
|
- `backend/app/services/storage_service.py`
|
|
- `backend/tests/test_geojson_dataset_service.py`
|
|
- `backend/tests/test_raster_service.py`
|
|
- `backend/tests/test_storage_service.py`
|
|
- `backend/README.md`
|
|
- `frontend/src/App.tsx`
|
|
- `frontend/src/services/api/datasets.ts`
|
|
- `frontend/src/types.ts`
|
|
- `docs/API_CONTRACTS.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
- `.env.example`
|
|
- `backend/README.md`
|
|
- `frontend/README.md`
|
|
- `CHANGELOG.md`
|
|
- `README.md`
|
|
|
|
### Tests run
|
|
- Sprint 2 tests to be executed via readiness commands (see below).
|
|
|
|
### Known limitations
|
|
- `rasterio` is not required by default; raster metadata extraction is unavailable until installed in the environment.
|
|
- Docker and PostGIS validation still depends on local availability of Docker/DB runtime.
|
|
|
|
### Next recommended pass
|
|
- Keep Sprint 1/2 architecture; implement raster clipping/tiling APIs and status jobs before Detection/Segmentation modules.
|
|
|
|
## Pass 13 — Sprint 2 validation and legacy compatibility hardening
|
|
Date: 2026-06-11
|
|
|
|
### Completed
|
|
- Verified Sprint 2 readiness commands in current environment (where tools are available).
|
|
- Revalidated backend tests and frontend typecheck/build after a Sprint 2 compatibility patch.
|
|
- Added frontend vector-detail compatibility for legacy `geojson` dataset rows in Sprint 1 records.
|
|
- Confirmed backend import smoke and dataset/raster/vector service behavior remain intact.
|
|
|
|
### Files changed
|
|
- `frontend/src/App.tsx`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Tests run
|
|
- `python -m compileall backend/app` (pass)
|
|
- `cd backend && python -m pytest` (pass, 11 tests)
|
|
- `bash scripts/run_readiness_check.sh` (pass)
|
|
- `bash scripts/smoke_backend_import.sh` (pass)
|
|
- `cd frontend && npm run typecheck` (pass)
|
|
- `cd frontend && npm run build` (pass)
|
|
- `cd backend; python3 -m compileall backend/app` (fails: python3 shim unavailable in this shell)
|
|
- `cd backend; python3 -c "from app.main import app; print(app.title)"` (fails: python3 shim unavailable in this shell)
|
|
- `docker compose config` (fails: docker command unavailable in this shell)
|
|
|
|
### Known limitations
|
|
- `python3` is not available in the current environment; use `python` commands for verification.
|
|
- `docker` is not available in the current environment.
|
|
|
|
### Next recommended pass
|
|
- Run the same sprint verification commands in an environment with `python3` and `docker` installed.
|
|
|
|
## Pass 14 — Sprint 3 operations + job foundation
|
|
Date: 2026-06-11
|
|
|
|
### Completed
|
|
- Added lightweight job model/migration and service layer with statuses `queued`, `running`, `success`, `failed`.
|
|
- Added job API endpoints for create/list/read/status under project scope.
|
|
- Added vector operation service foundation:
|
|
- inspect
|
|
- bbox
|
|
- stats
|
|
- clip by area
|
|
- buffer
|
|
- intersect
|
|
- Added raster operation foundation:
|
|
- inspect
|
|
- metadata
|
|
- preview readiness
|
|
- clip placeholder (dependency-aware)
|
|
- tile placeholder (dependency-aware)
|
|
- Added job-based execution wrappers for vector/raster operation endpoints.
|
|
- Updated dataset detail UI with available operations, job list/status, and derived output navigation.
|
|
- Added/updated Sprint 3 API contract and backend/frontend documentation updates.
|
|
|
|
### Files changed
|
|
- `backend/app/schemas/operations.py`
|
|
- `backend/app/schemas/__init__.py`
|
|
- `backend/app/services/vector_operations_service.py`
|
|
- `backend/app/services/raster_operations_service.py`
|
|
- `backend/app/services/job_service.py`
|
|
- `backend/app/api/routes/jobs.py`
|
|
- `backend/app/models/entities.py`
|
|
- `backend/app/models/__init__.py`
|
|
- `backend/alembic/versions/20260611212435_add_jobs_table.py`
|
|
- `backend/tests/test_vector_operations_service.py`
|
|
- `backend/tests/test_raster_operations_service.py`
|
|
- `backend/tests/test_storage_service.py`
|
|
- `frontend/src/App.tsx`
|
|
- `frontend/src/services/api/datasets.ts`
|
|
- `frontend/src/services/api/jobs.ts`
|
|
- `frontend/src/types.ts`
|
|
- `backend/README.md`
|
|
- `frontend/README.md`
|
|
- `docs/API_CONTRACTS.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
- `CHANGELOG.md`
|
|
|
|
### Tests run
|
|
- `python -m compileall backend/app` (pass)
|
|
- `cd backend && python -m pytest` (pass)
|
|
- `bash scripts/run_readiness_check.sh` (pass)
|
|
- `bash scripts/smoke_backend_import.sh` (pass)
|
|
- `cd frontend && npm run typecheck` (pass)
|
|
- `cd frontend && npm run build` (pass)
|
|
|
|
### Known limitations
|
|
- Raster processing beyond readiness/metadata is intentionally dependency-aware placeholder in Sprint 3.
|
|
- `docker` remains unavailable in this environment.
|
|
- `python3` still resolves to Windows Store stub; use `python` for verification.
|
|
|
|
### Next recommended pass
|
|
- Install rasterio/GDAL for real raster clipping/tile implementations.
|
|
- Add queue-backed worker (Redis/RQ) behind existing synchronous job facade.
|
|
- Add deeper operation acceptance tests for output dataset geometry correctness and persisted metadata.
|
|
|
|
## Pass 15 — Sprint 4 raster foundation
|
|
Date: 2026-06-11
|
|
|
|
### Completed
|
|
- Finalized dependency-aware raster processing behavior for clip/tile/preview and improved metadata enrichment.
|
|
- Ensured raster metadata and preview responses include deterministic file metadata (`size_bytes`, `checksum_sha256`, `path`) where available.
|
|
- Added raster tile manifest generation with deterministic storage paths and returned manifest payload.
|
|
- Updated API contracts and execution documentation from placeholder wording to actual raster-op foundations.
|
|
- Added targeted Sprint 4 tests for missing-area clip behavior and tile manifest payload shape.
|
|
|
|
### Files changed
|
|
- `backend/app/services/raster_operations_service.py`
|
|
- `backend/app/api/routes/health.py`
|
|
- `backend/tests/test_raster_operations_service.py`
|
|
- `docs/API_CONTRACTS.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
- `backend/README.md`
|
|
- `frontend/README.md`
|
|
- `README.md`
|
|
- `CHANGELOG.md`
|
|
|
|
### Tests run
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest`
|
|
- `bash scripts/run_readiness_check.sh`
|
|
- `bash scripts/smoke_backend_import.sh`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
|
|
### Known limitations
|
|
- Raster dependency packages remain optional and will report `RASTER_PROCESSING_UNAVAILABLE` if absent.
|
|
- Raster clip/tile remain synchronous under the current in-process job wrapper.
|
|
|
|
### Next recommended pass
|
|
- Run full raster end-to-end tests with real GeoTIFF fixtures and validate output dataset metadata persistence.
|
|
## Pass 16 � Sprint 4 raster operations foundation hardening
|
|
Date: 2026-06-11
|
|
|
|
### Completed
|
|
- Realized the Sprint 4 raster foundation readiness in implementation and docs alignment:
|
|
- metadata extraction now returns real raster profile fields when rasterio is available
|
|
- preview generation writes deterministic PNG artifacts and reuses cached previews
|
|
- clip and tile operations persist deterministic outputs and manifest structure
|
|
- all raster processing paths now use explicit dependency-aware errors when rasterio/numpy/pillow are missing
|
|
- Fixed remaining frontend render/type issues introduced during raster path handling.
|
|
- Strengthened raster tests for tile manifest minimum size and dependency-aware behavior.
|
|
- Updated sprint milestone docs to reflect Sprint 4 status:
|
|
- `backend/README.md`
|
|
- `frontend/README.md`
|
|
|
|
### Files changed
|
|
- `backend/app/services/raster_operations_service.py`
|
|
- `backend/tests/test_raster_operations_service.py`
|
|
- `backend/app/api/routes/datasets.py`
|
|
- `backend/app/schemas/operations.py`
|
|
- `frontend/src/App.tsx`
|
|
- `backend/README.md`
|
|
- `frontend/README.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Tests run
|
|
- `python -m compileall backend/app` (pass)
|
|
- `cd backend && python -m pytest` (pass, 26 tests)
|
|
- `bash scripts/run_readiness_check.sh` (pass)
|
|
- `bash scripts/smoke_backend_import.sh` (pass)
|
|
- `cd frontend && npm run typecheck` (pass)
|
|
- `cd frontend && npm run build` (pass)
|
|
|
|
### Known limitations
|
|
- `docker` command is unavailable in this environment, so `docker compose config` could not be executed.
|
|
- `python3` command is unavailable as an alias in this environment; use `python` instead.
|
|
- Full raster dependency stack may be unavailable in some dev environments; those cases intentionally return structured `RASTER_PROCESSING_UNAVAILABLE` responses.
|
|
|
|
### Next recommended pass
|
|
- Sprint 5: add end-to-end RasterOps coverage for CRS-preserving re-projection, export-quality metadata policy, and tile set artifact cleanup lifecycle.
|
|
## Pass 17 — Sprint 5 raster analytics hardening
|
|
|
|
Date: 2026-06-11
|
|
|
|
### Completed
|
|
- Implemented Sprint 5 raster analytics foundation:
|
|
- band statistics endpoint and service with dependency-aware unavailable mode.
|
|
- reproject operation with CRS validation and reprojection metadata persistence.
|
|
- clip/tile hardening for missing/invalid geometry and empty clip output handling.
|
|
- enriched tile manifest fields (`tile_set_id`, `tile_size`, `overlap`, `source_dataset_id`, `source_raster_id`, `bounds`, `count`, `tile_paths`, `ai_inference`, `tile_server`, `created_at`, parameters).
|
|
- Updated raster stats/job error tests:
|
|
- dependency-aware stats failure coverage.
|
|
- invalid CRS request validation for reproject.
|
|
- persisted derived dataset assertions for reproject and clip.
|
|
- failure persistence test for failed raster jobs (`_run_job_sync`).
|
|
- Strengthened raster frontend detail panel rendering:
|
|
- job result JSON is visible for raster/vector operations.
|
|
- clearer raster metadata/status visibility retained for CRS/bounds/resolution display.
|
|
- Updated docs:
|
|
- backend/README Sprint 5 section.
|
|
- frontend/README Sprint 5 section.
|
|
- CHANGELOG entry for Sprint 5.
|
|
|
|
### Files changed
|
|
- `backend/app/services/raster_operations_service.py` (final reproject and manifest hardening alignment)
|
|
- `backend/tests/test_raster_operations_service.py`
|
|
- `frontend/src/App.tsx`
|
|
- `backend/README.md`
|
|
- `frontend/README.md`
|
|
- `docs/API_CONTRACTS.md`
|
|
- `docs/RASTER_OPERATIONS_SPEC.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
- `CHANGELOG.md`
|
|
|
|
### Tests run
|
|
- `python -m compileall backend/app` (pass)
|
|
- `cd backend && python -m pytest` (pass)
|
|
- `bash scripts/run_readiness_check.sh` (pass)
|
|
- `bash scripts/smoke_backend_import.sh` (pass)
|
|
- `cd frontend && npm run typecheck` (pass)
|
|
- `cd frontend && npm run build` (pass)
|
|
|
|
### Known limitations
|
|
- Raster statistics and reproject operations still depend on environment availability of `rasterio`/`numpy`.
|
|
- Raster dependency checks and operation errors remain dependency-aware when libraries are unavailable.
|
|
|
|
### Next recommended pass
|
|
- Sprint 6: add raster index/mask workflows (NDVI/NDWI/NDBI), tile serving or export packaging, and AI-ready dataset linking.
|
|
## Pass 18 - Sprint 6 spectral indices
|
|
|
|
### Completed
|
|
- Added local raster index operations (`ndvi`, `ndwi`, `ndbi`) under the existing raster operation architecture.
|
|
- Added typed band payload validation and explicit `INVALID_PARAMETERS` handling for missing/invalid band indexes.
|
|
- Implemented dependency-aware failure behavior for index execution (`RASTER_PROCESSING_UNAVAILABLE`) when `rasterio` or `numpy` are unavailable.
|
|
- Implemented local index output generation with float32 raster derivation and persisted provenance metadata on derived datasets.
|
|
- Extended dataset detail UI with spectral index controls (NDVI/NDWI/NDBI), run actions, and output dataset navigation from jobs.
|
|
- Updated API contracts, raster operation spec, and project documentation for Sprint 6 behavior.
|
|
|
|
### Files changed
|
|
- `backend/app/schemas/operations.py`
|
|
- `backend/app/services/raster_operations_service.py`
|
|
- `backend/app/api/routes/datasets.py`
|
|
- `backend/tests/test_raster_operations_service.py`
|
|
- `frontend/src/services/api/datasets.ts`
|
|
- `frontend/src/App.tsx`
|
|
- `frontend/src/types.ts`
|
|
- `docs/API_CONTRACTS.md`
|
|
- `docs/RASTER_OPERATIONS_SPEC.md`
|
|
- `backend/README.md`
|
|
- `frontend/README.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Tests run
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest`
|
|
- `bash scripts/run_readiness_check.sh`
|
|
- `bash scripts/smoke_backend_import.sh`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
|
|
### Known limitations
|
|
- Raster dependency checks remain optional; missing raster packages return explicit `RASTER_PROCESSING_UNAVAILABLE` results.
|
|
- Job execution remains synchronous under current Sprint 3 job facade.
|
|
- `docker` and `python3` availability still depend on developer environment.
|
|
|
|
### Next recommended pass
|
|
- Keep operation architecture stable, then add threshold/mask workflows and output export packaging in a follow-up pass.
|
|
|
|
|
|
## Pass 19 - RC-2 stabilization
|
|
Date: 2026-06-12
|
|
|
|
### Completed
|
|
- Fixed backend compile/import blockers identified in RC-1:
|
|
- corrected dataset upload parameter ordering.
|
|
- corrected schema package re-exports for area schemas.
|
|
- Fixed frontend typecheck/build blockers in `App.tsx`.
|
|
- Added Alembic migration for dataset reference/provenance metadata columns required by current ORM models.
|
|
- Fixed QA comparison runtime crash and added focused QA service coverage.
|
|
- Corrected envelope response-model mismatches for vector inspect and raster stats endpoints.
|
|
- Strengthened `scripts/run_readiness_check.sh` so readiness runs backend compile, backend tests, frontend typecheck and frontend build.
|
|
- Improved readiness Python interpreter selection so it chooses an interpreter capable of running pytest.
|
|
|
|
### Files changed
|
|
- `backend/app/services/dataset_service.py`
|
|
- `backend/app/schemas/__init__.py`
|
|
- `backend/app/api/routes/datasets.py`
|
|
- `backend/app/services/qa_service.py`
|
|
- `backend/alembic/versions/202606120001_add_dataset_reference_metadata.py`
|
|
- `backend/tests/test_qa_service.py`
|
|
- `frontend/src/App.tsx`
|
|
- `scripts/run_readiness_check.sh`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Tests run
|
|
- `python -m compileall backend\\app` (pass)
|
|
- `cd backend && python -m pytest` (pass, 40 tests)
|
|
- `cd frontend && npm run typecheck` (pass)
|
|
- `cd frontend && npm run build` (pass; bundle-size warning only)
|
|
- `bash scripts/run_readiness_check.sh` (pass)
|
|
- `cd backend && python -m alembic heads` (pass, single head `202606120001`)
|
|
- `cd backend && python -m alembic upgrade head --sql` (pass, generated SQL includes required dataset metadata columns)
|
|
- `docker compose config` not runnable in this environment because `docker` is not installed.
|
|
|
|
### Known limitations
|
|
- Fresh online database migration was validated only as generated Alembic SQL in this environment; run `cd backend && python -m alembic upgrade head` against a live PostgreSQL/PostGIS database on a machine with Docker or Postgres available.
|
|
- Existing non-RC architecture limitations from RC-1 remain intentionally unfixed: synchronous job facade, file-first vector outputs, monolithic frontend component, and incomplete future AI/storage architecture.
|
|
|
|
### Next recommended pass
|
|
- Do not start Sprint 7 until RC-2 verification is repeated against a live Docker/PostGIS environment.
|
|
|
|
## Pass 20 - Sprint 7A persistence and QA foundation
|
|
Date: 2026-06-12
|
|
|
|
### Completed
|
|
- Added first-class `vector_features` ORM model and Alembic migration with dataset and GiST geometry indexes.
|
|
- Persisted uploaded vector GeoJSON features into PostGIS-backed `vector_features` while keeping original file storage intact.
|
|
- Added first-class `quality_checks` and `metrics` ORM models and Alembic migration indexes.
|
|
- Added `QualityService` for persisted QA/QC domain records and metric rows.
|
|
- Updated QA candidate-vs-reference route so successful QA jobs also persist a `QualityCheck` and metrics, and return `quality_check_id` in `result_json`.
|
|
- Hardened provider capability contracts for GRB and OSM as `not_configured` stubs with supported layers, geometry types and query modes.
|
|
- Added Sprint 7A tests for vector feature persistence, quality check persistence, metrics persistence, dataset role validation, provider contracts, migration integrity and QA route persistence.
|
|
- Updated database/API documentation for Vector Features Architecture, Quality Check Architecture, Metrics Architecture and Provider Architecture.
|
|
|
|
### Files changed
|
|
- `backend/app/models/entities.py`
|
|
- `backend/app/models/__init__.py`
|
|
- `backend/app/services/dataset_service.py`
|
|
- `backend/app/services/vector_feature_service.py`
|
|
- `backend/app/services/quality_service.py`
|
|
- `backend/app/api/routes/qa.py`
|
|
- `backend/app/providers/base.py`
|
|
- `backend/app/providers/grb.py`
|
|
- `backend/app/providers/osm.py`
|
|
- `backend/app/schemas/health.py`
|
|
- `backend/alembic/versions/202606120700_sprint7a_persistence_foundation.py`
|
|
- `backend/tests/test_sprint7a_persistence_foundation.py`
|
|
- `docs/DATABASE_IMPLEMENTATION_PLAN.md`
|
|
- `docs/API_CONTRACTS.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
- `CHANGELOG.md`
|
|
|
|
### Tests run
|
|
- `python -m compileall backend/app` (pass)
|
|
- `python -m pytest backend -q` (pass, 47 tests)
|
|
|
|
### Known limitations
|
|
- Sprint 7A intentionally does not implement GRB downloads, OSM downloads, Detection Lab, Segmentation Lab, LiDAR, AI Copilot, Training Studio or Reports.
|
|
- Job execution remains the existing synchronous facade.
|
|
- Live database migration still needs validation against a running PostgreSQL/PostGIS service in an environment with Docker or Postgres available.
|
|
|
|
### Next recommended pass
|
|
- Complete full release validation commands, including Alembic heads/SQL generation, readiness script, frontend typecheck/build and Docker config if Docker is available.
|
|
|
|
### Validation addendum
|
|
Date: 2026-06-12
|
|
|
|
Additional Sprint 7A validation completed after migration index cleanup:
|
|
|
|
- `python -m compileall backend/app` (pass)
|
|
- `cd backend && python -m pytest` (pass, 47 tests)
|
|
- `bash scripts/run_readiness_check.sh` (pass)
|
|
- `cd frontend && npm run typecheck` (pass)
|
|
- `cd frontend && npm run build` (pass; Vite chunk-size warning only)
|
|
- `cd backend && python -m alembic heads` (pass, single head `202606120700`)
|
|
- `cd backend && python -m alembic upgrade head --sql` (pass; generated SQL includes `vector_features`, `quality_checks`, `metrics` and the named GiST index `ix_vector_features_geometry`)
|
|
- `docker compose config` could not run because Docker is not installed in this shell.
|
|
|
|
## Pass 21 - Sprint 7B provider integration skeleton (2026-06-12)
|
|
|
|
- Implemented central provider registry for `grb`, `osm`, `manual` and `fixture`.
|
|
- Added provider capability, layer, status and future import-contract endpoints using the existing response envelope style.
|
|
- Preserved GRB and OSM as explicit `not_configured` providers; no live WFS, Overpass, download or fake provider data was introduced.
|
|
- Documented and tested provider-to-dataset mapping rules; future provider output must flow through `DatasetService` / `VectorFeatureService` rather than direct `vector_features` writes.
|
|
- Added frontend Provider Capabilities panel without live import buttons for GRB/OSM.
|
|
- Added opt-in `scripts/live_migration_smoke.sh` for real PostGIS migration smoke checks.
|
|
- Added Sprint 7B tests for provider registry, API envelopes, invalid provider handling, import contract and smoke script presence.
|
|
|
|
## Pass 22 - Sprint 8 Detection Lab foundation (2026-06-12)
|
|
|
|
### Completed
|
|
- Added first-class `detections` ORM model and Alembic migration with project, dataset, analysis run, class and GiST geometry indexes.
|
|
- Hardened `analysis_runs` with dataset, job, model, result and created-at fields while keeping jobs conceptually separate from analysis lifecycle.
|
|
- Added model registry capability service for `yolo-placeholder` (`not_configured`) and `manual-fixture-detector` (explicit fixture/demo only).
|
|
- Added `DetectionService` boundary for model listing, request validation, analysis run creation, job creation, unavailable model responses and explicit fixture persistence.
|
|
- Added Detection Lab API endpoints under `/api/v1/detection` using the existing response envelope style.
|
|
- Added minimal frontend Detection Lab panel for model capability status, raster dataset selection, confidence threshold and run result/error display.
|
|
- Updated database, API, AI pipeline, backend/frontend README, TODO and changelog docs.
|
|
|
|
### Known limitations
|
|
- Real YOLO/PyTorch inference is not enabled and no model downloads are performed.
|
|
- Fixture detector requires `fixture_mode=true` and explicit fixture detections; it is not production inference.
|
|
- Segmentation, LiDAR, AI Copilot, Training Studio and Reports remain out of scope.
|
|
|
|
### Next recommended pass
|
|
- Verify Sprint 8 with full backend/frontend/readiness/Alembic gates, then perform a Sprint 8 verification audit before Sprint 8B real YOLO integration.
|
|
|
|
## Pass 23 - Sprint 8B configured YOLO foundation (2026-06-12)
|
|
|
|
### Completed
|
|
- Added optional backend `ai` dependency group for `ultralytics` and `torch`; normal backend startup remains import-safe without those packages.
|
|
- Added YOLO configuration settings:
|
|
- `YOLO_ENABLED`
|
|
- `YOLO_MODEL_PATH`
|
|
- `YOLO_MODEL_ID`
|
|
- `YOLO_MODEL_DISPLAY_NAME`
|
|
- `YOLO_MODEL_VERSION`
|
|
- `YOLO_DEVICE`
|
|
- `YOLO_IMAGE_SIZE`
|
|
- `YOLO_MAX_TILES`
|
|
- `YOLO_BATCH_SIZE`
|
|
- Added `yolo-configured` model registry capability with honest `not_configured`, `dependency_unavailable` and `configured` states.
|
|
- Added `YoloDetectionAdapter` that imports Ultralytics only in the load path and refuses missing local model files before model construction.
|
|
- Added raster tile manifest validation and configured tile-limit enforcement for real YOLO runs.
|
|
- Added pixel bbox to EPSG:4326 polygon georeferencing from tile transform or bounds metadata.
|
|
- Routed configured YOLO outputs through existing `DetectionService`, `Job`, `AnalysisRun` and first-class `Detection` persistence.
|
|
- Added Detection Lab tile manifest path input for the configured YOLO model.
|
|
- Added mocked Sprint 8B tests for model registry status, dependency-unavailable behavior, tile manifest validation, georeferencing and persisted detections.
|
|
- Updated API, AI pipeline, backend/frontend README and changelog documentation.
|
|
|
|
### Known limitations
|
|
- Sprint 8B does not add workers/queues; configured YOLO runs remain synchronous behind the existing job abstraction.
|
|
- Real model loading is validated at execution time. The registry reports configured when dependencies and local model path are present.
|
|
- No model weights are downloaded by GeoIntel.
|
|
- Detection visualization/map overlays are deferred.
|
|
- Segmentation, LiDAR, AI Copilot, Training Studio and Reports remain out of scope.
|
|
|
|
### Next recommended pass
|
|
- Run full Sprint 8B validation and then perform a Sprint 8B verification audit before advancing to detection visualization/QA or segmentation planning.
|
|
|
|
## Pass 24 - Sprint 8C detection visualization and QA integration (2026-06-12)
|
|
|
|
### Completed
|
|
- Added detection result review endpoints for listing runs, listing detections by run/dataset, retrieving detection detail and returning persisted detections as GeoJSON FeatureCollections.
|
|
- Added lightweight detection filters for class name and minimum confidence.
|
|
- Added detection QA against persisted reference `vector_features` using the existing `QualityService`, `quality_checks` and `metrics` persistence path.
|
|
- Added frontend Detection Lab run selection, detection table, class/confidence filters and MapLibre detection GeoJSON overlay via the existing map component.
|
|
- Added frontend detection QA controls and metric summary display.
|
|
- Added Sprint 8C tests for GeoJSON output, list/filter behavior, detection detail, API envelope shape, QA persistence and no-match QA behavior.
|
|
- Added direct Sprint 8B tests for missing and invalid tile manifest files.
|
|
- Updated API, AI pipeline, backend/frontend README and changelog documentation.
|
|
|
|
### Known limitations
|
|
- Detection QA requires reference datasets to have persisted `vector_features`; unsupported references return a clear error instead of fake metrics.
|
|
- Detection overlays reuse the existing single GeoJSON map layer styling; complex class-based map styling is deferred.
|
|
- Segmentation, LiDAR, AI Copilot, Training Studio and Reports remain out of scope.
|
|
|
|
### Next recommended pass
|
|
- Run full Sprint 8C validation and perform a Sprint 8C verification audit before starting Sprint 9 Segmentation Lab.
|
|
|
|
## Pass 25 - Sprint 9 Segmentation Lab foundation (2026-06-12)
|
|
|
|
### Completed
|
|
- Added first-class `segmentations` ORM model and Alembic migration with project, dataset, job, analysis run, class and GiST geometry indexes.
|
|
- Added segmentation model registry capabilities for `segmentation-placeholder`, `fixture-segmenter`, `yolo-seg-configured` and `sam-configured`.
|
|
- Added `SegmentationService` boundary for raster validation, job/analysis-run lifecycle, unavailable model responses and explicit fixture-only persistence.
|
|
- Added segmentation adapter placeholder module with no SAM, YOLO-seg, torch or ultralytics imports.
|
|
- Added persisted segmentation GeoJSON output generated from PostGIS geometry and provenance properties.
|
|
- Added segmentation QA against persisted reference `vector_features` using existing `quality_checks` and `metrics`.
|
|
- Added minimal frontend Segmentation Lab panel for model states, raster selection, runs/results, GeoJSON map overlay and QA metric display.
|
|
- Updated API, AI pipeline, storage, database, backend/frontend README and changelog documentation.
|
|
|
|
### Files changed
|
|
- `backend/app/models/entities.py`
|
|
- `backend/app/models/__init__.py`
|
|
- `backend/alembic/versions/202606120900_sprint9_segmentation_foundation.py`
|
|
- `backend/app/schemas/segmentation.py`
|
|
- `backend/app/schemas/__init__.py`
|
|
- `backend/app/services/model_registry_service.py`
|
|
- `backend/app/services/segmentation_adapter.py`
|
|
- `backend/app/services/segmentation_service.py`
|
|
- `backend/app/api/routes/segmentation.py`
|
|
- `backend/app/main.py`
|
|
- `backend/tests/test_sprint9_segmentation_foundation.py`
|
|
- `frontend/src/types.ts`
|
|
- `frontend/src/services/api/segmentation.ts`
|
|
- `frontend/src/services/api/index.ts`
|
|
- `frontend/src/App.tsx`
|
|
- `docs/API_CONTRACTS.md`
|
|
- `docs/AI_PIPELINES.md`
|
|
- `docs/STORAGE_ARCHITECTURE.md`
|
|
- `docs/DATABASE_IMPLEMENTATION_PLAN.md`
|
|
- `backend/README.md`
|
|
- `frontend/README.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Tests run
|
|
- `python -m pytest backend/tests/test_sprint9_segmentation_foundation.py -q` (red first: missing `Segmentation` import, then pass)
|
|
- `cd backend && python -m pytest` (pass, 88 tests)
|
|
- `cd frontend && npm run typecheck` (pass)
|
|
|
|
### Known limitations
|
|
- Sprint 9 intentionally does not implement real SAM, real YOLO-seg, model downloads, new AI dependencies or production-scale async inference.
|
|
- Fixture segmenter requires explicit `fixture_mode=true` and explicit fixture segmentations; it is not production inference.
|
|
- Metric area is only persisted when provided by the fixture/output payload; Sprint 9 does not compute authoritative area from masks.
|
|
|
|
### Next recommended pass
|
|
- Run full Sprint 9 validation and then perform Sprint 9 Verification Audit before considering future real SAM/YOLO-seg integration.
|
|
|
|
### Validation addendum
|
|
- `python -m compileall backend/app` (pass)
|
|
- `cd backend && python -m pytest` (pass, 88 tests)
|
|
- `bash scripts/run_readiness_check.sh` (pass)
|
|
- `cd frontend && npm run typecheck` (pass)
|
|
- `cd frontend && npm run build` (pass; existing Vite chunk-size warning only)
|
|
- `cd backend && python -m alembic heads` (pass, single head `202606120900`)
|
|
- `cd backend && python -m alembic upgrade head --sql` (pass; generated SQL includes `segmentations` and GiST index)
|
|
- `bash -n scripts/live_migration_smoke.sh` (pass)
|
|
- `docker compose config` could not run because Docker is not installed in this shell.
|
|
|
|
## Pass 26 - Sprint 10 release hardening and frontend modularization (2026-06-13)
|
|
|
|
### Completed
|
|
- Extracted Provider Capabilities, Detection Lab and Segmentation Lab sections from `frontend/src/App.tsx` into focused frontend components.
|
|
- Preserved existing workbench state ownership, API client calls, map overlay behavior and UI copy.
|
|
- Hardened `scripts/run_readiness_check.sh` with Alembic head verification and live migration smoke script syntax validation.
|
|
- Updated frontend README and changelog documentation for Sprint 10 maintainability work.
|
|
|
|
### Files changed
|
|
- `frontend/src/App.tsx`
|
|
- `frontend/src/components/providers/ProviderPanel.tsx`
|
|
- `frontend/src/components/detection/DetectionLab.tsx`
|
|
- `frontend/src/components/segmentation/SegmentationLab.tsx`
|
|
- `scripts/run_readiness_check.sh`
|
|
- `frontend/README.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- Sprint 10 intentionally does not add new backend capabilities, migrations, product features, AI dependencies or live provider fetching.
|
|
- `App.tsx` still owns shared workbench state orchestration; further extraction can be considered in a later maintainability pass if needed.
|
|
|
|
### Sprint 10 addendum - additional frontend extraction
|
|
- Extracted `frontend/src/components/project/ProjectPanel.tsx` and `frontend/src/components/project/AreaPanel.tsx` from `frontend/src/App.tsx`.
|
|
- Kept project and area form state owned by `App.tsx`; extracted components receive state and callbacks only.
|
|
- `cd frontend && npm run typecheck` passed after the additional extraction.
|
|
|
|
## Pass 27 - Sprint 11 Live Docker/PostGIS Runtime Validation (2026-06-13)
|
|
|
|
### Completed
|
|
- Hardened `scripts/live_migration_smoke.sh` so it runs `SELECT 1`, applies `alembic upgrade head`, then checks `PostGIS_Version()`.
|
|
- Added migrated schema-object checks for core tables and geometry indexes after the live migration step.
|
|
- Added `backend/tests/test_live_migration_smoke_script.py` to lock the smoke-script ordering and schema-check contract.
|
|
- Documented the Docker/PostGIS validation command sequence, expected local `DATABASE_URL` and cleanup commands in `backend/README.md`.
|
|
|
|
### Files changed
|
|
- `scripts/live_migration_smoke.sh`
|
|
- `backend/tests/test_live_migration_smoke_script.py`
|
|
- `backend/README.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Runtime status
|
|
- Docker is not installed or not available in this shell, so `docker compose config`, `docker compose up -d db` and the live container-backed smoke could not be completed here.
|
|
- On a Docker-enabled machine, run:
|
|
- `docker compose config`
|
|
- `docker compose up -d db`
|
|
- `DATABASE_URL=postgresql+psycopg://geointel:geointel@localhost:5432/geointel bash scripts/live_migration_smoke.sh`
|
|
|
|
### Known limitations
|
|
- Sprint 11 did not add product behavior, API contracts, migrations, AI dependencies or provider fetching.
|
|
- Live runtime validation is partially blocked until Docker/PostGIS is available in the execution environment.
|
|
|
|
## Pass 28 - Sprint 12 QA/QC golden dataset and benchmarking (2026-06-15)
|
|
|
|
### Completed
|
|
- Added deterministic golden QA/QC fixtures for reference and predicted building polygons.
|
|
- Added `fixtures/golden/expected_qa_metrics.json` with the expected partial-match baseline.
|
|
- Added `scripts/run_golden_qa_benchmark.py` to run existing `QaService` logic and verify `QualityService` persistence output.
|
|
- Added backend tests for expected golden metrics, benchmark command output and persisted metric keys.
|
|
- Updated QA/QC specification, backend README and changelog documentation.
|
|
|
|
### Files changed
|
|
- `fixtures/golden/reference_buildings.geojson`
|
|
- `fixtures/golden/predicted_buildings.geojson`
|
|
- `fixtures/golden/expected_qa_metrics.json`
|
|
- `scripts/run_golden_qa_benchmark.py`
|
|
- `backend/tests/test_sprint12_golden_qa_benchmark.py`
|
|
- `docs/QA_QC_SPECIFICATION.md`
|
|
- `backend/README.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Expected benchmark metrics
|
|
- precision: `0.5`
|
|
- recall: `0.5`
|
|
- F1: `0.5`
|
|
- mean IoU: `0.8339768339761133`
|
|
- false positives: `1`
|
|
- false negatives: `1`
|
|
|
|
### Known limitations
|
|
- The benchmark uses explicit fixture/demo data and an in-memory persistence session; it does not replace the pending Docker/PostGIS live smoke.
|
|
- Sprint 12 does not add product features, API contracts, migrations, live providers, AI model execution or new dependencies.
|
|
|
|
## Pass 29 - Sprint 13 Real YOLO operational hardening (2026-06-15)
|
|
|
|
### Completed
|
|
- Added `YoloPreflightService` for local configured-YOLO readiness checks without model loading, inference or downloads.
|
|
- Added `scripts/yolo_preflight.py` for CLI checks of enabled state, dependencies, local model file, tile manifest validity, tile limit and tile paths.
|
|
- Added Sprint 13 backend tests for disabled, dependency-unavailable and ready preflight states plus CLI JSON output.
|
|
- Updated AI pipeline, backend README and changelog documentation.
|
|
|
|
### Files changed
|
|
- `backend/app/services/yolo_preflight_service.py`
|
|
- `scripts/yolo_preflight.py`
|
|
- `backend/tests/test_sprint13_yolo_preflight.py`
|
|
- `docs/AI_PIPELINES.md`
|
|
- `backend/README.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- Preflight does not prove model compatibility or inference correctness; it intentionally avoids loading YOLO models.
|
|
- Optional AI dependencies are still not installed by default.
|
|
- Docker/PostGIS live validation remains pending until Docker is available.
|
|
|
|
## Pass 30 - Release hardening audit pass (2026-06-15)
|
|
|
|
### Completed
|
|
- Audited release-readiness signals after Sprint 13, including timestamp warnings, frontend bundle output, migration SQL rendering and readiness coverage.
|
|
- Replaced backend `datetime.utcnow()` calls with timezone-aware UTC timestamps in service paths.
|
|
- Verified the affected backend tests with `DeprecationWarning` promoted to errors.
|
|
- Split frontend production output into app, React vendor and MapLibre vendor chunks, with an explicit chunk warning threshold for the known MapLibre GIS runtime.
|
|
- Updated backend/frontend README, TODO and changelog documentation.
|
|
|
|
### Files changed
|
|
- `backend/app/services/dataset_service.py`
|
|
- `backend/app/services/geojson_service.py`
|
|
- `backend/app/services/job_service.py`
|
|
- `backend/app/services/qa_service.py`
|
|
- `backend/app/services/quality_service.py`
|
|
- `frontend/vite.config.ts`
|
|
- `backend/README.md`
|
|
- `frontend/README.md`
|
|
- `docs/TODO.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- This pass does not add product features, migrations, API contracts, AI dependencies, provider fetching or model execution.
|
|
- Docker/PostGIS live validation still requires a Docker-enabled machine.
|
|
- Larger frontend architectural decomposition remains a separate low-risk planning item; this pass only hardened build output.
|
|
|
|
## Pass 31 - Extended release hardening sweep (2026-06-15)
|
|
|
|
### Completed
|
|
- Promoted the backend readiness gate to run `pytest` with `-W error::DeprecationWarning`.
|
|
- Added backend tests that verify readiness and pass-end scripts keep the stricter release checks in place.
|
|
- Hardened `scripts/codex_pass_end_check.sh` so placeholder scans skip `node_modules`, `dist` and `__pycache__` folders.
|
|
- Updated `docs/TODO.md` with a current implementation status layer while preserving older planning context.
|
|
- Re-ran pass-end checks and strict backend warning checks.
|
|
|
|
### Files changed
|
|
- `scripts/run_readiness_check.sh`
|
|
- `scripts/codex_pass_end_check.sh`
|
|
- `backend/tests/test_readiness_gate.py`
|
|
- `backend/README.md`
|
|
- `docs/TODO.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- This pass still does not add API contracts, migrations, product features, provider fetching, AI dependencies or model execution.
|
|
- Docker/PostGIS live validation remains blocked in this local environment because Docker is unavailable.
|
|
|
|
## Pass 32 - Readiness contract gate hardening (2026-06-15)
|
|
|
|
### Completed
|
|
- Added API contract smoke validation to `scripts/run_readiness_check.sh`.
|
|
- Added a regression test that requires the readiness gate to keep running `scripts/smoke_contracts.py`.
|
|
- Re-ran the full readiness gate after the change.
|
|
|
|
### Files changed
|
|
- `scripts/run_readiness_check.sh`
|
|
- `backend/tests/test_readiness_gate.py`
|
|
- `backend/README.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- Docker/PostGIS live validation remains blocked in this local environment because Docker is unavailable.
|
|
|
|
## Pass 33 - Docker runtime build hardening (2026-06-15)
|
|
|
|
### Completed
|
|
- Investigated Unraid/Tower Docker build failure from pasted server output.
|
|
- Fixed backend Docker build ordering so `README.md` and `app/` exist before `pip install .`.
|
|
- Removed mandatory root `.env` references from Compose; default local runtime now uses checked-in environment values.
|
|
- Added PostGIS healthcheck and backend `depends_on` health condition.
|
|
- Made the backend container run `python -m alembic upgrade head` before starting Uvicorn.
|
|
- Added backend and frontend `.dockerignore` files to reduce Docker build context and exclude dependency/build/cache outputs.
|
|
- Added regression tests for Dockerfile package-source ordering, Compose env behavior, DB health/migration startup and Docker ignore coverage.
|
|
|
|
### Files changed
|
|
- `backend/Dockerfile`
|
|
- `docker-compose.yml`
|
|
- `backend/.dockerignore`
|
|
- `frontend/.dockerignore`
|
|
- `backend/tests/test_docker_runtime_config.py`
|
|
- `README.md`
|
|
- `backend/README.md`
|
|
- `docs/TODO.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- Docker commands still cannot be executed in this local Codex environment because Docker is unavailable here.
|
|
- The server should rerun `docker compose build --no-cache && docker compose up -d` to verify the real Docker runtime.
|
|
|
|
## Pass 34 - Docker browser port 1202 (2026-06-15)
|
|
|
|
### Completed
|
|
- Changed Docker Compose frontend host publishing from `5173:5173` to `1202:5173`.
|
|
- Added backend Docker CORS defaults for `http://localhost:1202` and `http://127.0.0.1:1202`.
|
|
- Updated `.env.example` and local/Docker documentation to point browser users to `http://localhost:1202` for Docker Compose.
|
|
- Added a regression assertion to Docker runtime config tests.
|
|
|
|
### Files changed
|
|
- `docker-compose.yml`
|
|
- `.env.example`
|
|
- `backend/tests/test_docker_runtime_config.py`
|
|
- `README.md`
|
|
- `backend/README.md`
|
|
- `frontend/README.md`
|
|
- `docs/LOCAL_DEVELOPMENT_RUNBOOK.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- Docker commands still cannot be executed in this local Codex environment because Docker is unavailable here.
|
|
|
|
## Pass 35 - Docker backend database startup retry (2026-06-16)
|
|
|
|
### Completed
|
|
- Investigated Tower runtime logs showing backend Alembic startup failed with `connection refused` even after the db container reported healthy.
|
|
- Added `backend/docker_start.sh` to retry a real SQLAlchemy `SELECT 1` connection before migrations.
|
|
- Updated Compose to run `sh /app/docker_start.sh` for backend startup.
|
|
- Added regression tests that require the Docker start script and SQL readiness retry before migrations.
|
|
|
|
### Files changed
|
|
- `backend/docker_start.sh`
|
|
- `docker-compose.yml`
|
|
- `backend/tests/test_docker_runtime_config.py`
|
|
- `backend/README.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- Docker commands still cannot be executed in this local Codex environment because Docker is unavailable here.
|
|
|
|
## Pass 36 - Alembic Docker log formatting fix (2026-06-16)
|
|
|
|
### Completed
|
|
- Investigated backend Docker logs showing repeated literal `%(levelname)-5.5s [%(name)s] %(message)s` lines during migrations.
|
|
- Fixed `backend/alembic.ini` logging formatter from escaped `%%(...)` tokens to runtime interpolation `%(...)` tokens.
|
|
- Added a regression test for Alembic logging formatter correctness.
|
|
- Verified Alembic SQL rendering no longer emits literal formatter spam.
|
|
|
|
### Files changed
|
|
- `backend/alembic.ini`
|
|
- `backend/tests/test_alembic_logging_config.py`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
## Pass 37 - Frontend same-origin API proxy for Docker LAN access (2026-06-16)
|
|
|
|
### Completed
|
|
- Audited the running app at `http://192.168.10.150:1202` from Codex: frontend HTML and backend `/health` were reachable, but `/api/v1/projects` on the frontend origin returned the frontend HTML fallback.
|
|
- Changed the frontend API client default from `http://localhost:8000` to same-origin requests.
|
|
- Added Vite proxy routes for `/api` and `/health`.
|
|
- Added Docker Compose `VITE_API_PROXY_TARGET=http://backend:8000` so LAN browsers use `http://192.168.10.150:1202` only and the frontend container proxies API calls internally.
|
|
- Added regression tests for same-origin API/proxy behavior.
|
|
|
|
### Files changed
|
|
- `frontend/src/services/api/client.ts`
|
|
- `frontend/vite.config.ts`
|
|
- `docker-compose.yml`
|
|
- `backend/tests/test_docker_runtime_config.py`
|
|
- `README.md`
|
|
- `frontend/README.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- The running Tower deployment needs a rebuild/restart before this fix is active.
|
|
|
|
## Pass 38 - Browser runtime proxy guard (2026-06-16)
|
|
|
|
### Completed
|
|
- Added `scripts/verify_browser_runtime.sh` to verify the browser-facing frontend URL, `/api/v1/projects` proxy behavior and optional backend health endpoint.
|
|
- Added readiness syntax validation for the browser runtime verification script.
|
|
- Updated environment and local development documentation to prefer same-origin frontend API calls with Vite proxying in Docker/LAN deployments.
|
|
|
|
### Files changed
|
|
- `scripts/verify_browser_runtime.sh`
|
|
- `scripts/run_readiness_check.sh`
|
|
- `backend/tests/test_docker_runtime_config.py`
|
|
- `README.md`
|
|
- `docs/ENVIRONMENT_SPEC.md`
|
|
- `docs/LOCAL_DEVELOPMENT_RUNBOOK.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- The currently running Tower deployment at `http://192.168.10.150:1202` still returns frontend HTML for `/api/v1/projects` until the frontend image is rebuilt and restarted.
|
|
|
|
## Pass 39 - Environment contract cleanup (2026-06-16)
|
|
|
|
### Completed
|
|
- Corrected `.env.example` and `docs/ENVIRONMENT_SPEC.md` to use the YOLO environment variable names read by backend settings: `YOLO_ENABLED`, `YOLO_MODEL_PATH`, `YOLO_MODEL_VERSION` and `YOLO_MAX_TILES`.
|
|
- Updated frontend example settings to keep `VITE_API_BASE_URL` empty by default and expose `VITE_API_PROXY_TARGET` for Vite proxy routing.
|
|
- Added regression coverage for example environment names and browser runtime proxy verification.
|
|
|
|
### Files changed
|
|
- `.env.example`
|
|
- `docs/ENVIRONMENT_SPEC.md`
|
|
- `backend/tests/test_docker_runtime_config.py`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- Docker cannot be executed in this local Codex environment; Tower still needs an image rebuild/restart to activate the frontend proxy fix.
|
|
|
|
## Pass 40 - Frontend Docker nginx reverse proxy (2026-06-16)
|
|
|
|
### Completed
|
|
- Replaced the Docker frontend runtime with a production Vite build served by nginx.
|
|
- Added `frontend/nginx.conf` with explicit reverse proxy rules for `/api/` and `/health` to `backend:8000`.
|
|
- Changed Docker Compose frontend publishing from `1202:5173` to `1202:80`.
|
|
- Updated Docker runtime regression tests and documentation so the browser-facing API path is no longer dependent on Vite dev-server proxy behavior.
|
|
|
|
### Files changed
|
|
- `frontend/Dockerfile`
|
|
- `frontend/nginx.conf`
|
|
- `docker-compose.yml`
|
|
- `backend/tests/test_docker_runtime_config.py`
|
|
- `README.md`
|
|
- `frontend/README.md`
|
|
- `docs/ENVIRONMENT_SPEC.md`
|
|
- `docs/LOCAL_DEVELOPMENT_RUNBOOK.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- Docker still cannot be executed in this local Codex environment. Tower must rebuild the frontend image to activate the nginx runtime.
|
|
|
|
## Sprint 14 - Docker GIS runtime enablement (2026-06-16)
|
|
|
|
- Added backend `gis` optional dependency group for the approved Rasterio/GeoPandas runtime stack.
|
|
- Updated backend Docker image to install `.[gis]` and GDAL/GEOS/PROJ system packages.
|
|
- Added `scripts/verify_gis_runtime.sh` to verify PostGIS, Rasterio and GeoPandas capabilities through the browser-facing frontend proxy.
|
|
- Added readiness syntax coverage for the GIS runtime verification script.
|
|
- Added regression tests for Docker GIS dependency installation and capability verification coverage.
|
|
- Updated backend, environment, root README and changelog documentation with local/LAN verification commands.
|
|
- No API contracts, migrations, AI dependencies, provider fetching or product features were changed.
|
|
- Added `scripts/gis_import_smoke.py` and wired it into the backend Docker build so broken Rasterio/GeoPandas/pyogrio imports fail during image build.
|
|
- Added Docker Compose healthchecks for backend and frontend; frontend now waits for backend service health before starting.
|
|
- Corrected the GIS import smoke placement so the backend Docker build can access it inside the `./backend` build context; the root script now wraps the backend script.
|
|
|
|
## Sprint 15 - Explicit demo workflow seed (2026-06-16)
|
|
|
|
- Added `POST /api/v1/demo/workflow` for an explicit offline demo workflow seed.
|
|
- Added `DemoWorkflowService` to create or return a demo project, AOI, fixture reference dataset, fixture candidate dataset and persisted QA/QC metrics.
|
|
- Added `scripts/seed_demo_workflow.py` for terminal-based demo seeding.
|
|
- Added frontend `Load demo workflow` action in the Projects panel.
|
|
- Added endpoint/fixture-contract tests and readiness compile coverage for the demo seed script.
|
|
- No live GRB/OSM fetching, AI inference, migrations or new dependencies were introduced.
|
|
|
|
## Sprint 16 - QA/QC result visibility (2026-06-16)
|
|
|
|
- Added `GET /api/v1/projects/{project_id}/quality-checks` for read-only project QA/QC result listing.
|
|
- Added `QualityCheckService` to return persisted `quality_checks` with metric rows.
|
|
- Added frontend QA/QC Results panel and API client support.
|
|
- Demo workflow loading and QA actions now refresh persisted QA/QC results in the UI.
|
|
- Added backend tests for quality check listing and canonical envelopes.
|
|
- No migrations, live provider fetching, AI inference or new dependencies were introduced.
|
|
|
|
## Sprint 17 export foundation (2026-06-16)
|
|
|
|
Changed:
|
|
- Hardened GeoJSON exports so vector dataset, detection run and segmentation run exports persist `exports` rows and write JSON artifacts.
|
|
- Added project metadata JSON export plus export list/read/content endpoints.
|
|
- Added a frontend Export Center panel for creating exports, listing export records and previewing JSON content.
|
|
- Added backend tests for export persistence, artifact writing, raster rejection and canonical envelope behavior.
|
|
|
|
Tested:
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh`
|
|
- `cd backend && python -m alembic heads && python -m alembic upgrade head --sql`
|
|
- `bash -n scripts/live_migration_smoke.sh && bash -n scripts/verify_browser_runtime.sh && bash -n scripts/verify_gis_runtime.sh`
|
|
|
|
Open:
|
|
- Docker/live browser validation must be rerun on the deployment host after rebuild.
|
|
- YOLO-format export and report export remain documented future work; this pass only implements JSON/GeoJSON export foundation.
|
|
|
|
Limitations:
|
|
- Export artifacts are returned through API JSON content preview; browser file-download UX is not implemented yet.
|
|
- Detection and segmentation exports require existing persisted runs; no inference or fake output generation is introduced.
|
|
|
|
Next recommended pass:
|
|
- Rebuild/redeploy the Docker stack and verify `/api/v1/exports/*` through the LAN frontend proxy, then consider a lightweight file-download endpoint or report artifact pass.
|
|
|
|
## Sprint 17 export download hardening (2026-06-16)
|
|
|
|
Changed:
|
|
- Added `GET /api/v1/exports/{export_id}/download` as a raw file response for stored JSON/GeoJSON export artifacts.
|
|
- Reused the same export artifact existence validation for content preview and downloads.
|
|
- Added frontend Export Center download buttons using the configured/same-origin API base URL.
|
|
- Added backend tests for missing artifacts and file download response headers/content.
|
|
|
|
Tested:
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh`
|
|
- `cd backend && python -m alembic heads && python -m alembic upgrade head --sql`
|
|
- `bash -n scripts/live_migration_smoke.sh && bash -n scripts/verify_browser_runtime.sh && bash -n scripts/verify_gis_runtime.sh`
|
|
|
|
Open:
|
|
- Docker/LAN validation should be rerun after redeploy on the Tower host.
|
|
|
|
Limitations:
|
|
- Download endpoint is intentionally a raw file response, not a canonical JSON envelope, because it is a browser/file artifact path.
|
|
|
|
Next recommended pass:
|
|
- Rebuild Docker and verify Export Center create/preview/download against `http://192.168.10.150:1202`.
|
|
|
|
## Sprint 17 lightweight report artifact export (2026-06-16)
|
|
|
|
Changed:
|
|
- Added `POST /api/v1/exports/report` to create a lightweight HTML project report artifact from persisted project, dataset and QA/QC summary state.
|
|
- Added HTML escaping for report-rendered project and dataset values.
|
|
- Updated export downloads to return `text/html` for HTML report artifacts and `application/json` for JSON/GeoJSON artifacts.
|
|
- Added a frontend Export Center action for project report HTML export.
|
|
- Added backend tests for HTML report artifact creation and HTML download response behavior.
|
|
|
|
Tested:
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh`
|
|
- `cd backend && python -m alembic heads && python -m alembic upgrade head --sql`
|
|
- `bash -n scripts/live_migration_smoke.sh && bash -n scripts/verify_browser_runtime.sh && bash -n scripts/verify_gis_runtime.sh`
|
|
|
|
Open:
|
|
- Docker/LAN verification should be rerun after deployment rebuild.
|
|
|
|
Limitations:
|
|
- Report export is intentionally a simple HTML artifact, not a PDF designer or standalone Reports module.
|
|
- Report content is summary-only and uses existing persisted project, dataset and QA/QC rows.
|
|
|
|
Next recommended pass:
|
|
- Rebuild Docker and verify Export Center metadata, GeoJSON, report HTML and download flows through the LAN URL.
|
|
|
|
## Sprint 17 export audit trail and LAN demo/export smoke (2026-06-16)
|
|
|
|
Changed:
|
|
- Added export history to project metadata JSON and lightweight HTML report artifacts.
|
|
- Added `scripts/verify_demo_export_workflow.sh` to verify the browser-facing demo workflow, persisted QA/QC listing, metadata export, report export, vector GeoJSON export, export listing and artifact downloads.
|
|
- Included the demo/export workflow script in the main readiness syntax gate.
|
|
- Added backend tests to lock export-history content and demo/export script coverage.
|
|
|
|
Tested:
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh`
|
|
- `cd backend && python -m alembic heads && python -m alembic upgrade head --sql`
|
|
- `bash -n scripts/live_migration_smoke.sh && bash -n scripts/verify_browser_runtime.sh && bash -n scripts/verify_gis_runtime.sh && bash -n scripts/verify_demo_export_workflow.sh`
|
|
|
|
Open:
|
|
- `scripts/verify_demo_export_workflow.sh` still needs to be run against the rebuilt Tower deployment URL.
|
|
|
|
Limitations:
|
|
- The smoke script intentionally uses explicit fixture demo data and does not fetch live GRB/OSM or run AI inference.
|
|
|
|
Next recommended pass:
|
|
- Rebuild Docker on the Tower host and run `bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202`.
|
|
|
|
## Sprint 32 Unraid all-in-one runtime verification (2026-06-17)
|
|
|
|
Changed:
|
|
- Converted the Unraid/Tower runtime to a single editable `geointel` Docker container via `docker-compose.unraid.yml`.
|
|
- Built the all-in-one image from `postgres:16-bookworm` with PostgreSQL 16/PostGIS packages, the FastAPI backend, nginx and the React frontend.
|
|
- Removed the default nginx site from the image so `/api/v1/*` is proxied to the embedded backend instead of returning nginx 404s.
|
|
- Hardened live migration smoke and browser runtime verification with startup retries.
|
|
- Browser runtime verification now checks the frontend, `/api/v1/projects` canonical JSON envelope and `/geointel-icon.png`.
|
|
|
|
Tested:
|
|
- `python -m pytest backend\tests\test_sprint31_unraid_template.py`
|
|
- `bash -n deploy/unraid/all-in-one-start.sh`
|
|
- `bash -n scripts/live_migration_smoke.sh`
|
|
- `bash -n scripts/verify_browser_runtime.sh`
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1`
|
|
- `docker compose -f docker-compose.unraid.yml ps` on Tower
|
|
- `bash scripts/verify_browser_runtime.sh http://192.168.10.150:1202`
|
|
|
|
Verified runtime:
|
|
- Tower runs one GeoIntel app container: `geointel`.
|
|
- Published browser port is `0.0.0.0:1202->80/tcp`.
|
|
- Container health is `healthy`.
|
|
- Live migration smoke passed with PostGIS available and one Alembic head.
|
|
- Frontend, API proxy and icon are reachable at `http://192.168.10.150:1202`.
|
|
|
|
Open:
|
|
- Existing reused PostgreSQL volume logs a collation version mismatch because the old database was created on Debian glibc 2.31 and the all-in-one runtime uses glibc 2.36. Runtime and migrations pass; a future maintenance pass can rebuild/refresh collation metadata if needed.
|
|
|
|
## Sprint 32 DockerMan template/icon follow-up (2026-06-17)
|
|
|
|
Changed:
|
|
- Added a PNG icon for Unraid/DockerMan because DockerMan may not reliably render app-served SVG icons.
|
|
- Changed the Unraid template `<Name>` to `geointel` so it matches the running all-in-one container name.
|
|
- Changed the template icon URL to the absolute PNG URL `http://192.168.10.150:1202/geointel-icon.png`.
|
|
- Added DockerMan labels to `docker-compose.unraid.yml` so the running Compose container exposes Unraid managed/webui/icon metadata.
|
|
- Updated Tower deploy scripts to copy the editable template to `/boot/config/plugins/dockerMan/templates-user/my-geointel.xml`.
|
|
- Updated Tower deploy scripts to copy the PNG icon to `/boot/config/plugins/dockerMan/images/geointel-icon.png`.
|
|
|
|
Expected Unraid behavior:
|
|
- Refresh the Unraid Docker page after deploy so DockerMan reloads the user template metadata.
|
|
- The running `geointel` container should have template-backed editable fields and a PNG icon.
|
|
|
|
## Sprint 32 automatic DockerMan-native deploy follow-up (2026-06-17)
|
|
|
|
Changed:
|
|
- Added `deploy/unraid/run-dockerman-container.sh` as the single source of truth for the final Unraid container launch.
|
|
- Updated Tower deploy scripts so repository deploys build the image with Compose but start the final container with `docker run` and DockerMan labels.
|
|
- The launch script installs the DockerMan template/icon, removes old `geointel` containers, preserves/migrates the old Compose PostGIS volume when needed and starts the final `geointel` container as DockerMan-managed.
|
|
- Updated live migration smoke to support direct container execution through `LIVE_SMOKE_CONTAINER=geointel`.
|
|
|
|
Expected Unraid behavior:
|
|
- A deploy from the repo should no longer leave the final app as a plain Compose-owned container.
|
|
- The running `geointel` container should expose `net.unraid.docker.managed=dockerman`, web UI metadata and icon metadata immediately after deploy.
|