733 lines
27 KiB
Markdown
733 lines
27 KiB
Markdown
# GeoIntel Backend (Sprint 3 foundation layer)
|
|
|
|
FastAPI backend for GeoIntel Kempen Foundation Sprints.
|
|
|
|
## Scope implemented
|
|
- Project CRUD
|
|
- Area CRUD with PostGIS geometry
|
|
- Vector and raster dataset upload/registration
|
|
- Deterministic local storage metadata capture
|
|
- PostGIS migration and database foundation
|
|
- Job foundation for async-ready GIS operations
|
|
|
|
## Sprint 2 additions
|
|
- Dataset typing and lifecycle support:
|
|
- `uploaded`
|
|
- `validating`
|
|
- `ready`
|
|
- `failed`
|
|
- Vector metadata extraction:
|
|
- feature count
|
|
- geometry type summary
|
|
- bounds
|
|
- approximate area
|
|
- CRS and CRS assumption
|
|
- Raster metadata endpoint:
|
|
- returns raster profile when `rasterio` is available
|
|
- returns clear `RASTER_PROCESSING_UNAVAILABLE` error when dependency is missing
|
|
- Deterministic storage metadata capture:
|
|
- original filename
|
|
- stored filename
|
|
- MIME/content type
|
|
- size bytes
|
|
- checksum SHA-256
|
|
|
|
## Sprint 3 additions
|
|
- Lightweight job architecture:
|
|
- `jobs` table and migrations
|
|
- job create/list/read/status API
|
|
- synchronous execution behind job abstraction
|
|
- Vector operations foundation:
|
|
- inspect
|
|
- bbox
|
|
- stats
|
|
- clip by area
|
|
- buffer
|
|
- intersect
|
|
- invalid geometry rejection with typed errors
|
|
- Raster operation foundation:
|
|
- inspect
|
|
- metadata
|
|
- preview readiness
|
|
- clip by area (dependency-aware with unavailable fallback)
|
|
- tile generation with manifest output
|
|
- real preview image generation when dependencies are installed
|
|
|
|
## Sprint 4 additions
|
|
- Raster foundation is now implemented with real extraction and deterministic artifact outputs:
|
|
- metadata returns width, height, band count, CRS, bounds, resolution, dtype, nodata, transform
|
|
- preview endpoint generates and reuses PNG previews with width/height
|
|
- clip operation persists a derived raster dataset with:
|
|
- `source_dataset_id`
|
|
- `operation`
|
|
- `operation_parameters`
|
|
- tile operation writes deterministic raster tiles under `tiles/{project_id}/{source_dataset_id}/{tile_set_id}`
|
|
- tile manifest includes tile path, pixel window, bounds, transform, and count
|
|
- Dependency behavior:
|
|
- when `rasterio` is missing, raster processing returns `RASTER_PROCESSING_UNAVAILABLE`
|
|
- preview endpoint additionally requires numpy/pillow and returns `RASTER_PROCESSING_UNAVAILABLE` when missing
|
|
|
|
## Sprint 5 additions
|
|
- Raster analytics hardening:
|
|
- raster band statistics now include:
|
|
- min, max, mean, std
|
|
- nodata count and ratio
|
|
- valid pixel count
|
|
- dtype
|
|
- optional histogram bins (default 16 bins)
|
|
- raster reproject operation implemented (CRS transform + rasterio reprojection) using dependency-aware raster processing checks.
|
|
- reproject failures are explicit (`INVALID_PARAMETERS`, `INVALID_DATASET_CRS`, `RASTER_PROCESSING_UNAVAILABLE`).
|
|
- Raster clip and tile hardening:
|
|
- clip validates area presence and CRS alignment constraints.
|
|
- tile manifest records `tile_set_id`, `tile_size`, `overlap`, `source_dataset_id`, `source_raster_id`, bounds, parameters, count, tile paths, `ai_inference`, and `tile_server`.
|
|
- Job result persistence for raster ops:
|
|
- raster clip/reproject/tile job payloads persist derived dataset references when outputs are produced.
|
|
|
|
## Sprint 6 additions
|
|
- Added local spectral index operations:
|
|
- NDVI endpoint: `POST /raster/indices/ndvi`
|
|
- NDWI endpoint: `POST /raster/indices/ndwi`
|
|
- NDBI endpoint: `POST /raster/indices/ndbi`
|
|
- Spectral index input validation:
|
|
- band parameters must be positive integers
|
|
- band parameters must exist in source raster band count
|
|
- Dependency-aware execution:
|
|
- returns `RASTER_PROCESSING_UNAVAILABLE` when rasterio or numpy are unavailable
|
|
- Real index output handling:
|
|
- local windowed float32 GeoTIFF generation
|
|
- `NaN` strategy for invalid pixels / division by zero
|
|
- Provenance capture for derived index datasets:
|
|
- `source_dataset_id`, `operation`, `band_mapping`, `formula`
|
|
- `output_dtype`, `nodata_strategy`, `value_range_note`
|
|
- `output_dataset_id`, `created_at`, `path`
|
|
|
|
## Sprint 7B additions
|
|
- Added provider registry skeleton for `grb`, `osm`, `manual` and `fixture`.
|
|
- Added provider capability endpoints:
|
|
- `GET /api/v1/external/providers`
|
|
- `GET /api/v1/external/providers/{provider_name}`
|
|
- `GET /api/v1/external/providers/{provider_name}/layers`
|
|
- `GET /api/v1/external/providers/{provider_name}/status`
|
|
- `POST /api/v1/external/providers/{provider_name}/import`
|
|
- GRB and OSM imports return explicit `not_configured` responses; no live WFS or Overpass calls are made.
|
|
- Manual and fixture providers describe existing upload/fixture flows only.
|
|
- Added live PostGIS migration smoke script for environments with a real database:
|
|
|
|
```bash
|
|
bash scripts/live_migration_smoke.sh
|
|
```
|
|
|
|
## Sprint 8 additions
|
|
- Added Detection Lab foundation:
|
|
- `detections` ORM model and Alembic migration with PostGIS geometry storage.
|
|
- hardened `analysis_runs` for dataset/job/model/result metadata.
|
|
- model registry capability service for `yolo-placeholder` and `manual-fixture-detector`.
|
|
- detection service boundary for creating jobs, analysis runs and dependency-aware unavailable responses.
|
|
- Added detection endpoints:
|
|
- `GET /api/v1/detection/models`
|
|
- `GET /api/v1/detection/model-assets`
|
|
- `POST /api/v1/detection/run`
|
|
- `GET /api/v1/detection/runs/{analysis_run_id}`
|
|
- `GET /api/v1/detection/runs/{analysis_run_id}/detections`
|
|
- YOLO/PyTorch real inference is not enabled in Sprint 8.
|
|
- Fixture detector mode is test/demo-only and requires explicit `fixture_mode=true`.
|
|
|
|
## Sprint 8B additions
|
|
- Added optional configured YOLO integration foundation:
|
|
- `yolo-configured` model registry capability.
|
|
- import-safe adapter for local Ultralytics model files.
|
|
- raster tile manifest validation and tile limit enforcement.
|
|
- pixel bbox to EPSG:4326 detection polygon conversion.
|
|
- persisted detections through the existing detection/job/analysis-run path.
|
|
- YOLO dependencies are optional extras and are not required for backend startup.
|
|
- GeoIntel does not download YOLO model weights automatically.
|
|
|
|
## Sprint 8C additions
|
|
- Added detection visualization/review API support:
|
|
- list detection runs
|
|
- list detections by run or dataset with class/confidence filters
|
|
- get detection detail
|
|
- return persisted detections as GeoJSON FeatureCollections
|
|
- Added detection QA against reference vector datasets:
|
|
- compares persisted detection geometries against persisted `vector_features`
|
|
- persists `quality_checks` and `metrics`
|
|
- returns precision, recall, F1, mean IoU and false positive/negative counts
|
|
- Segmentation, LiDAR, AI Copilot, Training Studio and Reports remain out of scope.
|
|
|
|
## Sprint 9 additions
|
|
- Added Segmentation Lab foundation:
|
|
- `segmentations` ORM model and Alembic migration with PostGIS MultiPolygon geometry storage.
|
|
- segmentation model registry capabilities for `segmentation-placeholder`, `fixture-segmenter`, `yolo-seg-configured` and `sam-configured`.
|
|
- segmentation service boundary for creating jobs, analysis runs and unavailable model responses.
|
|
- explicit fixture segmenter mode for tests/demo fixtures only.
|
|
- Added segmentation endpoints:
|
|
- `GET /api/v1/segmentation/models`
|
|
- `POST /api/v1/segmentation/run`
|
|
- `GET /api/v1/segmentation/runs`
|
|
- `GET /api/v1/segmentation/runs/{analysis_run_id}`
|
|
- `GET /api/v1/segmentation/runs/{analysis_run_id}/segmentations`
|
|
- `GET /api/v1/segmentation/runs/{analysis_run_id}/geojson`
|
|
- `POST /api/v1/segmentation/runs/{analysis_run_id}/qa/reference`
|
|
- Real SAM and YOLO-seg inference are not enabled in Sprint 9.
|
|
- Mask paths are provenance/debug artifacts; persisted PostGIS geometry is authoritative for QA, map display and GeoJSON.
|
|
|
|
## Sprint 17 additions
|
|
- Added export foundation backed by the existing `exports` table.
|
|
- GeoJSON exports now persist export records and write JSON artifacts for:
|
|
- vector datasets
|
|
- detection analysis runs
|
|
- segmentation analysis runs
|
|
- Added project metadata JSON export for project, dataset and QA/QC summary state.
|
|
- Added export read/list/content endpoints:
|
|
- `POST /api/v1/exports/geojson`
|
|
- `POST /api/v1/exports/metadata`
|
|
- `GET /api/v1/exports/projects/{project_id}/exports`
|
|
- `GET /api/v1/exports/{export_id}`
|
|
- `GET /api/v1/exports/{export_id}/content`
|
|
- Exported detection and segmentation GeoJSON is generated from persisted first-class geometry rows.
|
|
- No new migrations, product lines, live providers or AI dependencies are introduced by this export pass.
|
|
- Old offline demo export artifacts can be inspected with `python scripts/cleanup_demo_artifacts.py`
|
|
and removed only with an explicit `--apply`. The script keeps the newest exports
|
|
per demo project, refuses to delete files outside `STORAGE_ROOT`, and blocks
|
|
apply runs above `--max-delete` until the cap is raised after a dry-run review.
|
|
Use repeated `--export-type` values to target only specific artifact kinds.
|
|
In Docker, use `docker compose exec -T backend python scripts/cleanup_demo_artifacts.py`.
|
|
- Live cleanup validation is available with `bash scripts/verify_demo_cleanup_dry_run.sh`.
|
|
It runs the same maintenance path without `--apply` and fails if the summary
|
|
reports anything other than a dry-run with zero deleted exports/files.
|
|
|
|
## Run locally
|
|
|
|
### Prerequisites
|
|
|
|
- Python 3.11+
|
|
- PostgreSQL with PostGIS
|
|
|
|
### Install dependencies
|
|
|
|
```bash
|
|
cd backend
|
|
python -m pip install -e .[dev]
|
|
```
|
|
|
|
Optional AI dependencies for configured local YOLO inference:
|
|
|
|
```bash
|
|
cd backend
|
|
python -m pip install -e .[ai]
|
|
```
|
|
|
|
Docker and Unraid builds keep AI dependencies disabled by default. To build an
|
|
image with local PyTorch/Ultralytics support, set:
|
|
|
|
```bash
|
|
GEOINTEL_INSTALL_AI=true
|
|
```
|
|
|
|
The default remains `false` so normal GIS deployments do not install the large AI
|
|
runtime. GeoIntel still requires an explicit local model path and never downloads
|
|
weights automatically.
|
|
|
|
AI-enabled Docker images include the native OpenCV runtime libraries required by
|
|
Ultralytics. Dependency availability is checked with real `torch` and
|
|
`ultralytics` imports, so missing shared libraries are reported as
|
|
`dependency_unavailable` instead of being treated as configured.
|
|
Docker/Unraid runtimes set `YOLO_CONFIG_DIR` to a writable storage path so
|
|
Ultralytics does not attempt to write settings under the root user config
|
|
directory.
|
|
|
|
Configured YOLO requires:
|
|
|
|
```bash
|
|
YOLO_ENABLED=true
|
|
YOLO_MODELS_DIR=/absolute/path/to/models
|
|
YOLO_MODEL_PATH=/absolute/path/to/local-model.pt
|
|
```
|
|
|
|
Optional local model compatibility smoke:
|
|
|
|
```bash
|
|
python scripts/yolo_preflight.py --model-path /absolute/path/to/local-model.pt --tile-manifest-path /absolute/path/to/manifest.json --check-model-load --json
|
|
```
|
|
|
|
In Docker, run the same smoke through the backend container:
|
|
|
|
```bash
|
|
docker compose exec -T backend python scripts/yolo_preflight.py --model-path /absolute/path/to/local-model.pt --tile-manifest-path /absolute/path/to/manifest.json --check-model-load --json
|
|
```
|
|
|
|
In the all-in-one Unraid runtime, place model files under the configured models
|
|
directory, mounted as `/app/models` by default:
|
|
|
|
```bash
|
|
GEOINTEL_MODELS_PATH=/mnt/user/appdata/geointel/models
|
|
YOLO_ENABLED=true
|
|
YOLO_MODELS_DIR=/app/models
|
|
YOLO_MODEL_PATH=/app/models/local-model.pt
|
|
```
|
|
|
|
The root helper can write those values safely after a local model is placed:
|
|
|
|
```bash
|
|
python scripts/configure_yolo_model.py \
|
|
--models-dir /mnt/user/appdata/geointel/models \
|
|
--env-file /mnt/user/appdata/geointel/.env \
|
|
--apply
|
|
```
|
|
|
|
The smoke loads only the supplied local model file, does not run inference and
|
|
does not download weights.
|
|
|
|
The backend also exposes a read-only model asset catalog for the mounted model
|
|
directory:
|
|
|
|
```bash
|
|
curl http://localhost:1202/api/v1/detection/model-assets
|
|
```
|
|
|
|
The catalog lists local `.pt`, `.onnx` and `.engine` files with size, SHA-256
|
|
and active-model status. Detection runs may submit `model_asset_id` with
|
|
`model_id="yolo-configured"` to use a cataloged local model for that run. The
|
|
backend resolves the ID to a file inside `YOLO_MODELS_DIR`; browser clients do
|
|
not send arbitrary model paths.
|
|
|
|
Configured YOLO inference uses raster tile artifacts from the existing tile
|
|
manifest flow. Single-band or otherwise non-RGB tile images are converted to a
|
|
temporary RGB prediction image before inference; georeferencing still comes
|
|
from the persisted tile manifest transform/bounds metadata.
|
|
|
|
Optional tuning:
|
|
|
|
```bash
|
|
YOLO_MODEL_ID=yolo-configured
|
|
YOLO_MODEL_DISPLAY_NAME="Configured YOLO detector"
|
|
YOLO_MODEL_VERSION=local-v1
|
|
YOLO_MODELS_DIR=/app/models
|
|
YOLO_CONFIG_DIR=/app/storage/ultralytics
|
|
YOLO_DEVICE=cpu
|
|
YOLO_IMAGE_SIZE=640
|
|
YOLO_MAX_TILES=100
|
|
YOLO_BATCH_SIZE=1
|
|
```
|
|
|
|
### YOLO local preflight
|
|
|
|
Sprint 13 adds a local-only preflight for configured YOLO paths:
|
|
|
|
```bash
|
|
python scripts/yolo_preflight.py --model-path /absolute/path/to/local-model.pt --tile-manifest-path /absolute/path/to/manifest.json
|
|
```
|
|
|
|
Machine-readable output:
|
|
|
|
```bash
|
|
python scripts/yolo_preflight.py --model-path /absolute/path/to/local-model.pt --tile-manifest-path /absolute/path/to/manifest.json --json
|
|
```
|
|
|
|
To validate only local model/manifest paths on a machine without optional AI dependencies:
|
|
|
|
```bash
|
|
python scripts/yolo_preflight.py --model-path /absolute/path/to/local-model.pt --tile-manifest-path /absolute/path/to/manifest.json --assume-dependencies --json
|
|
```
|
|
|
|
The preflight checks configuration, dependency availability, local model file existence, tile manifest validity, tile count and referenced tile paths. JSON output also includes runtime diagnostics for the model directory, `YOLO_CONFIG_DIR`, installed `torch`/`ultralytics` versions and CUDA availability when dependency checks pass. It does not load a YOLO model, run inference or download weights.
|
|
|
|
The same read-only status is available through the API and Detection Lab UI:
|
|
|
|
```bash
|
|
curl http://localhost:1202/api/v1/detection/yolo/preflight
|
|
```
|
|
|
|
To validate the full configured-YOLO runtime path against Docker/Tower after a
|
|
model is mounted and selected, run:
|
|
|
|
```bash
|
|
bash scripts/verify_model_asset_detection_workflow.sh http://192.168.10.150:1202
|
|
```
|
|
|
|
The smoke uses the existing demo raster to generate a tile manifest, selects a
|
|
cataloged local model asset, verifies read-only preflight, submits the existing
|
|
detection run endpoint and checks persisted AnalysisRun, Detection list and
|
|
Detection GeoJSON output. It does not download weights or inject detector
|
|
fixtures. A zero detection result is still a valid runtime smoke outcome on the
|
|
synthetic demo raster.
|
|
|
|
To validate the configured building model on operator-provided GIS data, mount
|
|
or copy a real georeferenced raster and a real reference-building GeoJSON onto
|
|
the runtime host, then run:
|
|
|
|
```bash
|
|
REAL_RASTER_PATH=/mnt/user/appdata/geointel/data/orthophoto.tif \
|
|
REAL_REFERENCE_VECTOR_PATH=/mnt/user/appdata/geointel/data/reference-buildings.geojson \
|
|
bash scripts/verify_real_data_detection_qa_workflow.sh http://192.168.10.150:1202
|
|
```
|
|
|
|
This smoke refuses missing/unsupported inputs, uploads the raster and reference
|
|
dataset through the normal dataset service, generates raster tiles, selects a
|
|
local model asset, runs configured YOLO detection, compares persisted
|
|
detections against persisted `vector_features`, persists QA/QC rows and exports
|
|
the detection GeoJSON. It never seeds demo detections, enables fixture mode,
|
|
fetches live providers or downloads model weights. Configured-YOLO model class
|
|
labels are normalized to lowercase for filtering and persisted detections while
|
|
the original model label is retained in detection provenance. Raster tile
|
|
manifests generated for AI handoff include source CRS metadata so pixel-space
|
|
model outputs can be transformed to WGS84 GeoJSON coordinates. Current V1 upload
|
|
support is limited to GeoTIFF-style rasters and GeoJSON/JSON reference vectors.
|
|
|
|
For model-quality calibration, run the confidence sweep wrapper:
|
|
|
|
```bash
|
|
REAL_RASTER_PATH=/mnt/user/appdata/geointel/storage/operator-data/geel_orthophoto_wms_512.tif \
|
|
REAL_REFERENCE_VECTOR_PATH=/mnt/user/appdata/geointel/storage/operator-data/geel_grb_gbg_buildings.geojson \
|
|
CALIBRATION_THRESHOLDS="0.50 0.35 0.25 0.15" \
|
|
bash scripts/run_detection_calibration_sweep.sh http://192.168.10.150:1202
|
|
```
|
|
|
|
The sweep creates one real persisted workflow run per threshold, fetches the
|
|
persisted `QualityCheck`/`Metric` rows and writes a `calibration_summary.json`
|
|
with detection count, score, precision, recall, F1, mean IoU and false
|
|
positive/negative counts. It is intended to tune confidence/IoU/model choices,
|
|
not to add new inference behavior.
|
|
|
|
To inspect the evidence behind a calibration run, export the persisted QA
|
|
evidence bundle:
|
|
|
|
```bash
|
|
CALIBRATION_SUMMARY_PATH=/mnt/user/appdata/geointel/artifacts/detection-calibration/20260707T002103Z/calibration_summary.json \
|
|
bash scripts/export_detection_calibration_evidence.sh http://192.168.10.150:1202
|
|
```
|
|
|
|
The bundle writes combined QA evidence GeoJSON plus a standalone HTML/SVG review
|
|
artifact that separates matched detections, matched references, false positives
|
|
and false negatives by role. It reads existing persisted `QualityCheck` evidence
|
|
only and does not rerun inference.
|
|
|
|
### Run backend
|
|
|
|
```bash
|
|
cd backend
|
|
python -m uvicorn app.main:app --reload
|
|
```
|
|
|
|
### Run backend tests
|
|
|
|
```bash
|
|
cd backend
|
|
python -m pytest
|
|
```
|
|
|
|
For warning-sensitive release checks, the backend is expected to pass with Python deprecation warnings promoted to errors for the timestamp-heavy service paths:
|
|
|
|
```bash
|
|
cd backend
|
|
python -m pytest -W error::DeprecationWarning tests/test_geojson_dataset_service.py tests/test_qa_service.py tests/test_sprint7a_persistence_foundation.py tests/test_sprint8c_detection_visualization_qa.py tests/test_sprint9_segmentation_foundation.py tests/test_vector_operations_service.py
|
|
```
|
|
|
|
The repository readiness gate now applies the same warning policy to the full backend suite:
|
|
|
|
```bash
|
|
bash scripts/run_readiness_check.sh
|
|
```
|
|
|
|
That readiness gate also runs the API contract smoke check before backend/frontend compilation and tests.
|
|
|
|
### Golden QA/QC benchmark
|
|
|
|
Sprint 12 includes a deterministic QA/QC regression benchmark using explicit fixture data:
|
|
|
|
```bash
|
|
python scripts/run_golden_qa_benchmark.py
|
|
```
|
|
|
|
Machine-readable output:
|
|
|
|
```bash
|
|
python scripts/run_golden_qa_benchmark.py --json
|
|
```
|
|
|
|
Shell wrapper used by release-readiness checks:
|
|
|
|
```bash
|
|
bash scripts/verify_golden_qa_benchmark.sh
|
|
```
|
|
|
|
The benchmark compares `fixtures/golden/predicted_buildings.geojson` against `fixtures/golden/reference_buildings.geojson` and fails on metric drift. Expected baseline:
|
|
|
|
- precision: `0.5`
|
|
- recall: `0.5`
|
|
- F1: `0.5`
|
|
- mean IoU: `0.8339768339761133`
|
|
- false positives: `1`
|
|
- false negatives: `1`
|
|
|
|
The command uses existing QA/QC service logic and verifies `QualityCheck`/`Metric` persistence through an in-memory test session. It does not require live providers, AI models, Docker or PostGIS.
|
|
|
|
`scripts/run_readiness_check.sh` runs this benchmark automatically, so any
|
|
change that alters the golden QA/QC metric baseline must update the fixture and
|
|
expected metrics deliberately.
|
|
|
|
### Demo workflow seed
|
|
|
|
Sprint 15 adds an explicit offline demo workflow seed. It creates or returns a
|
|
demo project, AOI, fixture reference buildings, fixture candidate buildings and
|
|
a persisted QA/QC result. It does not fetch live GRB/OSM data and does not run
|
|
AI inference.
|
|
|
|
API:
|
|
|
|
```bash
|
|
curl -X POST http://localhost:1202/api/v1/demo/workflow
|
|
```
|
|
|
|
CLI:
|
|
|
|
```bash
|
|
python scripts/seed_demo_workflow.py --json
|
|
```
|
|
|
|
In Docker Compose on a LAN host:
|
|
|
|
```bash
|
|
curl -X POST http://192.168.10.150:1202/api/v1/demo/workflow
|
|
```
|
|
|
|
### QA/QC result listing
|
|
|
|
Persisted project quality checks and metric rows can be listed with:
|
|
|
|
```bash
|
|
curl http://localhost:1202/api/v1/projects/{project_id}/quality-checks
|
|
```
|
|
|
|
The frontend QA/QC Results panel uses this endpoint after loading the demo
|
|
workflow or running QA.
|
|
|
|
### Export foundation
|
|
|
|
Persisted exports can be created from the existing workbench state:
|
|
|
|
```bash
|
|
curl -X POST http://localhost:1202/api/v1/exports/metadata \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"project_id":"PROJECT_UUID"}'
|
|
```
|
|
|
|
Vector dataset GeoJSON export:
|
|
|
|
```bash
|
|
curl -X POST http://localhost:1202/api/v1/exports/geojson \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"export_kind":"dataset","dataset_id":"DATASET_UUID"}'
|
|
```
|
|
|
|
Detection or segmentation run GeoJSON export:
|
|
|
|
```bash
|
|
curl -X POST http://localhost:1202/api/v1/exports/geojson \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"export_kind":"detection_run","analysis_run_id":"ANALYSIS_RUN_UUID"}'
|
|
```
|
|
|
|
List and inspect exports:
|
|
|
|
```bash
|
|
curl http://localhost:1202/api/v1/exports/projects/PROJECT_UUID/exports
|
|
curl http://localhost:1202/api/v1/exports/EXPORT_UUID/content
|
|
```
|
|
|
|
Download an artifact as a browser/file response:
|
|
|
|
```bash
|
|
curl -OJ http://localhost:1202/api/v1/exports/EXPORT_UUID/download
|
|
```
|
|
|
|
Create a lightweight HTML project report artifact:
|
|
|
|
```bash
|
|
curl -X POST http://localhost:1202/api/v1/exports/report \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"project_id":"PROJECT_UUID"}'
|
|
```
|
|
|
|
The report contains project, dataset, QA/QC summary and export history state
|
|
only. It is not a PDF designer and does not add a separate reporting module.
|
|
|
|
After rebuilding a Docker/LAN deployment, verify the end-to-end demo and export
|
|
flow through the browser-facing frontend proxy:
|
|
|
|
```bash
|
|
bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202
|
|
```
|
|
|
|
The script seeds the explicit demo workflow, verifies persisted QA/QC results,
|
|
creates metadata/report/vector GeoJSON exports, lists exports and downloads the
|
|
JSON/GeoJSON/HTML artifacts.
|
|
|
|
### Backend import smoke
|
|
|
|
```bash
|
|
cd backend
|
|
python -c "from app.main import app; print(app.title)"
|
|
```
|
|
|
|
### Dockerized backend
|
|
|
|
```bash
|
|
docker compose up --build backend db
|
|
```
|
|
|
|
The Docker Compose stack does not require a root `.env` file for the default local runtime. The database service exposes a container-internal Postgres healthcheck, and the backend also runs `docker_start.sh`, which retries an actual SQL `SELECT 1` connection before running `python -m alembic upgrade head` and starting Uvicorn.
|
|
|
|
PostGIS is not published on the host `5432` port by default. This avoids conflicts with existing Postgres/PostGIS services on NAS or server hosts. The backend connects over Docker networking with `db:5432`.
|
|
|
|
Backend and frontend Docker build contexts exclude dependency folders, build outputs and Python bytecode caches via `.dockerignore`.
|
|
|
|
The Docker Compose frontend is published at `http://localhost:1202`.
|
|
|
|
Compose healthchecks are enabled for all runtime services:
|
|
|
|
- `db` uses `pg_isready`.
|
|
- `backend` checks `http://127.0.0.1:8000/health` inside the container.
|
|
- `frontend` checks `http://127.0.0.1/health` through nginx, which also verifies the frontend-to-backend proxy path.
|
|
|
|
The frontend waits for a healthy backend before starting. Check runtime state:
|
|
|
|
```bash
|
|
docker compose ps
|
|
docker compose logs --tail=80 backend
|
|
docker compose logs --tail=80 frontend
|
|
```
|
|
|
|
The backend Docker image installs the approved GIS runtime extra (`.[gis]`) so
|
|
browser-facing Docker deployments can report raster/vector processing
|
|
capabilities accurately:
|
|
|
|
- `rasterio`
|
|
- `numpy`
|
|
- `pillow`
|
|
- `geopandas`
|
|
- `pyogrio`
|
|
- GDAL/GEOS/PROJ system libraries
|
|
|
|
After rebuilding the backend image, verify the LAN/browser runtime from the
|
|
repository root:
|
|
|
|
```bash
|
|
bash scripts/verify_gis_runtime.sh http://localhost:1202
|
|
```
|
|
|
|
On a NAS or server host, use the published LAN URL:
|
|
|
|
```bash
|
|
bash scripts/verify_gis_runtime.sh http://192.168.10.150:1202
|
|
```
|
|
|
|
The script calls `/api/v1/system/capabilities` through the frontend proxy and
|
|
fails if `postgis`, `rasterio` or `geopandas` are not reported as available.
|
|
|
|
The backend Docker build also runs:
|
|
|
|
```bash
|
|
python scripts/gis_import_smoke.py
|
|
```
|
|
|
|
Inside the backend Docker build context this resolves to
|
|
`backend/scripts/gis_import_smoke.py`. The root `scripts/gis_import_smoke.py`
|
|
wrapper calls the same smoke locally. The smoke imports `rasterio`, `geopandas`
|
|
and `pyogrio`; if one of those imports fails, the backend image build fails
|
|
before deployment.
|
|
|
|
### Live Docker/PostGIS migration smoke
|
|
|
|
Sprint 11 validates the real PostGIS runtime path with the existing database service. From the repository root:
|
|
|
|
```bash
|
|
docker compose config
|
|
docker compose up -d db
|
|
DATABASE_URL=postgresql+psycopg://geointel:geointel@localhost:5432/geointel bash scripts/live_migration_smoke.sh
|
|
```
|
|
|
|
The smoke script:
|
|
|
|
- opens a backend SQLAlchemy connection and runs `SELECT 1`
|
|
- runs `alembic upgrade head`
|
|
- checks `PostGIS_Version()` after migrations have created the extension
|
|
- verifies one Alembic head
|
|
- verifies required migrated tables and GiST indexes exist
|
|
|
|
Expected local environment:
|
|
|
|
```bash
|
|
DATABASE_URL=postgresql+psycopg://geointel:geointel@localhost:5432/geointel
|
|
```
|
|
|
|
If the database is not reachable, confirm Docker Desktop is running and that port `5432` is not already occupied. To clean up the local database container without deleting the named volume:
|
|
|
|
```bash
|
|
docker compose stop db
|
|
```
|
|
|
|
To remove the local PostGIS volume as well, use only when you explicitly want a fresh database:
|
|
|
|
```bash
|
|
docker compose down -v
|
|
```
|
|
|
|
## Key docs
|
|
- `docs/API_CONTRACTS.md`
|
|
- `docs/DATABASE_IMPLEMENTATION_PLAN.md`
|
|
- `docs/DEFINITION_OF_DONE.md`
|
|
- `docs/40-build-launch/SPRINT_1_SCOPE_FREEZE.md`
|
|
|
|
## Raster dependency note
|
|
|
|
Raster metadata and raster operations depend on local GDAL/rasterio availability.
|
|
|
|
To enable raster processing locally:
|
|
|
|
```bash
|
|
python -m pip install rasterio
|
|
```
|
|
|
|
If `rasterio` is unavailable:
|
|
- raster metadata responses return `503` with `RASTER_PROCESSING_UNAVAILABLE`
|
|
- raster clip/tile endpoints return explicit unavailable responses
|
|
|
|
## Export report artifact
|
|
|
|
`POST /api/v1/exports/report` creates the existing lightweight
|
|
`project_report_html` artifact. The report is a self-contained HTML handoff
|
|
view rendered from persisted project, dataset, QA/QC and export-history state.
|
|
It includes readiness scorecards, dataset inventory, QA/QC evidence, artifact
|
|
history, known limitations and print-friendly CSS.
|
|
|
|
This remains a simple HTML export. It does not add a PDF designer, report
|
|
builder, live provider fetching or new analysis behavior.
|
|
|
|
## Vector area selection
|
|
|
|
`POST /api/v1/projects/{project_id}/datasets/{dataset_id}/vector/select` runs a
|
|
read-only EPSG:4326 bbox query against persisted PostGIS `vector_features` and
|
|
returns a canonical-envelope GeoJSON FeatureCollection. It is intended for the
|
|
Map workspace area-extract flow and does not create derived datasets or export
|
|
records by itself.
|
|
|
|
`POST /api/v1/projects/{project_id}/datasets/{dataset_id}/vector/select/derive`
|
|
uses the same persisted `vector_features` selection but writes the result as a
|
|
new derived vector dataset. The created dataset uses
|
|
`source="operation:selection"`, `source_name="map_selection"` and
|
|
`derived_from_dataset_id` for source provenance, stores a GeoJSON artifact and
|
|
indexes its features back into `vector_features` for later QA/QC and analysis.
|
|
|
|
`POST /api/v1/exports/geojson` with `export_kind="vector_selection"` persists
|
|
the same bbox-selected FeatureCollection as a normal export record with
|
|
`export_type="vector_selection_geojson"`. This creates a handoff artifact only;
|
|
it does not create a derived dataset.
|
|
|
|
## Helpful repository scripts
|
|
|
|
- `bash scripts/backend_install.sh`
|
|
- `bash scripts/backend_test.sh`
|
|
- `bash scripts/backend_dev.sh`
|
|
- `bash scripts/smoke_backend_import.sh`
|