geointel/docs/AI_PIPELINES.md

# AI Pipelines

## 1. Object Detection Pipeline

```text
Raster dataset
↓
Clip to analysis area
↓
Tile raster
↓
Normalize tiles
↓
Run YOLO/PyTorch inference
↓
Filter by confidence
↓
Convert pixel boxes to geospatial polygons
↓
Merge overlapping detections
↓
Store in PostGIS
↓
Expose as GeoJSON layer
↓
Run QA/QC if reference data exists
```

### Sprint 8 foundation status

Sprint 8 implements the detection persistence and execution boundary only:

- `detections` are first-class PostGIS records linked to project, dataset, job and analysis run.
- `analysis_runs` remain separate from jobs and store model metadata, parameters, result summaries and lifecycle status.
- `yolo-placeholder` reports `not_configured`; no YOLO/PyTorch model is downloaded or executed.
- `manual-fixture-detector` is test/demo-only and persists detections only when `fixture_mode=true` and fixture detections are explicitly supplied.
- Normal application behavior must not create fake detections.

### Sprint 8B configured YOLO status

Sprint 8B adds an import-safe real YOLO adapter path:

- `ultralytics` and `torch` are optional backend extras, not default runtime dependencies.
- `yolo-configured` reports `not_configured` until `YOLO_ENABLED=true`, `YOLO_MODEL_PATH` points to an existing local model file and optional AI dependencies are installed.
- GeoIntel never downloads model weights automatically.
- Real YOLO inference uses an existing raster tile manifest generated by the raster tile operation.
- YOLO raster tiles are normalized to RGB for inference when the tile artifact is not already a 3-band RGB image; the persisted georeferencing still comes from the tile manifest.
- YOLO pixel boxes are converted to EPSG:4326 detection polygons from tile transform or tile bounds metadata.
- YOLO class labels are normalized to lowercase for persisted detection records and filtering, while the original model label remains available in detection provenance.
- Detection runs remain synchronous behind the existing job and analysis-run persistence boundary for Sprint 8B.

### Sprint 13 YOLO operational preflight

Sprint 13 adds a local preflight command for configured YOLO operation:

```bash
python scripts/yolo_preflight.py --model-path /absolute/path/to/model.pt --tile-manifest-path /absolute/path/to/manifest.json
```

For machines without optional AI dependencies, path and manifest checks can be exercised without pretending inference is available:

```bash
python scripts/yolo_preflight.py --model-path /absolute/path/to/model.pt --tile-manifest-path /absolute/path/to/manifest.json --assume-dependencies --json
```

The preflight checks:

- `YOLO_ENABLED` / explicit enabled state;
- optional dependency availability unless `--assume-dependencies` is used;
- local model file existence;
- tile manifest JSON validity;
- tile count against `YOLO_MAX_TILES`;
- referenced tile file existence.

JSON output also reports runtime diagnostics: whether dependencies were assumed,
the configured model directory, `YOLO_CONFIG_DIR`, installed `torch` and
`ultralytics` versions, and CUDA availability when dependency checks pass.

The preflight does not load the model, does not import Ultralytics unless dependency discovery requires package metadata, does not run inference and never downloads model weights.

Sprint 25 adds an explicit local model compatibility smoke:

```bash
python scripts/yolo_preflight.py --model-path /absolute/path/to/model.pt --tile-manifest-path /absolute/path/to/manifest.json --check-model-load --json
```

`--check-model-load` requires real optional AI dependencies and an existing local
model file. It loads that local file through the configured adapter to verify
Ultralytics/PyTorch compatibility, but it still does not run tile prediction and
does not download weights. It cannot be combined with `--assume-dependencies`
because that would turn the smoke into a false positive.

Docker and Unraid runtime support remains opt-in. Set `GEOINTEL_INSTALL_AI=true`
at build time to install the backend `.[gis,ai]` extra into the container. Leave
it unset or `false` for the default GIS-only image. Runtime model files should be
mounted into the container, for example `/app/models/local-model.pt`, and enabled
with `YOLO_ENABLED=true` plus `YOLO_MODEL_PATH=/app/models/local-model.pt`.
GeoIntel never downloads weights automatically.

Environment variables:

- `GEOINTEL_INSTALL_AI`
- `YOLO_ENABLED`
- `YOLO_MODELS_DIR`
- `YOLO_MODEL_PATH`
- `YOLO_MODEL_ID`
- `YOLO_MODEL_DISPLAY_NAME`
- `YOLO_MODEL_VERSION`
- `YOLO_DEVICE`
- `YOLO_IMAGE_SIZE`
- `YOLO_MAX_TILES`
- `YOLO_BATCH_SIZE`

### Local model asset catalog

GeoIntel can list local runtime model files mounted into the backend model
directory through `GET /api/v1/detection/model-assets`. The catalog is
filesystem-backed and read-only: it reports existing `.pt`, `.onnx` and
`.engine` files, size, checksum and whether the file matches `YOLO_MODEL_PATH`.

Detection runs still use `model_id="yolo-configured"` for the configured YOLO
execution path. A selected `model_asset_id` can be supplied to use one specific
cataloged file for that run. The backend resolves the ID to a local path and
persists the selected asset metadata in Job/AnalysisRun parameters. GeoIntel
does not download weights or accept arbitrary model paths from the browser.

Operational runtime validation can be run against Docker/Tower with:

```bash
bash scripts/verify_model_asset_detection_workflow.sh http://192.168.10.150:1202
```

The smoke seeds the explicit offline demo raster, creates a tile manifest,
selects a local model asset, checks read-only preflight, runs the existing
configured-YOLO detection endpoint and verifies persisted AnalysisRun,
Detection list and Detection GeoJSON outputs. It intentionally does not inject
detector fixtures or download weights. A zero detection count is acceptable on
the synthetic demo raster; production usefulness still requires validation on
real georeferenced orthophotos and reference vectors.

### Real-data detection and QA validation

The real operational validation path uses operator-provided files rather than
demo fixtures:

```bash
REAL_RASTER_PATH=/mnt/user/appdata/geointel/data/orthophoto.tif \
REAL_REFERENCE_VECTOR_PATH=/mnt/user/appdata/geointel/data/reference-buildings.geojson \
bash scripts/verify_real_data_detection_qa_workflow.sh http://192.168.10.150:1202
```

The script verifies the full persisted chain:

- source raster upload with CRS and bounds metadata;
- reference building vector upload as `dataset_role=reference`;
- raster inspect and tile manifest generation;
- local model asset selection and read-only YOLO preflight;
- configured-YOLO detection run through Job, AnalysisRun and Detection rows;
- detection GeoJSON generated from persisted geometry;
- detection QA against persisted reference `vector_features` with persisted
  `QualityCheck` and `Metric` rows;
- detection run GeoJSON export.

It refuses to run without a real GeoTIFF-style raster and GeoJSON/JSON reference
vector. It does not seed demo data, use `fixture_mode`, fetch live providers or
download model weights. A zero detection count is valid as runtime evidence only
when the selected model genuinely returns no usable detections after canonical
class filtering; it does not prove the model is useful for the target imagery.

### Sprint 8C detection visualization and QA status

Sprint 8C makes persisted detections reviewable:

- Detection runs can be listed and selected.
- Persisted detections can be listed and filtered by run, dataset, class and minimum confidence.
- Persisted detection geometries can be returned as GeoJSON FeatureCollections for MapLibre display.
- Detection QA compares candidate detection geometries against persisted reference `vector_features`.
- QA results reuse `quality_checks` and `metrics`; no parallel QA persistence system is introduced.
- Segmentation remains out of scope for Sprint 8C.

## 2. Tile Metadata

Elke tile moet opslaan:

- tile path
- parent raster id
- pixel window
- geospatial bounds
- transform
- CRS, and the manifest must also carry source CRS metadata
- tile size
- overlap

Zonder tile metadata kunnen modeloutputs niet correct teruggeprojecteerd worden.

## 3. Detection Output Contract

Elke detectie bevat:

- class_name
- confidence
- bbox pixel coords
- source tile
- geospatial polygon
- model id/version
- analysis run id

## 4. Segmentation Pipeline

```text
Raster dataset
↓
Clip/tile
↓
Run segmentation model
↓
Generate mask
↓
Georeference mask
↓
Polygonize mask
↓
Simplify/clean geometries
↓
Store polygons + mask path
↓
Expose as map layer
```

### Sprint 9 segmentation foundation status

Sprint 9 implements the segmentation persistence and review boundary only:

- `segmentations` are first-class PostGIS records linked to project, dataset, job and analysis run.
- PostGIS MultiPolygon geometry in EPSG:4326 is authoritative for map display, QA and GeoJSON output.
- Mask paths are persisted as artifact/provenance references, not authoritative feature state.
- `segmentation-placeholder`, `yolo-seg-configured` and `sam-configured` report `not_configured`.
- `fixture-segmenter` is test/demo-only and persists segmentations only when `fixture_mode=true` and fixture segmentations are explicitly supplied.
- Segmentation QA compares persisted segmentation geometries against persisted reference `vector_features`.
- QA results reuse `quality_checks` and `metrics`; no parallel QA system is introduced.
- GeoIntel does not install SAM, run YOLO-seg, download model weights or fake production segmentations in Sprint 9.

## 5. Change Detection Pipeline

Fase 1: vector/detection based.

```text
Run A detections
+
Run B detections
↓
Spatial matching
↓
added / removed / changed
↓
Change polygons
↓
Metrics
```

Fase 2: raster index based.

```text
Raster A index
+
Raster B index
↓
Difference raster
↓
Threshold
↓
Polygonize changed zones
```

Fase 3: segmentation based.

```text
Mask A
+
Mask B
↓
Class difference
↓
Change polygons
```

## 6. Model Strategy

V1:

- gebruik een bestaande YOLO-integratie met configureerbaar modelpad
- demo-model mag lokaal worden geplaatst in `models/`
- code moet ook zonder model kunnen starten, maar detection job moet dan duidelijke fout geven

V2:

- SAM/YOLO segmentation

V3:

- annotation export
- finetuning

## 7. Reproduceerbaarheid

Elke analysis run moet bewaren:

- model id
- model version
- parameters
- confidence threshold
- tile size
- overlap
- input dataset id
- code path/version indien mogelijk