Initial GeoIntel V1 foundation
This commit is contained in:
Codex
@@ -0,0 +1,232 @@
|
||||
# AI Pipelines
|
||||
|
||||
## 1. Object Detection Pipeline
|
||||
|
||||
```text
|
||||
Raster dataset
|
||||
↓
|
||||
Clip to analysis area
|
||||
↓
|
||||
Tile raster
|
||||
↓
|
||||
Normalize tiles
|
||||
↓
|
||||
Run YOLO/PyTorch inference
|
||||
↓
|
||||
Filter by confidence
|
||||
↓
|
||||
Convert pixel boxes to geospatial polygons
|
||||
↓
|
||||
Merge overlapping detections
|
||||
↓
|
||||
Store in PostGIS
|
||||
↓
|
||||
Expose as GeoJSON layer
|
||||
↓
|
||||
Run QA/QC if reference data exists
|
||||
```
|
||||
|
||||
### Sprint 8 foundation status
|
||||
|
||||
Sprint 8 implements the detection persistence and execution boundary only:
|
||||
|
||||
- `detections` are first-class PostGIS records linked to project, dataset, job and analysis run.
|
||||
- `analysis_runs` remain separate from jobs and store model metadata, parameters, result summaries and lifecycle status.
|
||||
- `yolo-placeholder` reports `not_configured`; no YOLO/PyTorch model is downloaded or executed.
|
||||
- `manual-fixture-detector` is test/demo-only and persists detections only when `fixture_mode=true` and fixture detections are explicitly supplied.
|
||||
- Normal application behavior must not create fake detections.
|
||||
|
||||
### Sprint 8B configured YOLO status
|
||||
|
||||
Sprint 8B adds an import-safe real YOLO adapter path:
|
||||
|
||||
- `ultralytics` and `torch` are optional backend extras, not default runtime dependencies.
|
||||
- `yolo-configured` reports `not_configured` until `YOLO_ENABLED=true`, `YOLO_MODEL_PATH` points to an existing local model file and optional AI dependencies are installed.
|
||||
- GeoIntel never downloads model weights automatically.
|
||||
- Real YOLO inference uses an existing raster tile manifest generated by the raster tile operation.
|
||||
- YOLO pixel boxes are converted to EPSG:4326 detection polygons from tile transform or tile bounds metadata.
|
||||
- Detection runs remain synchronous behind the existing job and analysis-run persistence boundary for Sprint 8B.
|
||||
|
||||
### Sprint 13 YOLO operational preflight
|
||||
|
||||
Sprint 13 adds a local preflight command for configured YOLO operation:
|
||||
|
||||
```bash
|
||||
python scripts/yolo_preflight.py --model-path /absolute/path/to/model.pt --tile-manifest-path /absolute/path/to/manifest.json
|
||||
```
|
||||
|
||||
For machines without optional AI dependencies, path and manifest checks can be exercised without pretending inference is available:
|
||||
|
||||
```bash
|
||||
python scripts/yolo_preflight.py --model-path /absolute/path/to/model.pt --tile-manifest-path /absolute/path/to/manifest.json --assume-dependencies --json
|
||||
```
|
||||
|
||||
The preflight checks:
|
||||
|
||||
- `YOLO_ENABLED` / explicit enabled state;
|
||||
- optional dependency availability unless `--assume-dependencies` is used;
|
||||
- local model file existence;
|
||||
- tile manifest JSON validity;
|
||||
- tile count against `YOLO_MAX_TILES`;
|
||||
- referenced tile file existence.
|
||||
|
||||
The preflight does not load the model, does not import Ultralytics unless dependency discovery requires package metadata, does not run inference and never downloads model weights.
|
||||
|
||||
Environment variables:
|
||||
|
||||
- `YOLO_ENABLED`
|
||||
- `YOLO_MODEL_PATH`
|
||||
- `YOLO_MODEL_ID`
|
||||
- `YOLO_MODEL_DISPLAY_NAME`
|
||||
- `YOLO_MODEL_VERSION`
|
||||
- `YOLO_DEVICE`
|
||||
- `YOLO_IMAGE_SIZE`
|
||||
- `YOLO_MAX_TILES`
|
||||
- `YOLO_BATCH_SIZE`
|
||||
|
||||
### Sprint 8C detection visualization and QA status
|
||||
|
||||
Sprint 8C makes persisted detections reviewable:
|
||||
|
||||
- Detection runs can be listed and selected.
|
||||
- Persisted detections can be listed and filtered by run, dataset, class and minimum confidence.
|
||||
- Persisted detection geometries can be returned as GeoJSON FeatureCollections for MapLibre display.
|
||||
- Detection QA compares candidate detection geometries against persisted reference `vector_features`.
|
||||
- QA results reuse `quality_checks` and `metrics`; no parallel QA persistence system is introduced.
|
||||
- Segmentation remains out of scope for Sprint 8C.
|
||||
|
||||
## 2. Tile Metadata
|
||||
|
||||
Elke tile moet opslaan:
|
||||
|
||||
- tile path
|
||||
- parent raster id
|
||||
- pixel window
|
||||
- geospatial bounds
|
||||
- transform
|
||||
- CRS
|
||||
- tile size
|
||||
- overlap
|
||||
|
||||
Zonder tile metadata kunnen modeloutputs niet correct teruggeprojecteerd worden.
|
||||
|
||||
## 3. Detection Output Contract
|
||||
|
||||
Elke detectie bevat:
|
||||
|
||||
- class_name
|
||||
- confidence
|
||||
- bbox pixel coords
|
||||
- source tile
|
||||
- geospatial polygon
|
||||
- model id/version
|
||||
- analysis run id
|
||||
|
||||
## 4. Segmentation Pipeline
|
||||
|
||||
```text
|
||||
Raster dataset
|
||||
↓
|
||||
Clip/tile
|
||||
↓
|
||||
Run segmentation model
|
||||
↓
|
||||
Generate mask
|
||||
↓
|
||||
Georeference mask
|
||||
↓
|
||||
Polygonize mask
|
||||
↓
|
||||
Simplify/clean geometries
|
||||
↓
|
||||
Store polygons + mask path
|
||||
↓
|
||||
Expose as map layer
|
||||
```
|
||||
|
||||
### Sprint 9 segmentation foundation status
|
||||
|
||||
Sprint 9 implements the segmentation persistence and review boundary only:
|
||||
|
||||
- `segmentations` are first-class PostGIS records linked to project, dataset, job and analysis run.
|
||||
- PostGIS MultiPolygon geometry in EPSG:4326 is authoritative for map display, QA and GeoJSON output.
|
||||
- Mask paths are persisted as artifact/provenance references, not authoritative feature state.
|
||||
- `segmentation-placeholder`, `yolo-seg-configured` and `sam-configured` report `not_configured`.
|
||||
- `fixture-segmenter` is test/demo-only and persists segmentations only when `fixture_mode=true` and fixture segmentations are explicitly supplied.
|
||||
- Segmentation QA compares persisted segmentation geometries against persisted reference `vector_features`.
|
||||
- QA results reuse `quality_checks` and `metrics`; no parallel QA system is introduced.
|
||||
- GeoIntel does not install SAM, run YOLO-seg, download model weights or fake production segmentations in Sprint 9.
|
||||
|
||||
## 5. Change Detection Pipeline
|
||||
|
||||
Fase 1: vector/detection based.
|
||||
|
||||
```text
|
||||
Run A detections
|
||||
+
|
||||
Run B detections
|
||||
↓
|
||||
Spatial matching
|
||||
↓
|
||||
added / removed / changed
|
||||
↓
|
||||
Change polygons
|
||||
↓
|
||||
Metrics
|
||||
```
|
||||
|
||||
Fase 2: raster index based.
|
||||
|
||||
```text
|
||||
Raster A index
|
||||
+
|
||||
Raster B index
|
||||
↓
|
||||
Difference raster
|
||||
↓
|
||||
Threshold
|
||||
↓
|
||||
Polygonize changed zones
|
||||
```
|
||||
|
||||
Fase 3: segmentation based.
|
||||
|
||||
```text
|
||||
Mask A
|
||||
+
|
||||
Mask B
|
||||
↓
|
||||
Class difference
|
||||
↓
|
||||
Change polygons
|
||||
```
|
||||
|
||||
## 6. Model Strategy
|
||||
|
||||
V1:
|
||||
|
||||
- gebruik een bestaande YOLO-integratie met configureerbaar modelpad
|
||||
- demo-model mag lokaal worden geplaatst in `models/`
|
||||
- code moet ook zonder model kunnen starten, maar detection job moet dan duidelijke fout geven
|
||||
|
||||
V2:
|
||||
|
||||
- SAM/YOLO segmentation
|
||||
|
||||
V3:
|
||||
|
||||
- annotation export
|
||||
- finetuning
|
||||
|
||||
## 7. Reproduceerbaarheid
|
||||
|
||||
Elke analysis run moet bewaren:
|
||||
|
||||
- model id
|
||||
- model version
|
||||
- parameters
|
||||
- confidence threshold
|
||||
- tile size
|
||||
- overlap
|
||||
- input dataset id
|
||||
- code path/version indien mogelijk
|
||||
Reference in New Issue
Block a user