Document model and reference catalog plan
This commit is contained in:
Codex
@@ -0,0 +1,217 @@
|
||||
# Model And Reference Catalog Design
|
||||
|
||||
## Goal
|
||||
|
||||
Make GeoIntel operationally clearer for V1 users by separating two concepts that currently look too similar in the UI:
|
||||
|
||||
- local AI model assets that can be selected for configured YOLO inference;
|
||||
- official or contextual reference data providers such as GRB, OSM, manual uploads and fixtures.
|
||||
|
||||
This pass must not download model weights, fetch live GRB/OSM data, add training, add auth, or bypass the existing dataset, job, analysis run and detection persistence architecture.
|
||||
|
||||
## Current State
|
||||
|
||||
Detection currently exposes a model capability list with:
|
||||
|
||||
- `yolo-placeholder`;
|
||||
- one configured slot, `yolo-configured`, backed by `YOLO_MODEL_PATH`;
|
||||
- `manual-fixture-detector`.
|
||||
|
||||
This is import-safe and honest, but it does not feel like a model picker. A user can place multiple files in `/app/models`, yet the UI can only show the single configured environment slot.
|
||||
|
||||
Reference data currently exposes provider capabilities for:
|
||||
|
||||
- `grb`;
|
||||
- `osm`;
|
||||
- `manual`;
|
||||
- `fixture`.
|
||||
|
||||
This is architecturally correct, but the UI does not yet make the distinction explicit enough between authoritative reference sources and AI model assets.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- No automatic model downloads.
|
||||
- No bundled production model files in git.
|
||||
- No live GRB WFS or OSM Overpass import.
|
||||
- No direct writes from providers into `vector_features`.
|
||||
- No new database tables for model assets in this pass.
|
||||
- No training studio, model management workflow, LiDAR, reports, copilot or multi-user scope.
|
||||
|
||||
## Options Considered
|
||||
|
||||
### Option A: Keep Only `YOLO_MODEL_PATH`
|
||||
|
||||
Keep the current single configured model slot and document that users must edit `.env`.
|
||||
|
||||
Benefits:
|
||||
|
||||
- smallest code change;
|
||||
- preserves all existing contracts.
|
||||
|
||||
Drawbacks:
|
||||
|
||||
- poor operator experience;
|
||||
- no visible list of available local model files;
|
||||
- users cannot tell whether the model directory contains other usable files.
|
||||
|
||||
### Option B: Filesystem-Backed Model Asset Catalog
|
||||
|
||||
Scan a configured model directory, expose local model files through an API, and let the UI select one asset for `yolo-configured` runs.
|
||||
|
||||
Benefits:
|
||||
|
||||
- aligns with the current runtime model mount (`/app/models`);
|
||||
- no database migration;
|
||||
- no downloads or fake model metadata;
|
||||
- can show file existence, size, checksum and active environment model;
|
||||
- keeps actual inference inside `DetectionService` and `YoloDetectionAdapter`.
|
||||
|
||||
Drawbacks:
|
||||
|
||||
- metadata is limited unless optional sidecar files are added later;
|
||||
- model classes are not guaranteed without loading the model.
|
||||
|
||||
### Option C: Persisted Model Registry
|
||||
|
||||
Create database tables for model registry records, model versions, model artifacts and model lifecycle state.
|
||||
|
||||
Benefits:
|
||||
|
||||
- strong long-term foundation for training studio and MLOps;
|
||||
- full metadata and auditability.
|
||||
|
||||
Drawbacks:
|
||||
|
||||
- too broad for V1;
|
||||
- adds migration and lifecycle complexity before runtime needs justify it;
|
||||
- risks distracting from core GIS workflow completion.
|
||||
|
||||
## Recommended Approach
|
||||
|
||||
Use Option B.
|
||||
|
||||
Add a filesystem-backed model asset catalog for local runtime model files. It should scan `YOLO_MODELS_DIR`, defaulting to `/app/models`, and fall back to the parent directory of `YOLO_MODEL_PATH` when appropriate. It should only report local files with known model suffixes such as `.pt`, `.onnx` and `.engine`.
|
||||
|
||||
The catalog must be read-only. It must never download, create, mutate, move or delete model files.
|
||||
|
||||
Detection runs should still use `model_id="yolo-configured"` for the real YOLO execution path, but may include a selected `model_asset_id`. The backend resolves that ID to a path inside the configured model directory and uses that path for the run. This avoids arbitrary path injection while keeping the existing detection contract compatible.
|
||||
|
||||
## Backend Design
|
||||
|
||||
Create `ModelAssetCatalogService`.
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- resolve the model directory from settings;
|
||||
- scan known model file suffixes;
|
||||
- return deterministic asset IDs derived from file names;
|
||||
- compute SHA-256 and size for visible provenance;
|
||||
- mark which asset matches the active `YOLO_MODEL_PATH`;
|
||||
- resolve a selected asset ID to a local path;
|
||||
- reject missing, unknown or out-of-directory model assets.
|
||||
|
||||
Add schemas:
|
||||
|
||||
- `ModelAssetRead`;
|
||||
- `ModelAssetListResponse`.
|
||||
|
||||
Add endpoint:
|
||||
|
||||
- `GET /api/v1/detection/model-assets`
|
||||
|
||||
Extend existing endpoints without breaking older clients:
|
||||
|
||||
- `GET /api/v1/detection/yolo/preflight` accepts optional `model_asset_id`;
|
||||
- `POST /api/v1/detection/run` accepts optional `model_asset_id`.
|
||||
|
||||
When `model_asset_id` is supplied, `DetectionService` should use a settings copy with `yolo_model_path` replaced by the resolved asset path. The run parameters should persist the selected asset ID and path for reproducibility.
|
||||
|
||||
## Frontend Design
|
||||
|
||||
Detection Lab should show:
|
||||
|
||||
- model capability cards;
|
||||
- a local model asset picker for configured YOLO;
|
||||
- active model indicator;
|
||||
- asset size and checksum prefix;
|
||||
- selected asset passed to preflight and detection run;
|
||||
- clear warning that GeoIntel does not download weights.
|
||||
|
||||
Provider panel should show:
|
||||
|
||||
- official reference source catalog;
|
||||
- GRB as authoritative but not configured for live fetch;
|
||||
- OSM as contextual and not configured for live fetch;
|
||||
- manual uploads as the configured way to add real reference datasets now;
|
||||
- fixtures as demo/test only.
|
||||
|
||||
## Data Flow
|
||||
|
||||
```text
|
||||
/app/models/*.pt
|
||||
-> ModelAssetCatalogService
|
||||
-> GET /api/v1/detection/model-assets
|
||||
-> Detection Lab model asset picker
|
||||
-> POST /api/v1/detection/run model_id=yolo-configured + model_asset_id
|
||||
-> DetectionService resolves local path
|
||||
-> YoloDetectionAdapter loads selected local model
|
||||
-> Job + AnalysisRun + Detection persistence
|
||||
```
|
||||
|
||||
Reference data remains:
|
||||
|
||||
```text
|
||||
Provider registry
|
||||
-> capabilities/status/limitations
|
||||
-> manual upload or future provider import
|
||||
-> DatasetService / VectorFeatureService
|
||||
-> vector_features
|
||||
-> detection/segmentation QA
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
- Unknown `model_asset_id`: `DETECTION_MODEL_ASSET_NOT_FOUND`.
|
||||
- Model asset outside configured directory: `DETECTION_MODEL_ASSET_INVALID`.
|
||||
- Missing configured YOLO dependencies: existing dependency unavailable behavior.
|
||||
- Missing tile manifest: existing tile manifest required behavior.
|
||||
- Missing model file after catalog resolution: existing model unavailable behavior.
|
||||
|
||||
## Tests
|
||||
|
||||
Backend tests should cover:
|
||||
|
||||
- catalog lists only supported local model files;
|
||||
- catalog marks the active model;
|
||||
- checksum and size are reported;
|
||||
- invalid assets are ignored;
|
||||
- unknown asset ID fails cleanly;
|
||||
- selected model asset is persisted in job and analysis run parameters;
|
||||
- model assets endpoint uses canonical envelope;
|
||||
- preflight accepts selected asset without model downloads.
|
||||
|
||||
Frontend tests should cover:
|
||||
|
||||
- Detection Lab exposes local model asset selection;
|
||||
- selected model asset is sent to run and preflight requests;
|
||||
- Provider Panel copy distinguishes reference providers from model assets.
|
||||
|
||||
## Documentation
|
||||
|
||||
Update:
|
||||
|
||||
- `docs/API_CONTRACTS.md`;
|
||||
- `docs/AI_PIPELINES.md`;
|
||||
- `backend/README.md`;
|
||||
- `frontend/README.md`;
|
||||
- `docs/CODEX_EXECUTION_LOG.md`;
|
||||
- `CHANGELOG.md`;
|
||||
- `docs/TODO.md`.
|
||||
|
||||
## Future Work
|
||||
|
||||
- Sidecar model metadata files, for example `model.pt.json`, for class names, source, license and intended task;
|
||||
- optional model compatibility smoke per selected asset;
|
||||
- persisted model registry after V1 foundation is stable;
|
||||
- live GRB/OSM imports through provider contracts;
|
||||
- official reference dataset browser after live provider imports exist.
|
||||
Reference in New Issue
Block a user