Add local model asset catalog
This commit is contained in:
@@ -100,6 +100,7 @@ Environment variables:
|
||||
|
||||
- `GEOINTEL_INSTALL_AI`
|
||||
- `YOLO_ENABLED`
|
||||
- `YOLO_MODELS_DIR`
|
||||
- `YOLO_MODEL_PATH`
|
||||
- `YOLO_MODEL_ID`
|
||||
- `YOLO_MODEL_DISPLAY_NAME`
|
||||
@@ -109,6 +110,19 @@ Environment variables:
|
||||
- `YOLO_MAX_TILES`
|
||||
- `YOLO_BATCH_SIZE`
|
||||
|
||||
### Local model asset catalog
|
||||
|
||||
GeoIntel can list local runtime model files mounted into the backend model
|
||||
directory through `GET /api/v1/detection/model-assets`. The catalog is
|
||||
filesystem-backed and read-only: it reports existing `.pt`, `.onnx` and
|
||||
`.engine` files, size, checksum and whether the file matches `YOLO_MODEL_PATH`.
|
||||
|
||||
Detection runs still use `model_id="yolo-configured"` for the configured YOLO
|
||||
execution path. A selected `model_asset_id` can be supplied to use one specific
|
||||
cataloged file for that run. The backend resolves the ID to a local path and
|
||||
persists the selected asset metadata in Job/AnalysisRun parameters. GeoIntel
|
||||
does not download weights or accept arbitrary model paths from the browser.
|
||||
|
||||
### Sprint 8C detection visualization and QA status
|
||||
|
||||
Sprint 8C makes persisted detections reviewable:
|
||||
|
||||
@@ -636,12 +636,51 @@ Returns object-detection model capability descriptors.
|
||||
}
|
||||
```
|
||||
|
||||
### GET `/api/v1/detection/model-assets`
|
||||
|
||||
Returns local runtime model files discovered in the configured model directory.
|
||||
This is a read-only catalog. GeoIntel never downloads, creates, mutates or
|
||||
deletes model weights from this endpoint.
|
||||
|
||||
The backend scans `YOLO_MODELS_DIR` (default `/app/models`) and reports
|
||||
supported local model files such as `.pt`, `.onnx` and `.engine`. The active
|
||||
model is the file matching `YOLO_MODEL_PATH`.
|
||||
|
||||
Response data:
|
||||
|
||||
```json
|
||||
{
|
||||
"items": [
|
||||
{
|
||||
"model_asset_id": "building-detector-pt",
|
||||
"filename": "building-detector.pt",
|
||||
"display_name": "building-detector",
|
||||
"model_path": "/app/models/building-detector.pt",
|
||||
"suffix": ".pt",
|
||||
"framework": "ultralytics/pytorch",
|
||||
"task_type": "object_detection",
|
||||
"size_bytes": 123456,
|
||||
"sha256": "sha256hex",
|
||||
"active": true,
|
||||
"status": "available",
|
||||
"limitation_message": "Local runtime model asset. GeoIntel will not download or mutate model weights.",
|
||||
"will_download_models": false
|
||||
}
|
||||
],
|
||||
"total": 1,
|
||||
"model_directory": "/app/models"
|
||||
}
|
||||
```
|
||||
|
||||
### GET `/api/v1/detection/yolo/preflight`
|
||||
|
||||
Returns a canonical envelope with read-only configured-YOLO runtime preflight
|
||||
state. Optional query parameters:
|
||||
|
||||
- `tile_manifest_path`: existing raster tile manifest path to validate.
|
||||
- `model_asset_id`: optional local model asset ID from
|
||||
`GET /api/v1/detection/model-assets`; when supplied, preflight validates that
|
||||
asset path instead of the default `YOLO_MODEL_PATH`.
|
||||
- `check_model_load`: default `false`; when `true`, explicitly loads only the
|
||||
configured local model file for compatibility smoke. It never downloads
|
||||
weights and never runs inference.
|
||||
@@ -651,6 +690,7 @@ Response data:
|
||||
```json
|
||||
{
|
||||
"model_id": "yolo-configured",
|
||||
"model_asset_id": null,
|
||||
"model_path": null,
|
||||
"tile_manifest_path": null,
|
||||
"status": "not_configured",
|
||||
@@ -693,6 +733,7 @@ Request:
|
||||
"project_id": "uuid",
|
||||
"dataset_id": "uuid",
|
||||
"model_id": "yolo-placeholder",
|
||||
"model_asset_id": null,
|
||||
"confidence_threshold": 0.5,
|
||||
"class_filter": ["building"],
|
||||
"tile_manifest_path": null,
|
||||
@@ -707,6 +748,12 @@ Sprint 8B configured YOLO mode uses `model_id: "yolo-configured"`. It requires:
|
||||
- backend optional AI dependencies installed with `geointel-backend[ai]`
|
||||
- `tile_manifest_path` pointing to an existing raster tile manifest generated by the raster tile operation
|
||||
|
||||
`model_asset_id` may be supplied with `model_id: "yolo-configured"` to select a
|
||||
specific local model file from the read-only model asset catalog. The backend
|
||||
resolves the ID to a file inside the configured model directory and persists the
|
||||
asset ID, path and SHA-256 in the job and analysis-run parameters for
|
||||
reproducibility. Clients must not submit arbitrary model paths.
|
||||
|
||||
GeoIntel does not download model weights automatically. Configured YOLO runs read existing tile files from the manifest, convert YOLO pixel-space boxes to EPSG:4326 detection polygons and persist detections as first-class records.
|
||||
|
||||
Unavailable model response:
|
||||
@@ -729,6 +776,7 @@ Validation errors:
|
||||
|
||||
- `INVALID_DATASET_TYPE` when the dataset is not raster.
|
||||
- `DETECTION_MODEL_NOT_FOUND` when the model id is unknown.
|
||||
- `DETECTION_MODEL_ASSET_NOT_FOUND` when `model_asset_id` is not present in the configured model directory.
|
||||
- `FIXTURE_MODE_REQUIRED` when `manual-fixture-detector` is requested without `parameters_json.fixture_mode=true`.
|
||||
- `DETECTION_TILE_MANIFEST_REQUIRED` when `yolo-configured` is requested without `tile_manifest_path`.
|
||||
- `DETECTION_TILE_MANIFEST_NOT_FOUND` when the provided manifest path does not exist.
|
||||
|
||||
@@ -1,3 +1,39 @@
|
||||
## Sprint 118 Local model and reference catalog clarity (2026-07-06)
|
||||
|
||||
Changed:
|
||||
- Added a read-only backend model asset catalog through `GET /api/v1/detection/model-assets`.
|
||||
- Added `YOLO_MODELS_DIR` to backend settings, Compose, Unraid env examples and all-in-one runtime startup so `/app/models` is the explicit model catalog directory.
|
||||
- Extended configured YOLO preflight and detection runs with optional `model_asset_id`, resolved server-side against the model asset catalog.
|
||||
- Detection jobs and analysis runs now persist selected model asset ID, path and SHA-256 in parameters for reproducibility.
|
||||
- Detection Lab now loads local model assets, selects the active model by default and lets operators choose a cataloged local model file for `yolo-configured`.
|
||||
- Provider Capabilities now distinguishes GRB/OSM/manual/fixture reference-data sources from AI model choices.
|
||||
- Updated `docs/API_CONTRACTS.md`, `docs/AI_PIPELINES.md`, `backend/README.md`, `frontend/README.md`, `deploy/unraid/README.md`, `scripts/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
||||
- Added design/plan documents under `docs/superpowers/`.
|
||||
|
||||
Validation:
|
||||
- RED: `python -m pytest backend/tests/test_model_asset_catalog.py -q` failed before implementation because `app.services.model_asset_catalog_service` did not exist.
|
||||
- `python -m pytest backend/tests/test_model_asset_catalog.py -q` passed: 5 tests.
|
||||
- RED: `python -m pytest backend/tests/test_sprint118_yolo_preflight_ui.py -q` failed before frontend wiring because the model asset types/API/hook/UI were absent.
|
||||
- `python -m pytest backend/tests/test_sprint118_yolo_preflight_ui.py -q` passed: 2 tests.
|
||||
- RED: runtime config tests failed before `YOLO_MODELS_DIR` was added to env examples, Unraid runtime and `scripts/configure_yolo_model.py`.
|
||||
- `python -m pytest backend/tests/test_docker_runtime_config.py::test_env_example_uses_runtime_env_names_read_by_backend_and_frontend backend/tests/test_docker_runtime_config.py::test_unraid_deploy_passes_ai_build_arg_and_yolo_runtime_env backend/tests/test_sprint119_yolo_model_configuration.py -q` passed: 6 tests.
|
||||
- `python -m compileall backend/app` passed.
|
||||
- `cd backend && python -m pytest -q` passed: 383 tests.
|
||||
- `cd frontend && npm run typecheck` passed.
|
||||
- `cd frontend && npm run build` passed.
|
||||
- `python scripts/audit_api_contracts.py` passed: 81 implemented routes match docs; 2 explicit non-envelope endpoints tracked.
|
||||
- `bash scripts/run_readiness_check.sh` passed: 383 backend tests plus frontend typecheck/build, API contract audit, Alembic head and shell syntax checks.
|
||||
- `cd backend && python -m alembic upgrade head --sql` passed.
|
||||
- `bash -n scripts/live_migration_smoke.sh; bash -n deploy/unraid/run-dockerman-container.sh; bash -n deploy/unraid/all-in-one-start.sh; bash -n scripts/deploy_tower.sh` passed.
|
||||
- Local `docker compose config` could not run because Docker is not installed in this Windows Codex environment; Tower deploy validation remains required.
|
||||
|
||||
Limitations:
|
||||
- The catalog is intentionally filesystem-backed and read-only. It does not download, validate semantic class metadata, train models or manage model lifecycle records in the database.
|
||||
- GRB/OSM remain provider capabilities only; no live external fetching was added.
|
||||
|
||||
Next recommended pass:
|
||||
- Redeploy Tower, verify `/api/v1/detection/model-assets`, confirm Detection Lab shows the local model picker, then continue with real raster/model workflow validation on non-synthetic imagery.
|
||||
|
||||
## Sprint 117 Reusable GIS run and AI runtime opt-in (2026-07-05)
|
||||
|
||||
Changed:
|
||||
|
||||
@@ -81,6 +81,7 @@ This file now starts with the current implementation status. Older preparation/b
|
||||
- [x] Add reusable latest-result mode for repeated Map QA/QC runs without duplicate derived artifacts.
|
||||
- [x] Add opt-in Docker/Unraid AI build/runtime path for local PyTorch/Ultralytics YOLO operation.
|
||||
- [x] Surface configured-YOLO runtime preflight status through the API and Detection Lab UI.
|
||||
- [x] Add read-only local model asset catalog and Detection Lab model-file selection.
|
||||
- [x] Add one-click full GIS workflow action for query, derived dataset, QA/QC and export handoff.
|
||||
- [x] Add QA/QC workspace result hierarchy and filter density polish.
|
||||
- [x] Add Change Detection panel hierarchy and analysis workspace density polish.
|
||||
|
||||
Reference in New Issue
Block a user