Add real data detection QA workflow smoke
This commit is contained in:
@@ -137,6 +137,34 @@ detector fixtures or download weights. A zero detection count is acceptable on
|
||||
the synthetic demo raster; production usefulness still requires validation on
|
||||
real georeferenced orthophotos and reference vectors.
|
||||
|
||||
### Real-data detection and QA validation
|
||||
|
||||
The real operational validation path uses operator-provided files rather than
|
||||
demo fixtures:
|
||||
|
||||
```bash
|
||||
REAL_RASTER_PATH=/mnt/user/appdata/geointel/data/orthophoto.tif \
|
||||
REAL_REFERENCE_VECTOR_PATH=/mnt/user/appdata/geointel/data/reference-buildings.geojson \
|
||||
bash scripts/verify_real_data_detection_qa_workflow.sh http://192.168.10.150:1202
|
||||
```
|
||||
|
||||
The script verifies the full persisted chain:
|
||||
|
||||
- source raster upload with CRS and bounds metadata;
|
||||
- reference building vector upload as `dataset_role=reference`;
|
||||
- raster inspect and tile manifest generation;
|
||||
- local model asset selection and read-only YOLO preflight;
|
||||
- configured-YOLO detection run through Job, AnalysisRun and Detection rows;
|
||||
- detection GeoJSON generated from persisted geometry;
|
||||
- detection QA against persisted reference `vector_features` with persisted
|
||||
`QualityCheck` and `Metric` rows;
|
||||
- detection run GeoJSON export.
|
||||
|
||||
It refuses to run without a real GeoTIFF-style raster and GeoJSON/JSON reference
|
||||
vector. It does not seed demo data, use `fixture_mode`, fetch live providers or
|
||||
download model weights. A zero detection count is valid as runtime evidence but
|
||||
does not prove the model is useful for the target imagery.
|
||||
|
||||
### Sprint 8C detection visualization and QA status
|
||||
|
||||
Sprint 8C makes persisted detections reviewable:
|
||||
|
||||
@@ -1,3 +1,30 @@
|
||||
## Sprint 121 Real data detection and QA workflow smoke (2026-07-07)
|
||||
|
||||
Changed:
|
||||
- Added `scripts/verify_real_data_detection_qa_workflow.sh` for live-runtime validation with operator-provided real GIS inputs.
|
||||
- The smoke creates a project, uploads a real GeoTIFF-style raster as a source dataset, uploads a real reference-building GeoJSON/JSON as `dataset_role=reference`, validates CRS/bounds/features, tiles the raster, selects a mounted local model asset, runs read-only YOLO preflight, runs configured YOLO detection, runs detection QA against persisted `vector_features`, and exports the detection run GeoJSON.
|
||||
- Registered the smoke in `scripts/run_readiness_check.sh` as a syntax check only, so normal readiness does not require real orthophotos, reference vectors, optional AI dependencies or model files.
|
||||
- Documented Tower usage and limitations in `scripts/README.md`, `backend/README.md`, `docs/AI_PIPELINES.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
||||
|
||||
Validation:
|
||||
- RED: `python -m pytest backend/tests/test_sprint121_real_data_detection_qa_smoke.py -q` failed while `scripts/verify_real_data_detection_qa_workflow.sh` did not exist.
|
||||
- `python -m pytest backend/tests/test_sprint121_real_data_detection_qa_smoke.py -q` passed: 1 test.
|
||||
- `bash -n scripts/verify_real_data_detection_qa_workflow.sh` passed.
|
||||
- `bash scripts/verify_real_data_detection_qa_workflow.sh --help` passed and printed required `REAL_RASTER_PATH` and `REAL_REFERENCE_VECTOR_PATH` usage.
|
||||
- `python -m compileall backend/app` passed.
|
||||
- `python scripts/smoke_docs.py` passed.
|
||||
- `bash scripts/run_readiness_check.sh` passed: 385 backend tests, Alembic head check, frontend typecheck, frontend production build and shell syntax checks.
|
||||
- `cd backend && python -m alembic upgrade head --sql` passed.
|
||||
- Missing-input guard passed: `bash scripts/verify_real_data_detection_qa_workflow.sh http://localhost:1202` returned exit code 2 and printed usage.
|
||||
- Local `docker compose config` could not run in this Windows Codex environment because the `docker` command is not installed.
|
||||
|
||||
Limitations:
|
||||
- The full real-data smoke was not executed in this Codex workspace because no operator-provided real GeoTIFF and reference GeoJSON were found locally.
|
||||
- The script enforces real inputs and never seeds demo data, enables fixture detections, fetches live GRB/OSM/Sentinel data or downloads model weights.
|
||||
|
||||
Next recommended pass:
|
||||
- Place a target orthophoto/GeoTIFF and matching reference-building GeoJSON under the Tower appdata path and run `REAL_RASTER_PATH=... REAL_REFERENCE_VECTOR_PATH=... bash scripts/verify_real_data_detection_qa_workflow.sh http://192.168.10.150:1202`.
|
||||
|
||||
## Sprint 120 Model asset detection workflow smoke (2026-07-06)
|
||||
|
||||
Changed:
|
||||
|
||||
@@ -91,6 +91,7 @@ This file now starts with the current implementation status. Older preparation/b
|
||||
- [x] Add QA/QC and Exports usability layout pass with calmer evidence review and handoff artifact scanning.
|
||||
- [x] Add AI Labs Detection/Segmentation hierarchy and result density polish.
|
||||
- [x] Add Export/System handoff hierarchy and provider registry density polish.
|
||||
- [x] Add operator-provided real raster/reference detection + QA workflow smoke.
|
||||
- [ ] Validate the configured building model on a real georeferenced Kempen orthophoto/GeoTIFF with persisted reference vectors and QA/QC metrics.
|
||||
|
||||
## Sprint 8 status
|
||||
|
||||
Reference in New Issue
Block a user