feat: scope detection QA to inference coverage
GeoIntel CI / docs-smoke (push) Canceled after 0s
GeoIntel CI / contract-smoke (push) Canceled after 0s

This commit is contained in:
Codex
2026-07-14 11:25:28 +02:00
parent aae6315981
commit 948e50b5e1
17 changed files with 762 additions and 7 deletions
+9
View File
@@ -544,6 +544,15 @@ Sprint 8C makes persisted detections reviewable:
- Persisted detection geometries can be returned as GeoJSON FeatureCollections for MapLibre display.
- Detection QA compares candidate detection geometries against persisted reference `vector_features`.
- QA results reuse `quality_checks` and `metrics`; no parallel QA persistence system is introduced.
- Configured-YOLO QA derives its evaluation extent from the persisted tile
manifest. Tile bounds are transformed from their explicit source CRS to
EPSG:4326, unioned, and used to clip candidate/reference populations before
canonical footprint-IoU matching. Reference features wholly outside the
imagery presented to the model no longer count as false negatives.
- A separate reference-envelope IoU pass is persisted as
`box_to_footprint_diagnostics`. It quantifies possible matching artifacts from
comparing rectangular detections with irregular building footprints, but is
diagnostic only and never changes canonical QA metrics.
- Segmentation remains out of scope for Sprint 8C.
## 2. Tile Metadata
+23
View File
@@ -872,6 +872,29 @@ Response persists a `quality_check` and `metrics` rows through the existing QA/Q
- `false_negatives`
- `quality_check_id`
Configured-YOLO QA automatically reads `tile_manifest_path` from the persisted
`AnalysisRun.parameters_json`. Candidate and reference geometries are clipped
to the union of the manifest's tile bounds after explicit CRS transformation to
EPSG:4326. The response additionally returns:
- `candidate_feature_count_raw` and `reference_feature_count_raw`;
- `coverage`, including raw/evaluated/excluded/boundary-clipped population
counts, tile count, source CRS values and coverage mode;
- `box_to_footprint_diagnostics`, which compares candidate boxes with reference
envelopes at the same IoU threshold.
The canonical precision, recall, F1 and mean IoU always remain based on
candidate geometry versus the persisted reference footprint. Envelope results
are explicitly `diagnostic_only` and are persisted in
`quality_checks.findings_json`; they never replace or inflate canonical metrics.
Configured-YOLO QA fails closed with `DETECTION_QA_COVERAGE_UNAVAILABLE` when
manifest provenance is absent, `DETECTION_QA_COVERAGE_MISMATCH` when it belongs
to another raster, `DETECTION_QA_COVERAGE_INVALID` when bounds/CRS are invalid,
or `REFERENCE_FEATURES_OUTSIDE_COVERAGE` when no reference polygons overlap the
actual inference coverage. Explicit fixture/legacy runs without a manifest keep
the documented unbounded comparison behavior.
If the reference dataset has no persisted vector features, the endpoint returns `REFERENCE_FEATURES_NOT_FOUND`. It does not calculate fake QA metrics.
#### Future analysis route: `/api/v1/analysis/building-stats`
+43
View File
@@ -7556,3 +7556,46 @@ Open:
- Clip detection QA reference populations to actual raster/tile coverage and
add box-to-building-footprint matching diagnostics before any further model
training or promotion decision.
# Sprint 184 - Detection QA coverage and matching diagnostics
## Implementation
- Added a focused detection-QA coverage service that resolves the persisted
tile manifest from the analysis run, validates dataset ownership and CRS,
transforms each tile footprint to EPSG:4326 and unions the exact inference
coverage.
- Configured-YOLO QA now fails closed when persisted manifest provenance is
absent, invalid or belongs to another raster. Explicit fixture and legacy
runs without a manifest retain their existing unbounded QA behavior.
- Candidate Detection geometries and persisted reference VectorFeature
geometries are clipped to inference coverage before the existing one-to-one
IoU matcher runs. Raw, evaluated, excluded-outside and boundary-clipped
counts are persisted in `quality_checks.findings_json`.
- Canonical precision, recall, F1, mean IoU and metric rows remain strict
candidate-polygon versus reference-footprint results. A second
candidate-polygon versus reference-envelope pass is persisted and displayed
as diagnostic-only evidence; it never replaces or promotes canonical
metrics.
- Detection Lab now explains which reference population was evaluated and
clearly separates possible box-to-footprint artifacts from the canonical
scorecard.
- Updated the real-data operator smoke assertions, API/AI/database contracts
and backend/frontend operator documentation. No migration, request contract,
provider fetch or model dependency changed.
## Local validation
- `bash scripts/run_readiness_check.sh` passed end to end.
- Full backend suite passed: `506` tests.
- API contract audit passed with `81` implemented routes and the two documented
non-envelope endpoints.
- Alembic reports one head: `202606120900`.
- Frontend TypeScript checking and production build passed with `85` modules;
the dedicated MapLibre chunk remains intact.
## Next pass
- Deploy Sprint 184 to Tower, rerun QA for the persisted Mol-center analysis
run and verify both persisted coverage evidence and Detection Lab rendering
against live PostGIS before making any model-training decision.
+7
View File
@@ -189,6 +189,13 @@ Metrics may belong to a quality check, an analysis run, or both. Sprint 7A persi
Quality checks are domain records. Jobs track execution state; quality checks track the persisted QA/QC result; metrics track individual measurements.
Detection QA coverage and box-to-footprint diagnostics require no schema
change. The canonical metric rows remain precision, recall, F1, mean IoU and
false-positive/negative counts. Tile coverage population counts and the
diagnostic reference-envelope comparison are persisted in the existing
`quality_checks.findings_json`; `parameters_json.coverage_policy` records the
evaluation policy used for reproducibility.
### exports
- `id uuid primary key`
+2 -2
View File
@@ -26,7 +26,7 @@ This file now starts with the current implementation status. Older preparation/b
- [x] Separate persisted database layers from available analysis-result overlays with an explicit Map content mode.
- [x] Persist combined Mol operator evidence under the Unraid storage mount so reports survive all-in-one container replacement.
- [x] Visually review Mol Postel and Donk false-positive/false-negative evidence, classify the dominant error modes and only then decide whether another model-training pass is justified.
- [ ] Clip detection QA populations to persisted raster/tile coverage and add box-to-footprint matching diagnostics before reconsidering model training.
- [x] Clip detection QA populations to persisted raster/tile coverage and add box-to-footprint matching diagnostics before reconsidering model training.
- [x] Backend FastAPI foundation, health endpoint and service structure.
- [x] React/TypeScript frontend foundation and MapLibre workbench.
- [x] Map layer visibility, opacity and feature property inspection.
@@ -411,7 +411,7 @@ This file now starts with the current implementation status. Older preparation/b
- [x] Bound populated Project, Dataset, QA, AI and Export panels so long histories do not push core actions thousands of pixels down-page.
- [x] Move Data create/upload forms and Map/AI diagnostics into explicit progressive disclosures.
- [x] Reorder AI Labs around run controls and Map around layer selection plus the MapLibre canvas.
- [ ] Harden detection QA coverage and matching diagnostics before any further model training.
- [x] Harden detection QA coverage and matching diagnostics before any further model training.
- [x] Replace the one-page workflow panel stack with a task-based workbench shell.
- [x] Add persistent project/AOI/dataset/layer context.