Persist QA feature evidence
This commit is contained in:
+30
-1
@@ -988,6 +988,14 @@ Request:
|
||||
|
||||
Response is wrapped in the job envelope. On success, `result_json` includes precision, recall, F1, mean IoU, false positives, false negatives and `quality_check_id`.
|
||||
|
||||
Sprint 111 also includes feature-level evidence arrays for map/review handoff:
|
||||
|
||||
- `match_evidence`: matched candidate/reference feature ids with IoU.
|
||||
- `false_positive_evidence`: unmatched candidate feature ids.
|
||||
- `false_negative_evidence`: unmatched reference feature ids.
|
||||
|
||||
These arrays are derived from the same persisted/source geometries used for IoU matching. They are not separate QA records yet; they are persisted inside `quality_checks.findings_json`.
|
||||
|
||||
Sprint 7A persists the QA/QC result as:
|
||||
|
||||
- `jobs`: execution state.
|
||||
@@ -1016,7 +1024,28 @@ Response:
|
||||
"status": "ok",
|
||||
"score": 0.5,
|
||||
"parameters_json": {},
|
||||
"findings_json": {},
|
||||
"findings_json": {
|
||||
"matches": 1,
|
||||
"false_positives": 1,
|
||||
"false_negatives": 1,
|
||||
"match_evidence": [
|
||||
{
|
||||
"candidate_feature_id": "candidate-feature-id",
|
||||
"reference_feature_id": "reference-feature-id",
|
||||
"iou": 0.83
|
||||
}
|
||||
],
|
||||
"false_positive_evidence": [
|
||||
{
|
||||
"candidate_feature_id": "candidate-extra-id"
|
||||
}
|
||||
],
|
||||
"false_negative_evidence": [
|
||||
{
|
||||
"reference_feature_id": "reference-missing-id"
|
||||
}
|
||||
]
|
||||
},
|
||||
"metrics": [
|
||||
{
|
||||
"metric_key": "precision",
|
||||
|
||||
@@ -1,3 +1,37 @@
|
||||
## Sprint 111 QA feature evidence persistence (2026-06-25)
|
||||
|
||||
Changed:
|
||||
- Added feature-level evidence extraction to the shared QA IoU matcher.
|
||||
- Dataset QA now returns and persists `match_evidence`, `false_positive_evidence` and `false_negative_evidence`.
|
||||
- Detection QA and segmentation QA now use the same evidence-aware matcher and persist the same evidence keys in `quality_checks.findings_json`.
|
||||
- Extended the QA/QC drilldown with compact matched, false-positive and false-negative feature id lists before the raw findings JSON.
|
||||
- Updated `docs/API_CONTRACTS.md`, `frontend/README.md`, `CHANGELOG.md` and `docs/TODO.md`.
|
||||
- Added/extended regression coverage in `backend/tests/test_qa_service.py`, `backend/tests/test_sprint7a_persistence_foundation.py` and `backend/tests/test_sprint111_qa_feature_evidence.py`.
|
||||
|
||||
Validation:
|
||||
- RED: `python -m pytest backend\tests\test_qa_service.py -q` failed before implementation because `QaProviderComparisonResult` had no `match_evidence`.
|
||||
- RED: `python -m pytest backend\tests\test_sprint111_qa_feature_evidence.py -q` failed before docs were updated because `docs/API_CONTRACTS.md` did not document the evidence keys.
|
||||
- `python -m pytest backend\tests\test_qa_service.py -q` passed: 3 tests.
|
||||
- `python -m pytest backend\tests\test_sprint8c_detection_visualization_qa.py backend\tests\test_sprint9_segmentation_foundation.py -q` passed: 18 tests.
|
||||
- `python -m pytest backend\tests\test_sprint111_qa_feature_evidence.py -q` passed: 2 tests.
|
||||
- `python -m pytest backend\tests\test_qa_service.py backend\tests\test_sprint7a_persistence_foundation.py -q` passed: 10 tests.
|
||||
- `python -m compileall backend/app` passed.
|
||||
- `cd backend && python -m pytest -q` passed: 352 tests.
|
||||
- `cd frontend && npm run typecheck` passed.
|
||||
- `cd frontend && npm run build` passed.
|
||||
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
||||
- `cd backend && python -m alembic upgrade head --sql` passed.
|
||||
- `bash -n scripts/live_migration_smoke.sh` passed.
|
||||
- `bash scripts/run_readiness_check.sh` passed.
|
||||
|
||||
Limitations:
|
||||
- Feature-level evidence is persisted as ids/IoU metadata in `quality_checks.findings_json`; no `quality_check_items` table or first-class evidence geometry table was introduced.
|
||||
- Evidence map overlays can now be built from persisted ids, but overlay generation remains future work.
|
||||
- No migration, provider fetching, AI dependency, real model behavior or new product domain was added.
|
||||
|
||||
Next recommended pass:
|
||||
- Add QA evidence overlay generation by resolving persisted evidence ids back to candidate/reference geometries and rendering false positives/false negatives as MapLibre layers.
|
||||
|
||||
## Sprint 110 Map QA evidence drilldown (2026-06-25)
|
||||
|
||||
Changed:
|
||||
|
||||
@@ -377,3 +377,4 @@ This file now starts with the current implementation status. Older preparation/b
|
||||
- [x] Persist map area selections as reusable derived vector datasets indexed into `vector_features`.
|
||||
- [x] Add Map workspace QA/QC shortcut for saved derived selection datasets.
|
||||
- [x] Add Map workspace QA/QC evidence drilldown handoff for saved selection comparisons.
|
||||
- [x] Persist QA/QC feature-level evidence for matches, false positives and false negatives.
|
||||
|
||||
Reference in New Issue
Block a user