Add persisted false-positive visual review gate
GeoIntel CI / docs-smoke (push) Canceled after 0s
GeoIntel CI / contract-smoke (push) Canceled after 0s

This commit is contained in:
Codex
2026-07-13 17:12:59 +02:00
parent 2bd38556e8
commit 1322a5d66a
13 changed files with 1379 additions and 0 deletions
+21
View File
@@ -129,6 +129,27 @@ candidates with EPSG:4326 geometry IoU greater than or equal to
raw, persisted and suppressed detection counts so calibration evidence remains
auditable.
### Persisted false-positive visual review
Detection QA labels a candidate as a false-positive only relative to the
selected persisted reference dataset and matching tolerance. That finding is
not automatically a model error: the reference can be incomplete or stale, and
alignment can be wrong. GeoIntel therefore exposes persisted detection
confidence/model/tile/bbox provenance in the existing QA evidence GeoJSON and
provides a read-only contact-sheet workflow.
The operator must explicitly select one of:
- `confirmed_model_false_positive`;
- `reference_gap_or_change`;
- `qa_alignment_mismatch`;
- `uncertain`;
- `unreviewed`.
Only records explicitly marked `confirmed_model_false_positive` are emitted by
the validator as possible hard-negative review input. The workflow does not
train a model, mutate QA persistence, fetch data or infer review decisions.
### Local model asset catalog
GeoIntel can list local runtime model files mounted into the backend model
+8
View File
@@ -1215,6 +1215,14 @@ Response:
`false_positive` or `false_negative`. Missing persisted feature ids are reported
in `warnings`; no fake geometries are produced.
Detection-backed candidate evidence also exposes provenance read from the
persisted `detections` row: `detection_id`, `job_id`, `confidence`,
`model_name`, `model_version`, `source_tile_path` and `bbox_json`. Existing
`properties_json` fields such as `tile_index` remain present. Segmentation-backed
candidate evidence exposes the equivalent persisted model/source fields plus
`segmentation_id`, `mask_path` and `area_m2`. These are additive GeoJSON
properties; the canonical envelope and endpoint path are unchanged.
## Exports
### POST `/api/v1/exports/geojson`
+35
View File
@@ -7189,3 +7189,38 @@ Open:
## Next recommended pass
- Visually classify a stratified false-positive sample from Turnhout, Herentals and Geel before deciding whether any confirmed examples belong in a new hard-negative corpus. Review the remaining 5,838 persistent false negatives in the same evidence-led pass; do not start another blind training run.
# Sprint 176 - Detection false-positive visual review gate
## Persisted provenance
- Extended the existing read-only QA evidence GeoJSON conversion so detection-backed evidence carries the persisted detection id, job id, confidence, model name/version, source tile path and pixel bbox.
- Added equivalent persisted segmentation provenance fields without changing the endpoint, canonical envelope, ORM or migration chain.
- Historical `QualityCheck` evidence can be re-exported against existing persisted `Detection` rows; no QA rerun or data rewrite is required.
## Manual visual review
- Added a storage-root-confined contact-sheet renderer for persisted detection false-positive evidence.
- The renderer validates portfolio role counts, polygon geometry, source imagery and persisted provenance, then selects deterministically across AOI, WGS84 area bucket and confidence band.
- Source imagery is rendered with the candidate pixel bbox plus persisted matched-reference and missed-reference overlays.
- Added an explicit five-state review CSV: `confirmed_model_false_positive`, `reference_gap_or_change`, `qa_alignment_mismatch`, `uncertain` and `unreviewed`.
- Added a separate validator that rejects missing, duplicate, unexpected or invalid decisions. `--require-complete` exits with code `2` while any record remains unreviewed.
- Only explicitly confirmed model false-positives are emitted to `confirmed_model_false_positives.geojson`; no QA result is automatically converted into a model label or training artifact.
## Validation
- `python -m compileall backend/app`: passed.
- `python -m pytest`: 478 passed.
- Focused provenance/render/path-confinement/incomplete-review/export tests: passed.
- `python -m ruff check` for changed Python services, scripts and tests: passed.
- Generated fixture contact sheet was visually inspected at 128 px thumbnails; candidate/reference/missed-reference overlays and header provenance remained readable.
- `npm run typecheck`: passed.
- `npm run build`: passed; app bundle `217.00 kB`, MapLibre bundle `801.82 kB` before gzip.
- `bash scripts/run_readiness_check.sh`: passed with 478 tests and the new operator-script compile gates.
- `python -m alembic heads`: one head, `202606120900`.
- `python -m alembic upgrade head --sql`: complete migration chain rendered successfully.
- Local Docker validation remains unavailable because Docker CLI is not installed on the Windows host; live all-in-one/PostGIS validation follows on Tower after deployment.
## Next recommended pass
- Re-export the seven-AOI evidence portfolio from the deployed backend, render the Turnhout/Herentals/Geel sheets and inspect the real orthophoto evidence. Keep all CSV decisions `unreviewed` until an operator makes an explicit visual classification; do not start another model training run yet.
+1
View File
@@ -139,6 +139,7 @@ This file now starts with the current implementation status. Older preparation/b
- [x] Complete rebuild/restart and browser/runtime smoke for the guarded promoted V1 building detector activation.
- [x] Expand focused small-building training evidence after reviewing persistent false negatives, train one inactive candidate and pass it through positive, pure-empty and fixed-reference promotion evidence before guarded activation.
- [x] Audit the promoted model's increased false-positive load from persisted seven-AOI QA evidence, including geodetic area buckets, AOI-qualified tile hotspots and combined review GeoJSON.
- [x] Add persisted detection provenance, stratified visual contact sheets and an explicit manual-decision gate for false-positive review.
- [ ] Visually classify representative false-positive evidence from Turnhout, Herentals and Geel and review the remaining 5,838 persistent false negatives before any further model training; do not start another blind run.
## Sprint 8 status