Add persisted false-positive visual review gate
This commit is contained in:
@@ -129,6 +129,27 @@ candidates with EPSG:4326 geometry IoU greater than or equal to
|
||||
raw, persisted and suppressed detection counts so calibration evidence remains
|
||||
auditable.
|
||||
|
||||
### Persisted false-positive visual review
|
||||
|
||||
Detection QA labels a candidate as a false-positive only relative to the
|
||||
selected persisted reference dataset and matching tolerance. That finding is
|
||||
not automatically a model error: the reference can be incomplete or stale, and
|
||||
alignment can be wrong. GeoIntel therefore exposes persisted detection
|
||||
confidence/model/tile/bbox provenance in the existing QA evidence GeoJSON and
|
||||
provides a read-only contact-sheet workflow.
|
||||
|
||||
The operator must explicitly select one of:
|
||||
|
||||
- `confirmed_model_false_positive`;
|
||||
- `reference_gap_or_change`;
|
||||
- `qa_alignment_mismatch`;
|
||||
- `uncertain`;
|
||||
- `unreviewed`.
|
||||
|
||||
Only records explicitly marked `confirmed_model_false_positive` are emitted by
|
||||
the validator as possible hard-negative review input. The workflow does not
|
||||
train a model, mutate QA persistence, fetch data or infer review decisions.
|
||||
|
||||
### Local model asset catalog
|
||||
|
||||
GeoIntel can list local runtime model files mounted into the backend model
|
||||
|
||||
@@ -1215,6 +1215,14 @@ Response:
|
||||
`false_positive` or `false_negative`. Missing persisted feature ids are reported
|
||||
in `warnings`; no fake geometries are produced.
|
||||
|
||||
Detection-backed candidate evidence also exposes provenance read from the
|
||||
persisted `detections` row: `detection_id`, `job_id`, `confidence`,
|
||||
`model_name`, `model_version`, `source_tile_path` and `bbox_json`. Existing
|
||||
`properties_json` fields such as `tile_index` remain present. Segmentation-backed
|
||||
candidate evidence exposes the equivalent persisted model/source fields plus
|
||||
`segmentation_id`, `mask_path` and `area_m2`. These are additive GeoJSON
|
||||
properties; the canonical envelope and endpoint path are unchanged.
|
||||
|
||||
## Exports
|
||||
|
||||
### POST `/api/v1/exports/geojson`
|
||||
|
||||
@@ -7189,3 +7189,38 @@ Open:
|
||||
## Next recommended pass
|
||||
|
||||
- Visually classify a stratified false-positive sample from Turnhout, Herentals and Geel before deciding whether any confirmed examples belong in a new hard-negative corpus. Review the remaining 5,838 persistent false negatives in the same evidence-led pass; do not start another blind training run.
|
||||
|
||||
# Sprint 176 - Detection false-positive visual review gate
|
||||
|
||||
## Persisted provenance
|
||||
|
||||
- Extended the existing read-only QA evidence GeoJSON conversion so detection-backed evidence carries the persisted detection id, job id, confidence, model name/version, source tile path and pixel bbox.
|
||||
- Added equivalent persisted segmentation provenance fields without changing the endpoint, canonical envelope, ORM or migration chain.
|
||||
- Historical `QualityCheck` evidence can be re-exported against existing persisted `Detection` rows; no QA rerun or data rewrite is required.
|
||||
|
||||
## Manual visual review
|
||||
|
||||
- Added a storage-root-confined contact-sheet renderer for persisted detection false-positive evidence.
|
||||
- The renderer validates portfolio role counts, polygon geometry, source imagery and persisted provenance, then selects deterministically across AOI, WGS84 area bucket and confidence band.
|
||||
- Source imagery is rendered with the candidate pixel bbox plus persisted matched-reference and missed-reference overlays.
|
||||
- Added an explicit five-state review CSV: `confirmed_model_false_positive`, `reference_gap_or_change`, `qa_alignment_mismatch`, `uncertain` and `unreviewed`.
|
||||
- Added a separate validator that rejects missing, duplicate, unexpected or invalid decisions. `--require-complete` exits with code `2` while any record remains unreviewed.
|
||||
- Only explicitly confirmed model false-positives are emitted to `confirmed_model_false_positives.geojson`; no QA result is automatically converted into a model label or training artifact.
|
||||
|
||||
## Validation
|
||||
|
||||
- `python -m compileall backend/app`: passed.
|
||||
- `python -m pytest`: 478 passed.
|
||||
- Focused provenance/render/path-confinement/incomplete-review/export tests: passed.
|
||||
- `python -m ruff check` for changed Python services, scripts and tests: passed.
|
||||
- Generated fixture contact sheet was visually inspected at 128 px thumbnails; candidate/reference/missed-reference overlays and header provenance remained readable.
|
||||
- `npm run typecheck`: passed.
|
||||
- `npm run build`: passed; app bundle `217.00 kB`, MapLibre bundle `801.82 kB` before gzip.
|
||||
- `bash scripts/run_readiness_check.sh`: passed with 478 tests and the new operator-script compile gates.
|
||||
- `python -m alembic heads`: one head, `202606120900`.
|
||||
- `python -m alembic upgrade head --sql`: complete migration chain rendered successfully.
|
||||
- Local Docker validation remains unavailable because Docker CLI is not installed on the Windows host; live all-in-one/PostGIS validation follows on Tower after deployment.
|
||||
|
||||
## Next recommended pass
|
||||
|
||||
- Re-export the seven-AOI evidence portfolio from the deployed backend, render the Turnhout/Herentals/Geel sheets and inspect the real orthophoto evidence. Keep all CSV decisions `unreviewed` until an operator makes an explicit visual classification; do not start another model training run yet.
|
||||
|
||||
@@ -139,6 +139,7 @@ This file now starts with the current implementation status. Older preparation/b
|
||||
- [x] Complete rebuild/restart and browser/runtime smoke for the guarded promoted V1 building detector activation.
|
||||
- [x] Expand focused small-building training evidence after reviewing persistent false negatives, train one inactive candidate and pass it through positive, pure-empty and fixed-reference promotion evidence before guarded activation.
|
||||
- [x] Audit the promoted model's increased false-positive load from persisted seven-AOI QA evidence, including geodetic area buckets, AOI-qualified tile hotspots and combined review GeoJSON.
|
||||
- [x] Add persisted detection provenance, stratified visual contact sheets and an explicit manual-decision gate for false-positive review.
|
||||
- [ ] Visually classify representative false-positive evidence from Turnhout, Herentals and Geel and review the remaining 5,838 persistent false negatives before any further model training; do not start another blind run.
|
||||
|
||||
## Sprint 8 status
|
||||
|
||||
Reference in New Issue
Block a user