Add Mol false-negative visual review evidence

2026-07-13 20:22:53 +02:00
parent de4e15f166
commit 50952e18f9
11 changed files with 967 additions and 3 deletions
@@ -7301,3 +7301,37 @@ Open:
 - Ran the interactive PostGIS AOI query against `vector_features`: `126` GRB buildings returned with no truncation. The guided GIS smoke persisted a derived dataset and GeoJSON export, then produced candidate/reference QA F1 `0.9582` with `126` matches, `0` false positives and `11` false negatives; `263` persisted evidence features rendered back on the map.
 - Remaining evidence caveat: the dataset QA result reports weak CRS-assumption warnings, so those geometry metrics remain explicitly approximate until CRS provenance handling is reviewed.
 - Host observation outside GeoIntel: Unraid recovered after reboot and serves the app, but still reports one disabled/invalid array device. Storage administration should resolve that independently of application development.
+
+# Sprint 179 - Mol Donk/Postel detection evidence diagnosis
+
+## Implementation
+
+- Added `render_detection_false_negative_review_contact_sheets.py` as a read-only counterpart to the persisted false-positive review workflow.
+- Resolved each sample's exact persisted tile manifest from its fixed-threshold run summary; manifests and source tiles are confined to `/app/storage`.
+- Projected WGS84 missed-reference geometry onto the real inference tiles and rendered nearby persisted candidate detections plus matched-reference context.
+- Added deterministic AOI/area stratification and an explicit five-state manual decision CSV. No decision is inferred and no training input is exported automatically.
+- Separated references outside every persisted source tile into `false_negatives_outside_tile_coverage.geojson` instead of hiding them or calling them model misses.
+- Added focused rendering, manifest, source-coverage and storage-confinement regression tests; wired the script into readiness compilation and the all-in-one image.
+
+## Live Mol evidence
+
+- Re-exported the four-zone fixed-threshold portfolio from existing persisted QualityChecks without rerunning inference or mutating application data.
+- The complete portfolio contains `7,283` evidence features. Donk contributes `424` false positives and `553` false negatives; Postel contributes `43` and `85`.
+- Rendered and inspected 48 stratified false-positive cases over Donk/Postel. Explicit decisions: `37` QA alignment mismatches, `7` confirmed model false positives, `3` reference gaps/changes and `1` uncertain. The existing validator passed with status `complete`.
+- Rendered and inspected 48 stratified false-negative cases. Explicit decisions: `27` QA alignment mismatches, `6` confirmed model false negatives, `3` reference gaps/changes and `12` imagery-obscured/uncertain.
+- Found `41/638` false-negative evidence records outside every persisted inference tile: Donk `28`, Postel `13`. Directionally excluding those records raises Donk recall from `0.5519` to `0.5647` and Postel from `0.3796` to `0.4194`; these are audit diagnostics only and no persisted metric was changed.
+- The dominant visual mode is rectangle-to-footprint mismatch on large industrial roofs and dense residential blocks, often with a blue persisted candidate already overlapping the red missed GRB footprint. Postel additionally contains many tiny/vegetation-obscured references.
+- Persistent evidence and the assessment are stored below `/app/storage/operator-evidence/mol-operational-review/20260713`.
+
+## Decision and next pass
+
+- **NO-GO for immediate retraining.** Only `7/48` reviewed false positives and `6/48` reviewed false negatives were confirmed model errors; evaluation alignment and coverage defects dominate the selected evidence.
+- Next harden detection QA to restrict candidate/reference populations to persisted raster/tile coverage and expose best-IoU/overlap/unmatched diagnostics. Rerun Donk/Postel QA against the unchanged persisted detections before deciding whether the confirmed model-error subset justifies curated training.
+
+## Validation
+
+- Focused false-positive/false-negative audit and visual-review coverage: `8 passed`.
+- Ruff passed for the changed Python renderer and regression tests.
+- `bash scripts/run_readiness_check.sh`: passed with `487` backend tests, the 81-route API contract audit, one Alembic head, frontend typecheck and production build.
+- `python -m alembic upgrade head --sql` rendered the complete migration chain through `202606120900`; no migration changed.
+- Local `docker compose config` was unavailable because the Windows workstation has no Docker CLI. The repository-driven Tower deployment remains the required live Docker validation.