Add persisted false-positive visual review gate

2026-07-13 17:12:59 +02:00
parent 2bd38556e8
commit 1322a5d66a
13 changed files with 1379 additions and 0 deletions
@@ -811,6 +811,53 @@ evidence properties and geometry are preserved. Confidence statistics are only
 computed when confidence is actually present in persisted evidence; missing
 coverage is reported explicitly and never inferred from the run threshold.

+Render a deterministic, stratified visual review over persisted false-positive
+evidence. Static portfolios created before detection provenance was added must
+first be re-exported from the current backend; existing `QualityCheck` and
+`Detection` rows do not need to be rerun:
+
+```bash
+docker exec \
+  -e CALIBRATION_EVIDENCE_MODE=all \
+  -e CALIBRATION_PORTFOLIO_OUTPUT_DIR=/app/storage/operator-data/model-review/small-building-candidate/evidence-portfolio-enriched \
+  geointel bash /app/scripts/assemble_detection_calibration_evidence_portfolio.sh \
+    http://127.0.0.1 \
+    /app/storage/operator-data/model-review/small-building-candidate/evidence-inputs/calibration-evidence-portfolio-manifest.json
+
+docker exec geointel /opt/geointel/venv/bin/python \
+  /app/scripts/render_detection_false_positive_review_contact_sheets.py \
+  --portfolio /app/storage/operator-data/model-review/small-building-candidate/evidence-portfolio-enriched/calibration_evidence_portfolio.json \
+  --storage-root /app/storage \
+  --output-dir /app/storage/operator-data/model-review/small-building-candidate/false-positive-visual-review \
+  --sample-slugs turnhout,herentals,geel \
+  --max-features 48 \
+  --columns 4 \
+  --cards-per-sheet 16 \
+  --thumb-size 256
+```
+
+The renderer validates source paths against `--storage-root`, checks persisted
+confidence/bbox/tile provenance, samples across AOI, WGS84 area bucket and
+confidence band, and overlays persisted matched/missed reference polygons. It
+writes PNG sheets, a JSON/Markdown summary and
+`false_positive_review_decisions.csv` with every row set to `unreviewed`.
+
+After manual inspection, validate the edited CSV:
+
+```bash
+docker exec geointel /opt/geointel/venv/bin/python \
+  /app/scripts/validate_detection_false_positive_review_decisions.py \
+  --review-summary /app/storage/operator-data/model-review/small-building-candidate/false-positive-visual-review/detection_false_positive_review_summary.json \
+  --decisions-csv /app/storage/operator-data/model-review/small-building-candidate/false-positive-visual-review/false_positive_review_decisions.csv \
+  --output-dir /app/storage/operator-data/model-review/small-building-candidate/false-positive-visual-review/validated \
+  --require-complete
+```
+
+`--require-complete` exits with code `2` while any record is still `unreviewed`.
+Only explicit `confirmed_model_false_positive` decisions are written to
+`confirmed_model_false_positives.geojson`; the tool never promotes generic QA
+false-positives into training labels.
+
 Docker images install only the GIS runtime by default. To build a local/Tower
 image with PyTorch/Ultralytics available for the configured-YOLO preflight and
 runtime path, set: