Harden detection result review
This commit is contained in:
@@ -352,6 +352,16 @@ remains available as a higher-precision legacy `0.15` choice. The older
|
||||
default-promotion blocker. Every production-like run still requires persisted
|
||||
QA/QC against suitable reference data.
|
||||
|
||||
The persisted seven-AOI evidence for this profile contains 5,568 false
|
||||
positives among 13,613 candidate detections. The read-only audit command in
|
||||
`scripts/README.md` reports the largest review volumes in Turnhout, Herentals
|
||||
and Geel, a median false-positive geometry area of about 184.5 m2, and 25.8%
|
||||
tiny/small geometry below 100 m2. Current evidence does not include
|
||||
per-detection confidence, so model-review reports must retain confidence
|
||||
coverage as zero rather than treating threshold `0.15` as an observed score.
|
||||
Combined false-positive GeoJSON is evidence for operator review only; a feature
|
||||
must be visually confirmed before it is used as a hard-negative label.
|
||||
|
||||
To update a Tower/Unraid `.env` from a promoted report, use the guarded
|
||||
activation helper. It validates the exact report candidate key, verifies that
|
||||
the candidate has `promotion_status=promote_candidate`, resolves the local model
|
||||
|
||||
@@ -7138,3 +7138,41 @@ Open:
|
||||
## Next recommended pass
|
||||
|
||||
- After redeploy, verify the active model SHA, local model-load preflight, live PostGIS migration smoke and browser profile selection. Then review false-positive evidence and the remaining 5,838 persistent misses before any further training.
|
||||
|
||||
# Sprint 175 - Detection result scale and false-positive evidence review
|
||||
|
||||
## UI hardening
|
||||
|
||||
- Confirmed that Detection Lab rendered every persisted detection row at once; Westerlo alone produced 1,172 body rows in the browser.
|
||||
- Added local 25/50/100-row pagination with a default of 50 rows, bounded page controls and automatic page-one reset after run/filter/result changes.
|
||||
- Kept the full persisted collection unchanged for the existing MapLibre GeoJSON overlay and detection QA/QC. No endpoint, response envelope or persistence contract changed.
|
||||
|
||||
## Persisted false-positive evidence
|
||||
|
||||
- Added `scripts/audit_detection_false_positive_evidence.py` as a read-only evidence consumer.
|
||||
- The audit validates FeatureCollection/polygon geometry, compares declared portfolio role counts with actual evidence, computes WGS84 geodesic areas, preserves original feature provenance and emits combined `false_positives.geojson`.
|
||||
- Source tile summaries are qualified by AOI because `tile_index` is local to each raster manifest.
|
||||
- The active fixed-threshold seven-AOI portfolio produced:
|
||||
- 5,568 false positives among 13,613 candidate detections (`0.4090` false-positive share);
|
||||
- median false-positive geometry area `184.5 m2`, p90 `607.7 m2`;
|
||||
- 52 below 25 m2, 1,382 between 25-100 m2, 3,412 between 100-500 m2 and 722 at or above 500 m2;
|
||||
- largest AOI review volumes: Turnhout `1,102`, Herentals `917`, Geel `913`;
|
||||
- largest AOI-qualified tile hotspot: `turnhout:0` with 236 false positives.
|
||||
- Existing persisted QA evidence carries run threshold and tile index but no per-detection confidence. The audit reports confidence coverage `0/5,568` and does not invent confidence statistics.
|
||||
- Added focused regression coverage, readiness compilation and all-in-one image inclusion. No training, inference, provider fetch, model download or activation occurred.
|
||||
|
||||
## Local validation
|
||||
|
||||
- `python -m compileall backend/app`: passed.
|
||||
- `python -m pytest`: 475 passed.
|
||||
- `python -m ruff check` for the new audit/test modules: passed.
|
||||
- `npm run typecheck`: passed.
|
||||
- `npm run build`: passed; app bundle `216.83 kB`, MapLibre bundle `801.82 kB` before gzip.
|
||||
- `bash scripts/run_readiness_check.sh`: passed with 475 tests and all release-critical syntax gates.
|
||||
- `python -m alembic heads`: one head, `202606120900`.
|
||||
- `python -m alembic upgrade head --sql`: complete migration chain rendered successfully.
|
||||
- Local `docker compose config` could not run because Docker CLI is not installed on the Windows host; live image/PostGIS validation is delegated to the Docker-enabled Tower deployment.
|
||||
|
||||
## Next recommended pass
|
||||
|
||||
- Redeploy and verify bounded table rendering against the live 1,172-detection Westerlo run. Then visually classify a stratified sample from Turnhout, Herentals and Geel before deciding whether any confirmed false positives belong in a new hard-negative corpus.
|
||||
|
||||
+2
-1
@@ -138,7 +138,8 @@ This file now starts with the current implementation status. Older preparation/b
|
||||
- [x] Review per-AOI false-negative evidence, expand positive sample/label coverage and verify the resulting candidate improves false-negative rate in every validated AOI.
|
||||
- [x] Complete rebuild/restart and browser/runtime smoke for the guarded promoted V1 building detector activation.
|
||||
- [x] Expand focused small-building training evidence after reviewing persistent false negatives, train one inactive candidate and pass it through positive, pure-empty and fixed-reference promotion evidence before guarded activation.
|
||||
- [ ] Review the remaining 5,838 persistent false negatives and the increased false-positive load before any further model training; do not start another blind run.
|
||||
- [x] Audit the promoted model's increased false-positive load from persisted seven-AOI QA evidence, including geodetic area buckets, AOI-qualified tile hotspots and combined review GeoJSON.
|
||||
- [ ] Visually classify representative false-positive evidence from Turnhout, Herentals and Geel and review the remaining 5,838 persistent false negatives before any further model training; do not start another blind run.
|
||||
|
||||
## Sprint 8 status
|
||||
|
||||
|
||||
Reference in New Issue
Block a user