Log live operational YOLO QA smoke

2026-07-11 15:40:27 +02:00
parent 3e220f10d3
commit 5d24242fb4
1 changed files with 47 additions and 0 deletions
@@ -6593,3 +6593,50 @@ Open:
 ## Next recommended pass

 - Push this helper to Tower, run it first as dry-run against the high-threshold promotion report, then apply and redeploy/restart only if the emitted `YOLO_MODEL_PATH` matches the promoted local asset.
+
+# Sprint 164 - Live operational YOLO detection and QA smoke
+
+## What changed
+
+- No product code, API contract, migration, model artifact, fake-data path or provider-fetching behavior changed.
+- Ran the deployed all-in-one Tower runtime at `http://192.168.10.150:1202` through a real raster/reference detection and QA workflow using existing operator data and the local promoted YOLO model asset.
+
+## Tower runtime evidence
+
+- Runtime URL: `http://192.168.10.150:1202`
+- Raster input: `/mnt/user/appdata/geointel/storage/operator-data/operator-samples-1024/geel_orthophoto_wms_1024.tif`
+- Reference input: `/mnt/user/appdata/geointel/storage/operator-data/operator-samples-1024/geel_grb_gbg_buildings.geojson`
+- Model asset: `geointel-building-yolov8s-aoi1024bg512r3e50-pt`
+- Project: `865746ee-10ce-40a1-a3da-98b2182200e5`
+- Raster dataset: `c87ed009-0bf0-4a34-adc5-51e6747d847b`
+- Reference dataset: `d9bca687-20ca-4609-8c15-d24d240cfae6`
+- Tile manifest: `/app/storage/tiles/865746ee-10ce-40a1-a3da-98b2182200e5/c87ed009-0bf0-4a34-adc5-51e6747d847b/b1335951-4ead-4e08-9c5f-08c67e026a8f/manifest.json`
+- Analysis run: `45159e0b-36be-4300-8132-fef3a1e6b667`
+- Persisted detections: `333`
+- Quality check: `2e696dca-dea6-42d1-af2a-4894b182d427`
+- Detection export: `339344f7-38e3-4558-b66b-459726051bac`
+
+## Validation
+
+- Ran `scripts/verify_real_data_detection_qa_workflow.sh` against the deployed Tower runtime: passed.
+- Confirmed detection run status: `success`.
+- Confirmed detection list endpoint returned `333` persisted detections with real source tile provenance.
+- Confirmed detection GeoJSON endpoint returned a `FeatureCollection` with `333` persisted geometry features.
+- Confirmed detection export content returned a `detection_geojson` `FeatureCollection` with `333` features.
+- Confirmed QA/QC persisted metrics:
+  - precision: `0.21621621621621623`
+  - recall: `0.031746031746031744`
+  - F1: `0.05536332179930796`
+  - mean IoU: `0.5697275247203281`
+  - false positives: `261`
+  - false negatives: `2196`
+- Confirmed QA evidence overlay endpoint returned a `FeatureCollection` with `2601` features and no warnings.
+
+## Known limitations
+
+- The model is operational and demonstrable, but the Geel smoke confirms low recall at the current conservative threshold. It should remain an operator-review detector, not an automated decision engine.
+- Further training data quality work remains required before treating the detector as production-grade.
+
+## Next recommended pass
+
+- Add/curate more high-quality positive AOIs and cleaner building labels, then rerun the multi-AOI calibration and promotion gate before changing default operator thresholds.