diff --git a/docs/CODEX_EXECUTION_LOG.md b/docs/CODEX_EXECUTION_LOG.md index 4006f353..945d1b7c 100644 --- a/docs/CODEX_EXECUTION_LOG.md +++ b/docs/CODEX_EXECUTION_LOG.md @@ -6593,3 +6593,50 @@ Open: ## Next recommended pass - Push this helper to Tower, run it first as dry-run against the high-threshold promotion report, then apply and redeploy/restart only if the emitted `YOLO_MODEL_PATH` matches the promoted local asset. + +# Sprint 164 - Live operational YOLO detection and QA smoke + +## What changed + +- No product code, API contract, migration, model artifact, fake-data path or provider-fetching behavior changed. +- Ran the deployed all-in-one Tower runtime at `http://192.168.10.150:1202` through a real raster/reference detection and QA workflow using existing operator data and the local promoted YOLO model asset. + +## Tower runtime evidence + +- Runtime URL: `http://192.168.10.150:1202` +- Raster input: `/mnt/user/appdata/geointel/storage/operator-data/operator-samples-1024/geel_orthophoto_wms_1024.tif` +- Reference input: `/mnt/user/appdata/geointel/storage/operator-data/operator-samples-1024/geel_grb_gbg_buildings.geojson` +- Model asset: `geointel-building-yolov8s-aoi1024bg512r3e50-pt` +- Project: `865746ee-10ce-40a1-a3da-98b2182200e5` +- Raster dataset: `c87ed009-0bf0-4a34-adc5-51e6747d847b` +- Reference dataset: `d9bca687-20ca-4609-8c15-d24d240cfae6` +- Tile manifest: `/app/storage/tiles/865746ee-10ce-40a1-a3da-98b2182200e5/c87ed009-0bf0-4a34-adc5-51e6747d847b/b1335951-4ead-4e08-9c5f-08c67e026a8f/manifest.json` +- Analysis run: `45159e0b-36be-4300-8132-fef3a1e6b667` +- Persisted detections: `333` +- Quality check: `2e696dca-dea6-42d1-af2a-4894b182d427` +- Detection export: `339344f7-38e3-4558-b66b-459726051bac` + +## Validation + +- Ran `scripts/verify_real_data_detection_qa_workflow.sh` against the deployed Tower runtime: passed. +- Confirmed detection run status: `success`. +- Confirmed detection list endpoint returned `333` persisted detections with real source tile provenance. +- Confirmed detection GeoJSON endpoint returned a `FeatureCollection` with `333` persisted geometry features. +- Confirmed detection export content returned a `detection_geojson` `FeatureCollection` with `333` features. +- Confirmed QA/QC persisted metrics: + - precision: `0.21621621621621623` + - recall: `0.031746031746031744` + - F1: `0.05536332179930796` + - mean IoU: `0.5697275247203281` + - false positives: `261` + - false negatives: `2196` +- Confirmed QA evidence overlay endpoint returned a `FeatureCollection` with `2601` features and no warnings. + +## Known limitations + +- The model is operational and demonstrable, but the Geel smoke confirms low recall at the current conservative threshold. It should remain an operator-review detector, not an automated decision engine. +- Further training data quality work remains required before treating the detector as production-grade. + +## Next recommended pass + +- Add/curate more high-quality positive AOIs and cleaner building labels, then rerun the multi-AOI calibration and promotion gate before changing default operator thresholds.