Jens/geointel
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Document detection quality matrix results
GeoIntel CI / docs-smoke (push) Has been cancelled
Details
GeoIntel CI / contract-smoke (push) Has been cancelled
Details

Browse Source
This commit is contained in:
Codex
2026-07-07 04:31:26 +02:00
parent e728f7004f
commit 75b4b55ea7
2 changed files with 16 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/CODEX_EXECUTION_LOG.md
+15 -2
View File
@@ -18,16 +18,29 @@ Tested:
- `python scripts\smoke_docs.py` passed.
- `git diff --check` passed.
- `bash scripts/run_readiness_check.sh` passed: 390 backend tests, frontend typecheck/build, Alembic head `202606120900`, live smoke syntax checks and the new matrix syntax check.
- Pushed commit `e728f70` to Gitea and fast-forwarded Tower `/mnt/user/appdata/geointel` to the same commit.
- Tower live matrix passed against the Geel operator sample:
- command: `QUALITY_MODEL_ASSET_IDS="yolov8n-building-segmentation-pt yolov8n-pt" QUALITY_TILE_SIZES="512 640" QUALITY_TILE_OVERLAPS="64" QUALITY_THRESHOLDS="0.50 0.15" bash scripts/run_detection_quality_matrix.sh http://192.168.10.150:1202`
- output: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/20260707T023000Z/quality_matrix_summary.json`
- run count: 8
- best by score and recall: `yolov8n-building-segmentation-pt`, tile `512`, overlap `64`, threshold `0.15`, 80 detections, 6 matches, 74 false positives, 611 false negatives, precision `0.075`, recall `0.009724473257698542`, F1 `0.017216642754662843`.
- best by precision: `yolov8n-building-segmentation-pt`, tile `640`, overlap `64`, threshold `0.50`, 4 detections, 1 match, 3 false positives, 616 false negatives, precision `0.25`, recall `0.0016207455429497568`, F1 `0.0032206119162640897`.
- generic `yolov8n-pt` produced zero building detections for all tested tile/threshold combinations.
- Tower best-run evidence export passed from the matrix summary:
- `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/20260707T023000Z/calibration_evidence.geojson`
- `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/20260707T023000Z/calibration_evidence_summary.json`
- `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/20260707T023000Z/calibration_evidence_review.html`
- evidence features: 697 total, 611 false negatives, 74 false positives, 6 matched detections and 6 matched references.
Open:
- Live Tower matrix run still needed against the current Geel operator sample.
- None for matrix tooling.
Limitations:
- This is operator benchmarking tooling only. It does not change inference behavior, add model downloads, seed fixture detections, fetch providers, change API contracts or change migrations.
- A single Geel sample is not enough to declare a production V1 building-extraction baseline; additional orthophoto/reference samples are still needed before picking defaults.
Next recommended pass:
- Run the matrix on Tower for the current Geel operator sample, then decide whether the active building model should remain the default evaluation model or be replaced.
- Add at least two more local orthophoto/reference samples and run the same quality matrix before choosing V1 defaults; the current Geel evidence says runtime plumbing works, but the active model is still not extraction-quality.
## Sprint 125 Detection calibration evidence bundle (2026-07-07)