Jens/geointel
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add multi-sample detection quality calibration
GeoIntel CI / docs-smoke (push) Has been cancelled
Details
GeoIntel CI / contract-smoke (push) Has been cancelled
Details

Browse Source
This commit is contained in:
Codex
2026-07-07 04:52:10 +02:00
parent 75b4b55ea7
commit 06dfc5f769
10 changed files with 747 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/CODEX_EXECUTION_LOG.md
+34
View File
@@ -1,3 +1,37 @@
## Sprint 127 Multi-sample detection quality calibration tooling (2026-07-07)
Changed:
- Added `scripts/prepare_operator_real_data_samples.py` as an explicit operator/runtime helper for documented Geel, Mol and Turnhout real-data samples.
- The helper downloads small Digitaal Vlaanderen OMWRGBMRVL WMS `Ortho` GeoTIFFs and GRB OGC API Features `GBG` building GeoJSON references for the documented AOIs only.
- The helper writes `operator_samples_manifest.json`, sample metadata, source URLs and attribution under the runtime operator-data directory and reuses existing files by default.
- Added `scripts/run_multi_sample_detection_quality_matrix.sh` to run `scripts/run_detection_quality_matrix.sh` once per manifest sample.
- The multi-sample wrapper combines per-sample `quality_matrix_summary.json` files into `multi_sample_quality_summary.json` with `best_overall_by_score`, `best_overall_by_recall`, `best_overall_by_precision` and `best_by_sample`.
- Added readiness checks for Python compile and shell syntax.
- Added regression coverage in `backend/tests/test_sprint127_operator_sample_quality_matrix.py`.
- Updated `scripts/README.md`, `backend/README.md`, `docs/AI_PIPELINES.md`, `docs/TODO.md` and `CHANGELOG.md`.
Tested:
- RED: `python -m pytest backend\tests\test_sprint127_operator_sample_quality_matrix.py -q` failed because the sample-preparation and multi-sample scripts did not exist.
- RED: `python -m pytest backend\tests\test_sprint127_operator_sample_quality_matrix.py::test_prepare_operator_real_data_samples_help_does_not_require_gis_dependencies -q` failed because `--help` required missing GIS dependencies.
- `python -m pytest backend\tests\test_sprint127_operator_sample_quality_matrix.py -q` passed.
- `python scripts\prepare_operator_real_data_samples.py --help` passed without requiring local GIS dependencies.
- `python -m py_compile scripts\prepare_operator_real_data_samples.py` passed.
- `bash -n scripts/run_multi_sample_detection_quality_matrix.sh` passed.
- `python -m pytest backend\tests\test_sprint127_operator_sample_quality_matrix.py backend\tests\test_sprint126_detection_quality_matrix.py backend\tests\test_sprint125_detection_calibration_evidence_bundle.py backend\tests\test_sprint124_detection_calibration_sweep.py -q` passed.
- `python scripts\smoke_docs.py` passed.
- `git diff --check` passed.
- `bash scripts/run_readiness_check.sh` passed: 393 backend tests, frontend typecheck/build, Alembic head `202606120900`, live smoke syntax checks and the new sample/multi-sample checks.
Open:
- Live Tower sample preparation and multi-sample matrix run still needed.
Limitations:
- This is operator tooling only. It does not add a live GRB provider, live orthophoto provider, application endpoint, migration, frontend feature, model download or fixture inference path.
- The prepared sample files are runtime artifacts under appdata/storage and remain excluded from Git.
Next recommended pass:
- Run the sample-preparation helper in the Tower all-in-one container, then run the multi-sample matrix from the Tower checkout and document the combined quality baseline.
## Sprint 126 Detection quality matrix tooling (2026-07-07)
Changed: