Add operator YOLO tile dataset exporter
GeoIntel CI / docs-smoke (push) Has been cancelled
GeoIntel CI / contract-smoke (push) Has been cancelled

This commit is contained in:
Codex
2026-07-07 21:02:09 +02:00
parent 404b37fa07
commit 1d27e4a059
9 changed files with 604 additions and 1 deletions
+19
View File
@@ -266,6 +266,25 @@ This remains operator tooling only. GeoIntel does not expose Training Studio in
V1, does not generate labels from predictions and does not treat the trained
artifact as useful until it passes the same real-data Detection + QA matrix.
If the whole-image dataset underfits or produces unusable detections, export
overlapping tile-level samples:
```bash
docker exec -it geointel python3 /app/scripts/export_operator_yolo_tile_dataset.py \
--manifest-path /app/storage/operator-data/operator_samples_manifest.json \
--output-dir /app/storage/operator-data/yolo-building-tile-dataset \
--tile-size 192 \
--stride 96 \
--negative-keep-ratio 0.5 \
--val-samples turnhout \
--force
```
The tile exporter clips reference building boxes into tile-local YOLO labels
and records the positive/negative tile counts. This gives the training smoke
more image samples while preserving the same explicit operator-data and QA/QC
validation boundary.
For visual error inspection, export the persisted QA evidence from a calibration
summary:
+25
View File
@@ -1,3 +1,28 @@
## Sprint 130 Operator YOLO tile-level dataset tooling (2026-07-07)
Changed:
- Added `scripts/export_operator_yolo_tile_dataset.py`.
- The exporter reads `operator_samples_manifest.json`, opens each raster/reference pair, creates overlapping tile windows, clips GRB building bounding boxes into tile-local YOLO labels, writes `dataset.yaml`, and reports `yolo_tile_dataset_summary.json`.
- Added deterministic negative tile retention through `negative_keep_ratio`.
- Added readiness compile coverage for the tile exporter.
- Added regression coverage in `backend/tests/test_sprint130_operator_yolo_tile_dataset.py`.
- Updated `scripts/README.md`, `backend/README.md`, `docs/AI_PIPELINES.md`, `docs/TODO.md` and `CHANGELOG.md`.
Tested:
- RED: `python -m pytest backend\tests\test_sprint130_operator_yolo_tile_dataset.py -q` failed while `scripts/export_operator_yolo_tile_dataset.py` did not exist.
- `python -m pytest backend\tests\test_sprint130_operator_yolo_tile_dataset.py -q` passed.
- `python scripts\export_operator_yolo_tile_dataset.py --help` passed without requiring local GIS dependencies.
- `python -m py_compile scripts\export_operator_yolo_tile_dataset.py` passed.
Open:
- Run the tile exporter inside the AI-enabled Tower runtime, train a local tile-level model, and benchmark it through the existing multi-sample Detection + QA matrix.
Limitations:
- This remains operator tooling only. It does not add Training Studio, browser training controls, provider fetching, fake detections, model auto-provisioning or API contract changes.
Next recommended pass:
- Export the tile-level dataset on Tower with `tile-size=192`, `stride=96`, train a longer local model, and compare it against the current `yolov8s-building-segmentation-pt` baseline.
## Sprint 129 Operator YOLO training dataset tooling (2026-07-07)
Changed:
+3 -1
View File
@@ -406,4 +406,6 @@ This file now starts with the current implementation status. Older preparation/b
- [x] Harden configured YOLO inference for single-band raster tiles and wrapped runtime errors.
- [x] Add operator-only YOLO dataset export and local training-smoke wrapper for real sample calibration.
- [x] Train/evaluate a small local GeoIntel building-detector smoke from the current operator samples and reject it because QA/QC did not improve.
- [ ] Build a larger tile-level training dataset with more AOIs, positive/negative tiles and validation splits before the next local model attempt.
- [x] Add operator-only tile-level YOLO dataset export with overlapping windows and deterministic negative tile retention.
- [ ] Run tile-level training on Tower and accept/reject the resulting local model through the persisted QA/QC matrix.
- [ ] Add more AOIs after the tile-level baseline so the next local model attempt is not limited to Geel/Mol/Turnhout.