Add operator YOLO tile dataset exporter
This commit is contained in:
@@ -266,6 +266,25 @@ This remains operator tooling only. GeoIntel does not expose Training Studio in
|
||||
V1, does not generate labels from predictions and does not treat the trained
|
||||
artifact as useful until it passes the same real-data Detection + QA matrix.
|
||||
|
||||
If the whole-image dataset underfits or produces unusable detections, export
|
||||
overlapping tile-level samples:
|
||||
|
||||
```bash
|
||||
docker exec -it geointel python3 /app/scripts/export_operator_yolo_tile_dataset.py \
|
||||
--manifest-path /app/storage/operator-data/operator_samples_manifest.json \
|
||||
--output-dir /app/storage/operator-data/yolo-building-tile-dataset \
|
||||
--tile-size 192 \
|
||||
--stride 96 \
|
||||
--negative-keep-ratio 0.5 \
|
||||
--val-samples turnhout \
|
||||
--force
|
||||
```
|
||||
|
||||
The tile exporter clips reference building boxes into tile-local YOLO labels
|
||||
and records the positive/negative tile counts. This gives the training smoke
|
||||
more image samples while preserving the same explicit operator-data and QA/QC
|
||||
validation boundary.
|
||||
|
||||
For visual error inspection, export the persisted QA evidence from a calibration
|
||||
summary:
|
||||
|
||||
|
||||
@@ -1,3 +1,28 @@
|
||||
## Sprint 130 Operator YOLO tile-level dataset tooling (2026-07-07)
|
||||
|
||||
Changed:
|
||||
- Added `scripts/export_operator_yolo_tile_dataset.py`.
|
||||
- The exporter reads `operator_samples_manifest.json`, opens each raster/reference pair, creates overlapping tile windows, clips GRB building bounding boxes into tile-local YOLO labels, writes `dataset.yaml`, and reports `yolo_tile_dataset_summary.json`.
|
||||
- Added deterministic negative tile retention through `negative_keep_ratio`.
|
||||
- Added readiness compile coverage for the tile exporter.
|
||||
- Added regression coverage in `backend/tests/test_sprint130_operator_yolo_tile_dataset.py`.
|
||||
- Updated `scripts/README.md`, `backend/README.md`, `docs/AI_PIPELINES.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
||||
|
||||
Tested:
|
||||
- RED: `python -m pytest backend\tests\test_sprint130_operator_yolo_tile_dataset.py -q` failed while `scripts/export_operator_yolo_tile_dataset.py` did not exist.
|
||||
- `python -m pytest backend\tests\test_sprint130_operator_yolo_tile_dataset.py -q` passed.
|
||||
- `python scripts\export_operator_yolo_tile_dataset.py --help` passed without requiring local GIS dependencies.
|
||||
- `python -m py_compile scripts\export_operator_yolo_tile_dataset.py` passed.
|
||||
|
||||
Open:
|
||||
- Run the tile exporter inside the AI-enabled Tower runtime, train a local tile-level model, and benchmark it through the existing multi-sample Detection + QA matrix.
|
||||
|
||||
Limitations:
|
||||
- This remains operator tooling only. It does not add Training Studio, browser training controls, provider fetching, fake detections, model auto-provisioning or API contract changes.
|
||||
|
||||
Next recommended pass:
|
||||
- Export the tile-level dataset on Tower with `tile-size=192`, `stride=96`, train a longer local model, and compare it against the current `yolov8s-building-segmentation-pt` baseline.
|
||||
|
||||
## Sprint 129 Operator YOLO training dataset tooling (2026-07-07)
|
||||
|
||||
Changed:
|
||||
|
||||
+3
-1
@@ -406,4 +406,6 @@ This file now starts with the current implementation status. Older preparation/b
|
||||
- [x] Harden configured YOLO inference for single-band raster tiles and wrapped runtime errors.
|
||||
- [x] Add operator-only YOLO dataset export and local training-smoke wrapper for real sample calibration.
|
||||
- [x] Train/evaluate a small local GeoIntel building-detector smoke from the current operator samples and reject it because QA/QC did not improve.
|
||||
- [ ] Build a larger tile-level training dataset with more AOIs, positive/negative tiles and validation splits before the next local model attempt.
|
||||
- [x] Add operator-only tile-level YOLO dataset export with overlapping windows and deterministic negative tile retention.
|
||||
- [ ] Run tile-level training on Tower and accept/reject the resulting local model through the persisted QA/QC matrix.
|
||||
- [ ] Add more AOIs after the tile-level baseline so the next local model attempt is not limited to Geel/Mol/Turnhout.
|
||||
|
||||
Reference in New Issue
Block a user