Jens/geointel
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Support larger operator training samples
GeoIntel CI / docs-smoke (push) Has been cancelled
Details
GeoIntel CI / contract-smoke (push) Has been cancelled
Details

Browse Source
This commit is contained in:
Codex
2026-07-09 13:28:54 +02:00
parent a20d9b70c7
commit a1b33555b9
7 changed files with 120 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/CODEX_EXECUTION_LOG.md
+7 -1
View File
@@ -6,6 +6,7 @@ Changed:
- Default remains `0` for legacy behavior; use `0.25` for the next overlap-heavy operator dataset experiment.
- Tile dataset summaries include `min_label_visible_ratio`.
- `scripts/audit_operator_yolo_dataset_quality.py` now reports `min_label_visible_ratio` in JSON and Markdown.
- Added operator-only `--width`, `--height` and `--half-size-scale` options to `scripts/prepare_operator_real_data_samples.py`; generated raster names now include the requested width.
- Updated operator script documentation.
Why:
@@ -18,9 +19,14 @@ Tested:
- `python -m pytest backend\tests\test_sprint130_operator_yolo_tile_dataset.py -q` (`6 passed`)
- Red step: `python -m pytest backend\tests\test_sprint146_operator_yolo_dataset_quality_audit.py -q` failed because the audit report did not expose `min_label_visible_ratio`.
- `python -m pytest backend\tests\test_sprint130_operator_yolo_tile_dataset.py backend\tests\test_sprint146_operator_yolo_dataset_quality_audit.py -q` (`7 passed`)
- Red step: `python -m pytest backend\tests\test_sprint127_operator_sample_quality_matrix.py backend\tests\test_sprint131_operator_sample_expansion.py -q` failed because sample prep lacked larger-AOI options.
- `python -m pytest backend\tests\test_sprint127_operator_sample_quality_matrix.py backend\tests\test_sprint131_operator_sample_expansion.py -q` (`8 passed`)
- Full readiness: `bash scripts/run_readiness_check.sh` (`427 passed`, frontend typecheck/build passed).
- Tower deploy: first AI rebuild failed with Docker storage full; after Docker build cache cleanup `/var/lib/docker` had 98G free and redeploy passed live migration smoke and browser runtime verification on `http://192.168.10.150:1202`.
- Tower dataset audit: exported `/app/storage/operator-data/yolo-building-aoi512-visible025` with `min_label_visible_ratio=0.25`; audit returned `needs_attention` because the current 512x512 source rasters still produce only 16 tiles and median normalized box area remains below gate.
Next:
- Run full readiness, deploy Tower, export a new visible-ratio-gated operator tile dataset, audit it, then decide whether it is good enough for another CPU training candidate.
- Prepare a larger explicit operator sample manifest, for example `/app/storage/operator-data/operator-samples-1024` with `--width 1024 --height 1024 --half-size-scale 2`, then export/audit `yolo-building-aoi1024-visible025` before another CPU training candidate.
## Sprint 149 YOLO duplicate suppression evidence (2026-07-09)
docs/TODO.md
+2 -1
View File
@@ -115,9 +115,10 @@ This file now starts with the current implementation status. Older preparation/b
- [x] Add configured-YOLO cross-tile duplicate suppression and raw/suppressed calibration evidence fields.
- [x] Rerun live dense-AOI calibration after redeploy with `YOLO_DUPLICATE_IOU_THRESHOLD=0.5`; Westerlo 0.25 improved to F1 `0.2537313432835821` and Turnhout 0.25 improved to F1 `0.14114114114114112`, but the candidate remains rejected.
- [x] Add `OPERATOR_YOLO_MIN_LABEL_VISIBLE_RATIO` so the next overlapping-tile dataset can drop tiny clipped edge-fragment labels.
- [x] Add operator-only larger-AOI sample prep flags so the next training dataset is not limited to one 512x512 tile per documented sample.
- [ ] Find or train a materially stronger aerial/Kempen building model candidate; `geointel-building-yolov8n-expanded160e50-pt` is the best current dense-AOI candidate but still too weak and too noisy for a V1 default.
- [ ] Train a higher-capacity local aerial-building detector with stronger positive recall while preserving the hard-negative false-positive gate.
- [ ] Export and audit a visible-ratio-gated tile dataset on Tower before the next default-model training attempt.
- [ ] Prepare `/app/storage/operator-data/operator-samples-1024` on Tower, then export and audit `yolo-building-aoi1024-visible025` before the next default-model training attempt.
- [ ] Build the next candidate gate around better positive AOI coverage, label strategy and hard-negative retention.
## Sprint 8 status