Record YOLOv8s partial candidate evaluation

2026-07-08 18:32:09 +02:00
parent 282f5b0d0c
commit e0ab504c81
3 changed files with 55 additions and 0 deletions
@@ -1,3 +1,48 @@
+## Sprint 144 YOLOv8s hardneg r8 partial candidate evaluation (2026-07-08)
+
+Changed:
+- Started a Tower-local YOLOv8s training run using:
+  - dataset `/app/storage/operator-data/yolo-building-tile-hardneg160r8/dataset.yaml`
+  - base model `/app/models/yolov8s.pt`
+  - requested epochs `60`
+  - image size `640`
+  - batch `2`
+  - device `cpu`
+- The Codex command reached its 1-hour timeout after 12 completed epochs; the run had produced `weights/best.pt` and `weights/last.pt`.
+- Preserved the partial best artifact as `/app/models/geointel-building-yolov8s-hardneg160r8e12partial.pt`.
+- Wrote `/app/storage/training/operator-yolo/geointel-building-yolov8s-hardneg160r8e60/training_summary_partial_e12.json`.
+- Treated the artifact explicitly as a partial evaluation candidate, not as a completed 60-epoch model.
+
+Tested:
+- Live model catalog listed `geointel-building-yolov8s-hardneg160r8e12partial-pt` with SHA256 `0246202cddc47eb994a0afc9ee10d56b72298bd1cdc0b72b75e12b28e2202330`.
+- Ran 7-AOI positive matrix:
+  - output `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/yolov8s-hardneg160r8e12partial-positive-20260708/multi_sample_quality_summary.json`
+  - sample count `7`
+  - run count `21`
+  - best result: Westerlo threshold `0.05`, F1/score `0.14826498422712936`, precision `0.15666666666666668`, recall `0.1407185628742515`, detections `300`, false positives `253`, false negatives `287`.
+- Ran hard-negative matrix:
+  - output `/mnt/user/appdata/geointel/artifacts/detection-hard-negatives/yolov8s-hardneg160r8e12partial-live/hard_negative_matrix_summary.json`
+  - Postel-bos detections `0/0/0` at thresholds `0.05/0.15/0.25`
+  - Lommel-heide detections `0/0/0`
+  - Kasterlee-bos detections `18/1/0`
+- Assembled evidence portfolio:
+  - output `/mnt/user/appdata/geointel/artifacts/detection-calibration-portfolio/yolov8s-hardneg160r8e12partial-positive-20260708/output/calibration_evidence_portfolio.json`
+  - sample count `7`
+  - evidence features `12908`
+  - role counts `false_negative=10861`, `false_positive=1785`, `match_candidate=131`, `match_reference=131`
+- Ran promotion report:
+  - output `/mnt/user/appdata/geointel/artifacts/detection-model-promotion/yolov8s-hardneg160r8e12partial-20260708/detection_model_promotion_report.json`
+  - evaluated `3` candidate thresholds
+  - recommended candidate `none`
+  - threshold `0.05` rejected for `positive_mean_f1_below_gate` and `background_false_positive_pressure`, with mean F1 `0.05026994383963278` and max background detections `18`
+  - threshold `0.15` rejected for the same reasons, with mean F1 `0.0022606965174129354` and max background detections `1`
+  - threshold `0.25` rejected for insufficient positive evidence and positive F1 below gate.
+
+Open:
+- The partial YOLOv8s r8 candidate is materially worse than the existing `expanded160e50` positive-AOI baseline and must not be activated.
+- CPU-only training is too slow for a complete 60-epoch YOLOv8s pass inside a 1-hour interactive command window.
+- Next pass should either resume/finish long-running training outside the interactive timeout or use GPU/accelerated runtime; only then rerun the same positive, hard-negative, evidence portfolio and promotion gates.
+
 ## Sprint 143 Detection model promotion decision report (2026-07-08)

 Changed:
@@ -431,5 +431,7 @@ This file now starts with the current implementation status. Older preparation/b
 - [x] Preserve model/tile provenance in calibration evidence bundle summaries.
 - [x] Prevent same-threshold calibration evidence responses from overwriting each other in multi-model portfolios.
 - [x] Add a model promotion decision report that combines positive-AOI score with hard-negative false-positive pressure.
+- [x] Train and reject a YOLOv8s hard-negative r8 partial candidate after 12 CPU epochs through the full positive/background promotion gate.
 - [ ] Add more AOIs after the tile-level baseline so the next local model attempt is not limited to Geel/Mol/Turnhout.
 - [ ] Add negative/background AOIs so the next tile dataset is not all positive tiles.
+- [ ] Finish or resume a long-running higher-capacity YOLO training pass outside the 1-hour interactive command window, then rerun the same promotion gate.