Continue rejected v37 through frozen loop
GeoIntel release gates / Compile, test, contracts and builds (push) Canceled after 0s
GeoIntel release gates / Python and npm vulnerability policy (push) Canceled after 0s
GeoIntel release gates / GIS image, SBOM and container scan (push) Canceled after 0s

This commit is contained in:
Jens
2026-07-29 16:23:28 +02:00
parent f18a23c060
commit b393d9f65d
5 changed files with 30 additions and 2 deletions
+13
View File
@@ -11804,6 +11804,19 @@ Deployment evidence:
malformed JSON and filesystem errors produce an explicit fail-closed
supervisor state before any subprocess is started. The updated supervisor
was activated live with one remaining v37 GPU process.
- V37 early-stopped normally at epoch 23; epoch 5 remained the immutable best
checkpoint. The supervisor observed `results.png` and executed its one-shot
completion handoff.
- Fixed a completed-checkpoint entry defect exposed by that live handoff: the
iteration directory is now created before copying/hash-binding an existing
model. The repaired calibration ran without opening test/background.
- V37 improved calibration aggregate F1 to `0.568` and Flanders F1 to `0.307`,
but failed Flanders F1/precision/recall and Wallonia precision. It was
rejected and produced checksummed failure-driven sampling for iteration 2.
- Made training patience an explicit orchestrator input. A pre-epoch iteration
2 process exposing the old hardcoded value `35` was terminated before any
result row existed; iteration 2 restarted from the identical checkpoint and
sampling with the frozen `patience=18` contract on CUDA.
- Added those aerial augmentation parameters to the orchestrator CLI and
training command, preventing later failure-driven checkpoints from silently
reverting to generic orientation assumptions.
+2
View File
@@ -976,6 +976,8 @@ This file now starts with the current implementation status. Older preparation/b
- [x] Add a guarded calibration-first entry point for completed checkpoints so v37 and future externally interrupted runs can rejoin the automated loop without redundant retraining.
- [x] Add and activate a host-side, exact-run-marker supervisor that resumes the v37 CUDA checkpoint after container recreation without launching concurrent trainers.
- [x] Bind v37 completion to a one-shot JSON command that enters calibration-first evaluation and up to 20 automatic failure-driven CUDA iterations with the frozen aerial augmentation contract.
- [x] Complete v37 by early stopping at epoch 23, reject its best epoch-5 checkpoint calibration-first, and start closed-loop iteration 2 from checksummed failure-driven sampling.
- [x] Make orchestrator patience explicit and retain the frozen value `18` in every subsequent CUDA iteration.
- [x] Evaluate the completed v36 YOLO11x checkpoint calibration-first on the rotated v30 holdouts; reject it before opening test/background because the regional calibration gate failed.
- [ ] Finish and assess the leak-free v37 YOLO11x failure-driven CUDA iteration; open test/background evidence only if every calibration gate passes.