Continue rejected v37 through frozen loop
This commit is contained in:
@@ -11804,6 +11804,19 @@ Deployment evidence:
|
||||
malformed JSON and filesystem errors produce an explicit fail-closed
|
||||
supervisor state before any subprocess is started. The updated supervisor
|
||||
was activated live with one remaining v37 GPU process.
|
||||
- V37 early-stopped normally at epoch 23; epoch 5 remained the immutable best
|
||||
checkpoint. The supervisor observed `results.png` and executed its one-shot
|
||||
completion handoff.
|
||||
- Fixed a completed-checkpoint entry defect exposed by that live handoff: the
|
||||
iteration directory is now created before copying/hash-binding an existing
|
||||
model. The repaired calibration ran without opening test/background.
|
||||
- V37 improved calibration aggregate F1 to `0.568` and Flanders F1 to `0.307`,
|
||||
but failed Flanders F1/precision/recall and Wallonia precision. It was
|
||||
rejected and produced checksummed failure-driven sampling for iteration 2.
|
||||
- Made training patience an explicit orchestrator input. A pre-epoch iteration
|
||||
2 process exposing the old hardcoded value `35` was terminated before any
|
||||
result row existed; iteration 2 restarted from the identical checkpoint and
|
||||
sampling with the frozen `patience=18` contract on CUDA.
|
||||
- Added those aerial augmentation parameters to the orchestrator CLI and
|
||||
training command, preventing later failure-driven checkpoints from silently
|
||||
reverting to generic orientation assumptions.
|
||||
|
||||
@@ -976,6 +976,8 @@ This file now starts with the current implementation status. Older preparation/b
|
||||
- [x] Add a guarded calibration-first entry point for completed checkpoints so v37 and future externally interrupted runs can rejoin the automated loop without redundant retraining.
|
||||
- [x] Add and activate a host-side, exact-run-marker supervisor that resumes the v37 CUDA checkpoint after container recreation without launching concurrent trainers.
|
||||
- [x] Bind v37 completion to a one-shot JSON command that enters calibration-first evaluation and up to 20 automatic failure-driven CUDA iterations with the frozen aerial augmentation contract.
|
||||
- [x] Complete v37 by early stopping at epoch 23, reject its best epoch-5 checkpoint calibration-first, and start closed-loop iteration 2 from checksummed failure-driven sampling.
|
||||
- [x] Make orchestrator patience explicit and retain the frozen value `18` in every subsequent CUDA iteration.
|
||||
- [x] Evaluate the completed v36 YOLO11x checkpoint calibration-first on the rotated v30 holdouts; reject it before opening test/background because the regional calibration gate failed.
|
||||
- [ ] Finish and assess the leak-free v37 YOLO11x failure-driven CUDA iteration; open test/background evidence only if every calibration gate passes.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user