Handoff completed training into closed loop
GeoIntel release gates / Compile, test, contracts and builds (push) Canceled after 0s
GeoIntel release gates / Python and npm vulnerability policy (push) Canceled after 0s
GeoIntel release gates / GIS image, SBOM and container scan (push) Canceled after 0s

This commit is contained in:
Jens
2026-07-29 15:45:41 +02:00
parent d5bf9c102e
commit d6502140da
8 changed files with 117 additions and 1 deletions
+5
View File
@@ -101,6 +101,11 @@ An already completed out-of-band checkpoint enters the same contract with
hashes the checkpoint, and begins at calibration. A rejection then follows
the identical failure-driven CUDA path and cannot open protected test evidence
early.
Tower's v37 supervisor binds the completed `results.png` artifact to a
versioned JSON argv list. The handoff starts the orchestrator detached exactly
once; shell strings are not accepted. Subsequent iterations retain the frozen
180-degree aerial rotation, vertical/horizontal flip, scale and translation
parameters rather than silently reverting to generic augmentation defaults.
The orchestrator refuses to start unless every automated frozen-dataset gate
passes and the corpus contains zero blank/low-variance positive tiles. The
audit status may remain `needs_human_review` while training and objective
+8
View File
@@ -11794,6 +11794,14 @@ Deployment evidence:
transient duplicate resume processes were detected and terminated before
another epoch completed. Detection now uses `docker top -eo pid,args`; a
live one-shot check returned `monitoring` with exactly one GPU process.
- Extended the supervisor with a one-time, JSON-list-only completion handoff.
The active v37 supervisor now starts the calibration-first closed loop when
`results.png` appears. The bound command permits up to 20 iterations, keeps
protected test/background closed until calibration passes, and preserves
the exact AdamW, 180-degree rotation, flip, scale and translation contract.
- Added those aerial augmentation parameters to the orchestrator CLI and
training command, preventing later failure-driven checkpoints from silently
reverting to generic orientation assumptions.
- Confirmed v37 epoch 1 completed on CUDA with validation precision `0.601`,
recall `0.455`, mAP50 `0.474` and mAP50-95 `0.205`; the run remains inactive
and these internal-validation metrics are not release evidence.
+1
View File
@@ -975,6 +975,7 @@ This file now starts with the current implementation status. Older preparation/b
- [x] Persist checksummed train-only failure-driven sampling after every rejected loop iteration and resume the next checkpoint from that exact dataset YAML.
- [x] Add a guarded calibration-first entry point for completed checkpoints so v37 and future externally interrupted runs can rejoin the automated loop without redundant retraining.
- [x] Add and activate a host-side, exact-run-marker supervisor that resumes the v37 CUDA checkpoint after container recreation without launching concurrent trainers.
- [x] Bind v37 completion to a one-shot JSON command that enters calibration-first evaluation and up to 20 automatic failure-driven CUDA iterations with the frozen aerial augmentation contract.
- [x] Evaluate the completed v36 YOLO11x checkpoint calibration-first on the rotated v30 holdouts; reject it before opening test/background because the regional calibration gate failed.
- [ ] Finish and assess the leak-free v37 YOLO11x failure-driven CUDA iteration; open test/background evidence only if every calibration gate passes.