Jens/geointel
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Recover active training loop checkpoints
GeoIntel release gates / Compile, test, contracts and builds (push) Canceled after 0s
Details
GeoIntel release gates / Python and npm vulnerability policy (push) Canceled after 0s
Details
GeoIntel release gates / GIS image, SBOM and container scan (push) Canceled after 0s
Details

Browse Source
This commit is contained in:
Jens
2026-07-29 16:32:01 +02:00
parent 18387448de
commit e73468318f
7 changed files with 174 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/CODEX_EXECUTION_LOG.md
+9
View File
@@ -11827,6 +11827,15 @@ Deployment evidence:
cleanly from the same candidate and checksummed sampling with
`warmup_epochs=1`, `warmup_bias_lr=0.01`, HSV `0.01/0.2/0.15`, patience 18
and the existing aerial rotation/flip contract.
- Added checkpoint-aware recovery for the state-pending current iteration.
When `runs/<iteration>/weights/last.pt` exists, the orchestrator uses YOLO's
exact `resume=<checkpoint>` CUDA path and records that provenance after the
iteration is assessed.
- Added and activated a host-side parent-loop supervisor. It monitors the exact
orchestrator marker and fail-closed JSON loop state, restores the current
script into a recreated container and relaunches only the versioned argv
command. Its live state is `monitoring`, with zero relaunches and one active
iteration-2 GPU process.
- Added those aerial augmentation parameters to the orchestrator CLI and
training command, preventing later failure-driven checkpoints from silently
reverting to generic orientation assumptions.