Launch v42 building training loop
This commit is contained in:
@@ -11905,3 +11905,38 @@ Open:
|
|||||||
and pure-background results only if every regional calibration gate passes.
|
and pure-background results only if every regional calibration gate passes.
|
||||||
- Continue failure-driven, train-only corpus iterations until all objective
|
- Continue failure-driven, train-only corpus iterations until all objective
|
||||||
gates pass; only then request the queued representative human review.
|
gates pass; only then request the queued representative human review.
|
||||||
|
|
||||||
|
## 2026-07-29 - V42 regional corpus expansion and v43 loop handoff
|
||||||
|
|
||||||
|
Changed:
|
||||||
|
|
||||||
|
- Added and acquired 14 independent train-only AOIs spanning Flemish coastal,
|
||||||
|
port, dunes, industrial and ribbon contexts plus Walloon architecture,
|
||||||
|
rural, industrial, quarry, forest and field contexts.
|
||||||
|
- Built immutable 156-sample v42 evidence, applied the governed temporal
|
||||||
|
filters, refined the new labels with SAM2 on CUDA and retiled all new source
|
||||||
|
labels with 384 px tiles and 128 px overlap without uncovered labels.
|
||||||
|
- Rotated the training corpus while retaining the exact protected v31
|
||||||
|
calibration, test and background assignments.
|
||||||
|
- Added the versioned v43 command for calibration-first evaluation followed by
|
||||||
|
at most 20 failure-driven CUDA iterations. It starts from the rejected v38
|
||||||
|
iteration-5 checkpoint and shortens patience to 8 because all preceding runs
|
||||||
|
selected epoch 2 while later epochs overfit the protected regional pattern.
|
||||||
|
|
||||||
|
Verified:
|
||||||
|
|
||||||
|
- Corpus audit: 156 samples, no automated failures and no reported leakage.
|
||||||
|
- Training tile audit: `ok`; 2,160 tiles, 56,474 valid labels, zero invalid or
|
||||||
|
missing labels, 270 negative tiles and zero repeated background negatives.
|
||||||
|
- New-AOI visual renderer: `ok`; 56 rendered tiles spanning all 14 added AOIs,
|
||||||
|
with zero missing inputs, invalid rows or low-variance imagery. Manual
|
||||||
|
AI-assisted inspection found labels aligned with visible roof footprints and
|
||||||
|
retained the deliberately sparse hard-negative contexts.
|
||||||
|
- NVIDIA preflight: RTX 4080 SUPER idle and available before the v43 handoff.
|
||||||
|
|
||||||
|
Open:
|
||||||
|
|
||||||
|
- Complete v43 calibration convergence, then and only then evaluate the closed
|
||||||
|
protected test and pure-background sets.
|
||||||
|
- Obtain final representative human contact-sheet approval, promote the exact
|
||||||
|
checksummed model, and redeploy from the canonical Tower directory.
|
||||||
|
|||||||
@@ -1010,3 +1010,14 @@ This file now starts with the current implementation status. Older preparation/b
|
|||||||
- [x] Produce a dark campaign cover, architecture visual, AI pipeline visual and animated project GIF.
|
- [x] Produce a dark campaign cover, architecture visual, AI pipeline visual and animated project GIF.
|
||||||
- [x] Generate and visually verify a six-page portfolio case-study PDF.
|
- [x] Generate and visually verify a six-page portfolio case-study PDF.
|
||||||
- [x] Commit, push and redeploy the complete presentation release from `/mnt/user/appdata/geointel`.
|
- [x] Commit, push and redeploy the complete presentation release from `/mnt/user/appdata/geointel`.
|
||||||
|
|
||||||
|
# Sprint 232 - V42 Belgian building generalisation loop
|
||||||
|
|
||||||
|
- [x] Add 14 independent coastal, industrial, rural, quarry, forest, field and port training AOIs across Flanders and Wallonia.
|
||||||
|
- [x] Acquire, temporally filter, SAM2-refine and overlap-retile the new AOIs on the RTX 4080 SUPER.
|
||||||
|
- [x] Assemble and checksum the immutable 156-sample v42 corpus while preserving the protected v31 calibration/test/background assignments.
|
||||||
|
- [x] Pass structural corpus and 56,474-label tile-quality audits with zero invalid or missing labels and zero spatial leakage failures.
|
||||||
|
- [x] Render and inspect a 56-tile contact sheet covering all 14 new AOIs before retraining.
|
||||||
|
- [ ] Run the v43 calibration-first, failure-driven CUDA loop against the frozen regional release gates.
|
||||||
|
- [ ] Open protected test and pure-background evidence only after every calibration gate passes.
|
||||||
|
- [ ] Queue final representative human sign-off only after all automated gates pass, then promote and redeploy the exact checksummed model.
|
||||||
|
|||||||
@@ -0,0 +1,73 @@
|
|||||||
|
[
|
||||||
|
"docker",
|
||||||
|
"exec",
|
||||||
|
"-d",
|
||||||
|
"geointel",
|
||||||
|
"/opt/geointel/venv/bin/python",
|
||||||
|
"/app/scripts/run_belgium_building_training_loop.py",
|
||||||
|
"--initial-model",
|
||||||
|
"/app/storage/training/building-be-v38-v31-closed-loop-r1/iteration-005/candidate.pt",
|
||||||
|
"--train-yaml",
|
||||||
|
"/app/storage/operator-data/building-be-v42-rotated-holdouts-r1/train/dataset.yaml",
|
||||||
|
"--train-summary",
|
||||||
|
"/app/storage/operator-data/building-be-v42-rotated-holdouts-r1/train/yolo_tile_dataset_summary.json",
|
||||||
|
"--dataset-audit",
|
||||||
|
"/app/storage/operator-data/building-be-v42-rotated-holdouts-r1/audit/belgium-building-corpus-audit.json",
|
||||||
|
"--train-quality-audit",
|
||||||
|
"/app/storage/operator-data/building-be-v42-rotated-holdouts-r1/train/audit/operator_yolo_dataset_quality_audit.json",
|
||||||
|
"--calibration-summary",
|
||||||
|
"/app/storage/operator-data/building-be-v42-rotated-holdouts-r1/calibration/yolo_tile_dataset_summary.json",
|
||||||
|
"--test-summary",
|
||||||
|
"/app/storage/operator-data/building-be-v42-rotated-holdouts-r1/test/yolo_tile_dataset_summary.json",
|
||||||
|
"--background-summary",
|
||||||
|
"/app/storage/operator-data/building-be-v42-rotated-holdouts-r1/background-test/yolo_tile_dataset_summary.json",
|
||||||
|
"--corpus-manifest",
|
||||||
|
"/app/storage/operator-data/building-be-v42-rotated-holdouts-r1/operator_samples_manifest.json",
|
||||||
|
"--output-dir",
|
||||||
|
"/app/storage/training/building-be-v43-v42-closed-loop-r1",
|
||||||
|
"--iterations",
|
||||||
|
"20",
|
||||||
|
"--epochs",
|
||||||
|
"80",
|
||||||
|
"--patience",
|
||||||
|
"8",
|
||||||
|
"--batch",
|
||||||
|
"8",
|
||||||
|
"--workers",
|
||||||
|
"0",
|
||||||
|
"--max-det",
|
||||||
|
"1000",
|
||||||
|
"--imgsz",
|
||||||
|
"640",
|
||||||
|
"--optimizer",
|
||||||
|
"AdamW",
|
||||||
|
"--lr0",
|
||||||
|
"0.0001",
|
||||||
|
"--mosaic",
|
||||||
|
"0",
|
||||||
|
"--scale",
|
||||||
|
"0.15",
|
||||||
|
"--translate",
|
||||||
|
"0.05",
|
||||||
|
"--degrees",
|
||||||
|
"180",
|
||||||
|
"--flipud",
|
||||||
|
"0.5",
|
||||||
|
"--fliplr",
|
||||||
|
"0.5",
|
||||||
|
"--warmup-epochs",
|
||||||
|
"1",
|
||||||
|
"--warmup-bias-lr",
|
||||||
|
"0.01",
|
||||||
|
"--hsv-h",
|
||||||
|
"0.01",
|
||||||
|
"--hsv-s",
|
||||||
|
"0.2",
|
||||||
|
"--hsv-v",
|
||||||
|
"0.15",
|
||||||
|
"--seed",
|
||||||
|
"20260942",
|
||||||
|
"--yolo",
|
||||||
|
"/opt/geointel/venv/bin/yolo",
|
||||||
|
"--evaluate-initial-model"
|
||||||
|
]
|
||||||
Reference in New Issue
Block a user