block checkpoint evaluation on ancestral exposure
GeoIntel release gates / Compile, test, contracts and builds (push) Canceled after 0s
GeoIntel release gates / Python and npm vulnerability policy (push) Canceled after 0s
GeoIntel release gates / GIS image, SBOM and container scan (push) Canceled after 0s

This commit is contained in:
Jens
2026-08-09 22:10:50 +02:00
parent fd45f37a38
commit 116b8e291e
8 changed files with 390 additions and 61 deletions
+34
View File
@@ -12737,3 +12737,37 @@ Open:
- Independent real-background AOIs are still required. Literal 100% model
correctness is not demonstrated; the system now refuses to misrepresent the
known overlap as proof.
## 2026-08-09 - Full ancestral-corpus exposure correction
### Corrected
- Reconstructed the retained checkpoint ancestry from exact Ultralytics
`args.yaml` files and model hashes. The active model descends from the
expanded-min4 corpus; the challenger descends from the active model.
- Found that the expanded ancestor used Postel-bos for training and
Turnhout/Westerlo for validation. All three v69 evaluation AOIs were therefore
exposed somewhere in the model family.
- Replaced last-corpus/train-only checking with fail-closed inspection of every
split in every supplied ancestral corpus. Legacy CLI spellings remain safe
aliases to the full-lineage behavior.
### Verified
- The Tower gate blocked all three exposed AOIs with zero model-loading and
zero GPU-inference attempts.
- Expanded ancestor summary SHA-256:
`1887e2ba5c719d33675a9ed85db99a274fd66d98d5f6ad44558d673620917201`.
- Blocked-manifest SHA-256:
`f6effd94befd6df8d699bc66fbbea8a860b056cff9996be5f867f858807af65e`.
- Model-lineage receipt SHA-256:
`84d8d7bd95acfb07b188b07ececfc5417fca80c087d7c04442dd2a4da5f43936`;
every retained training `best.pt` matches its named model copy byte-for-byte.
- The production checkpoint and container image remain unchanged.
### Accuracy boundary
- The old Turnhout/Westerlo/Postel numbers are regression diagnostics only.
No independent positive or background accuracy evidence currently exists for
this model family; new AOIs must be provisioned without ancestry or spatial
exposure before another ranking is defensible.
+3 -2
View File
@@ -1179,8 +1179,9 @@ This file now starts with the current implementation status. Older preparation/b
as release evidence. Postel occurs in both compared train splits and is only
a training-seen regression check, not independent validation.
- [x] Add a fail-closed checkpoint gate that checks every evaluation AOI
against the exact train split of every supplied model corpus before PyTorch
import, model loading or GPU inference.
against every split of every supplied ancestral model corpus before PyTorch
import, model loading or GPU inference. Turnhout/Westerlo were exposed as
validation in the base building model; Postel was exposed as training.
- [ ] Convert the AI-assisted ledger into no stronger claim than experimental
triage; a real human must independently review and sign the frozen artifacts
before the governed training wrapper may unlock.
@@ -237,3 +237,29 @@ mode, fails before PyTorch import, model loading or GPU inference when any
evaluation AOI overlaps any supplied train split. The reproduced gate blocked
on `postel_bos` for both checkpoints. Its immutable machine-readable record is
`artifacts/evidence/accuracy/model-training/20260809-v70-evaluation-independence-gate.json`.
## Full model-lineage correction
The preceding correction still considered only the final fine-tune corpus of
each checkpoint. Exact retained Ultralytics arguments establish a longer
ancestry: the active checkpoint was initialized from
`geointel-building-yolov8s-aoi1024expandedminpx4vis035e50`, which was initialized
from the generic `yolov8s.pt`; the challenger was then initialized from the
active checkpoint. The copied model assets and retained `best.pt` files match
byte-for-byte at each building-model stage.
The ancestral expanded corpus exposes all three evaluation AOIs: `postel_bos`
as train, and `turnhout` plus `westerlo` as validation. Therefore none of the
v69 AOIs is independent of the complete model family. Turnhout/Westerlo can
still be used as familiar regression diagnostics, but their metrics are not a
fresh candidate-ranking result and must not support accuracy, uncertainty,
generalisation or release claims.
The gate now requires every ancestral corpus summary and checks every recorded
split, including validation and calibration. A reproduced Tower run blocked on
all three AOIs before PyTorch import, model loading or GPU inference. Evidence:
`artifacts/evidence/accuracy/model-training/20260809-v71-full-lineage-independence-gate.json`.
The byte-matching parent/output chain is retained separately in
`artifacts/evidence/accuracy/model-training/20260809-v71-model-lineage-receipt.json`.
Fresh geographically separated AOIs with complete spatial and lineage checks
are required for the next meaningful evaluation.