block checkpoint evaluation on ancestral exposure
This commit is contained in:
@@ -12737,3 +12737,37 @@ Open:
|
||||
- Independent real-background AOIs are still required. Literal 100% model
|
||||
correctness is not demonstrated; the system now refuses to misrepresent the
|
||||
known overlap as proof.
|
||||
|
||||
## 2026-08-09 - Full ancestral-corpus exposure correction
|
||||
|
||||
### Corrected
|
||||
|
||||
- Reconstructed the retained checkpoint ancestry from exact Ultralytics
|
||||
`args.yaml` files and model hashes. The active model descends from the
|
||||
expanded-min4 corpus; the challenger descends from the active model.
|
||||
- Found that the expanded ancestor used Postel-bos for training and
|
||||
Turnhout/Westerlo for validation. All three v69 evaluation AOIs were therefore
|
||||
exposed somewhere in the model family.
|
||||
- Replaced last-corpus/train-only checking with fail-closed inspection of every
|
||||
split in every supplied ancestral corpus. Legacy CLI spellings remain safe
|
||||
aliases to the full-lineage behavior.
|
||||
|
||||
### Verified
|
||||
|
||||
- The Tower gate blocked all three exposed AOIs with zero model-loading and
|
||||
zero GPU-inference attempts.
|
||||
- Expanded ancestor summary SHA-256:
|
||||
`1887e2ba5c719d33675a9ed85db99a274fd66d98d5f6ad44558d673620917201`.
|
||||
- Blocked-manifest SHA-256:
|
||||
`f6effd94befd6df8d699bc66fbbea8a860b056cff9996be5f867f858807af65e`.
|
||||
- Model-lineage receipt SHA-256:
|
||||
`84d8d7bd95acfb07b188b07ececfc5417fca80c087d7c04442dd2a4da5f43936`;
|
||||
every retained training `best.pt` matches its named model copy byte-for-byte.
|
||||
- The production checkpoint and container image remain unchanged.
|
||||
|
||||
### Accuracy boundary
|
||||
|
||||
- The old Turnhout/Westerlo/Postel numbers are regression diagnostics only.
|
||||
No independent positive or background accuracy evidence currently exists for
|
||||
this model family; new AOIs must be provisioned without ancestry or spatial
|
||||
exposure before another ranking is defensible.
|
||||
|
||||
+3
-2
@@ -1179,8 +1179,9 @@ This file now starts with the current implementation status. Older preparation/b
|
||||
as release evidence. Postel occurs in both compared train splits and is only
|
||||
a training-seen regression check, not independent validation.
|
||||
- [x] Add a fail-closed checkpoint gate that checks every evaluation AOI
|
||||
against the exact train split of every supplied model corpus before PyTorch
|
||||
import, model loading or GPU inference.
|
||||
against every split of every supplied ancestral model corpus before PyTorch
|
||||
import, model loading or GPU inference. Turnhout/Westerlo were exposed as
|
||||
validation in the base building model; Postel was exposed as training.
|
||||
- [ ] Convert the AI-assisted ledger into no stronger claim than experimental
|
||||
triage; a real human must independently review and sign the frozen artifacts
|
||||
before the governed training wrapper may unlock.
|
||||
|
||||
@@ -237,3 +237,29 @@ mode, fails before PyTorch import, model loading or GPU inference when any
|
||||
evaluation AOI overlaps any supplied train split. The reproduced gate blocked
|
||||
on `postel_bos` for both checkpoints. Its immutable machine-readable record is
|
||||
`artifacts/evidence/accuracy/model-training/20260809-v70-evaluation-independence-gate.json`.
|
||||
|
||||
## Full model-lineage correction
|
||||
|
||||
The preceding correction still considered only the final fine-tune corpus of
|
||||
each checkpoint. Exact retained Ultralytics arguments establish a longer
|
||||
ancestry: the active checkpoint was initialized from
|
||||
`geointel-building-yolov8s-aoi1024expandedminpx4vis035e50`, which was initialized
|
||||
from the generic `yolov8s.pt`; the challenger was then initialized from the
|
||||
active checkpoint. The copied model assets and retained `best.pt` files match
|
||||
byte-for-byte at each building-model stage.
|
||||
|
||||
The ancestral expanded corpus exposes all three evaluation AOIs: `postel_bos`
|
||||
as train, and `turnhout` plus `westerlo` as validation. Therefore none of the
|
||||
v69 AOIs is independent of the complete model family. Turnhout/Westerlo can
|
||||
still be used as familiar regression diagnostics, but their metrics are not a
|
||||
fresh candidate-ranking result and must not support accuracy, uncertainty,
|
||||
generalisation or release claims.
|
||||
|
||||
The gate now requires every ancestral corpus summary and checks every recorded
|
||||
split, including validation and calibration. A reproduced Tower run blocked on
|
||||
all three AOIs before PyTorch import, model loading or GPU inference. Evidence:
|
||||
`artifacts/evidence/accuracy/model-training/20260809-v71-full-lineage-independence-gate.json`.
|
||||
The byte-matching parent/output chain is retained separately in
|
||||
`artifacts/evidence/accuracy/model-training/20260809-v71-model-lineage-receipt.json`.
|
||||
Fresh geographically separated AOIs with complete spatial and lineage checks
|
||||
are required for the next meaningful evaluation.
|
||||
|
||||
Reference in New Issue
Block a user