block training-seen checkpoint evaluation
This commit is contained in:
@@ -12704,3 +12704,36 @@ Open:
|
||||
- Two adjacent positive AOIs and one real background AOI are not representative
|
||||
release evidence. Human review, broader independent geography and exact
|
||||
challenger provenance remain blocking.
|
||||
|
||||
## 2026-08-09 - Training/evaluation membership gate
|
||||
|
||||
### Corrected
|
||||
|
||||
- Compared the non-overlap evaluation sample membership against the exact
|
||||
active and challenger training summaries. Turnhout and Westerlo are absent
|
||||
from both train splits; Postel-bos is present in both.
|
||||
- Reclassified the Postel detection counts as training-seen regression checks,
|
||||
not independent background validation. No existing metric or checkpoint was
|
||||
promoted on that evidence.
|
||||
- Extended the checkpoint evaluator with a strict multi-corpus membership gate.
|
||||
Governed runs now persist a blocked manifest and stop before PyTorch import,
|
||||
model loading and GPU inference if any evaluation AOI is training-seen.
|
||||
|
||||
### Reproduced on Tower
|
||||
|
||||
- Active training summary SHA-256:
|
||||
`49b2a07d2105d08356431757b83eafc1498eaf1fb76965b1efe05b776824942a`.
|
||||
- Challenger training summary SHA-256:
|
||||
`7c917e31216d1df2174c0f9c736f88a81f3835aa991971f8fb8665e17ddf5c9c`.
|
||||
- Evaluation summary SHA-256:
|
||||
`c17a8c47eaecc2dfb38778ff5a612f9e6c4d67c9e90e31d3a6a66d8857adec4c`.
|
||||
- The strict run blocked on `postel_bos`, with
|
||||
`model_loading_attempted=false` and `gpu_inference_attempted=false`.
|
||||
- Blocked-manifest SHA-256:
|
||||
`62e8a9377ac87552936f472b0acf55d1d659ebf27f84ba8c4dbcefe7a374e1da`.
|
||||
|
||||
### Remaining limitation
|
||||
|
||||
- Independent real-background AOIs are still required. Literal 100% model
|
||||
correctness is not demonstrated; the system now refuses to misrepresent the
|
||||
known overlap as proof.
|
||||
|
||||
Reference in New Issue
Block a user