block training-seen checkpoint evaluation
GeoIntel release gates / Compile, test, contracts and builds (push) Canceled after 0s
GeoIntel release gates / Python and npm vulnerability policy (push) Canceled after 0s
GeoIntel release gates / GIS image, SBOM and container scan (push) Canceled after 0s

This commit is contained in:
Jens
2026-08-09 21:01:12 +02:00
parent 48084799c9
commit fd45f37a38
7 changed files with 362 additions and 3 deletions
+33
View File
@@ -12704,3 +12704,36 @@ Open:
- Two adjacent positive AOIs and one real background AOI are not representative
release evidence. Human review, broader independent geography and exact
challenger provenance remain blocking.
## 2026-08-09 - Training/evaluation membership gate
### Corrected
- Compared the non-overlap evaluation sample membership against the exact
active and challenger training summaries. Turnhout and Westerlo are absent
from both train splits; Postel-bos is present in both.
- Reclassified the Postel detection counts as training-seen regression checks,
not independent background validation. No existing metric or checkpoint was
promoted on that evidence.
- Extended the checkpoint evaluator with a strict multi-corpus membership gate.
Governed runs now persist a blocked manifest and stop before PyTorch import,
model loading and GPU inference if any evaluation AOI is training-seen.
### Reproduced on Tower
- Active training summary SHA-256:
`49b2a07d2105d08356431757b83eafc1498eaf1fb76965b1efe05b776824942a`.
- Challenger training summary SHA-256:
`7c917e31216d1df2174c0f9c736f88a81f3835aa991971f8fb8665e17ddf5c9c`.
- Evaluation summary SHA-256:
`c17a8c47eaecc2dfb38778ff5a612f9e6c4d67c9e90e31d3a6a66d8857adec4c`.
- The strict run blocked on `postel_bos`, with
`model_loading_attempted=false` and `gpu_inference_attempted=false`.
- Blocked-manifest SHA-256:
`62e8a9377ac87552936f472b0acf55d1d659ebf27f84ba8c4dbcefe7a374e1da`.
### Remaining limitation
- Independent real-background AOIs are still required. Literal 100% model
correctness is not demonstrated; the system now refuses to misrepresent the
known overlap as proof.