correct overlapping checkpoint evaluation
GeoIntel release gates / Compile, test, contracts and builds (push) Canceled after 0s
GeoIntel release gates / Python and npm vulnerability policy (push) Canceled after 0s
GeoIntel release gates / GIS image, SBOM and container scan (push) Canceled after 0s

This commit is contained in:
Jens
2026-08-09 20:41:54 +02:00
parent 2de438b9cc
commit 48084799c9
11 changed files with 539 additions and 10 deletions
+33
View File
@@ -12671,3 +12671,36 @@ Open:
- The min-4px corpus is preferred for future experimental work, but remains
explicitly training/release-ineligible until real human review and governed
release evidence exist. The active GPU model was not changed.
## 2026-08-09 - Non-overlapping checkpoint correction
### Corrected
- Traced the prior 36-image checkpoint matrix to four AOIs tiled at 512/256.
The same interiors were repeated and nine Arendonk-heide tiles were visually
blank no-data, despite being counted as background observations.
- Added a deterministic non-overlap evaluation-view builder. It selects four
complete 512 px grid tiles per 1024 px AOI, excludes low-variance no-data
with reason codes, creates empty train directories and writes a hashed
`NO_TRAINING.json` marker.
- Added a fail-fast marker check to the operator YOLO training wrapper before
release verification, model loading or CUDA allocation.
- Corrected checkpoint overlap semantics: zero pixel overlap is reported, but
statistical independence is explicitly not established for adjacent tiles.
### GPU evidence
- The clean r2 view contains 12 images: eight positive Turnhout/Westerlo tiles
and four real Postel backgrounds. It has 2,551 labels, 2,384 unique interior
objects, 167 edge labels and zero repeated interior objects.
- Active: precision 0.542632, recall 0.435317, mAP50 0.342034, mAP50-95
0.141316. Challenger: 0.581715, 0.462300, 0.368364 and 0.155318.
- At confidence 0.15 the active model has zero Postel detections and the
challenger one; at 0.25 both have zero. The threshold-sensitive relative
challenger advantage is confirmed without changing production.
### Remaining limitations
- Two adjacent positive AOIs and one real background AOI are not representative
release evidence. Human review, broader independent geography and exact
challenger provenance remain blocking.
+7
View File
@@ -1170,6 +1170,13 @@ This file now starts with the current implementation status. Older preparation/b
- [ ] Compare the existing 0.35 edge-visibility policy against a separately
versioned stricter ablation from original source geometry before any new
governed training; do not approximate this by dropping every edge label.
- [x] Replace the overlapping checkpoint ranking with a checksum-bound,
training-disabled non-overlap view: 12 real tiles, 2,384 unique interior
objects, zero repeated interiors and explicit removal of blank Arendonk
no-data imagery.
- [ ] Add multiple independent real-background AOIs and more geographically
separated positive validation AOIs before interpreting non-overlap ranking
as release evidence; Postel alone is not representative.
- [ ] Convert the AI-assisted ledger into no stronger claim than experimental
triage; a real human must independently review and sign the frozen artifacts
before the governed training wrapper may unlock.
@@ -187,3 +187,32 @@ visible; it was not suppressed or relabelled as success.
This min-4px version is the preferred experimental successor to the legacy
min-3px corpus. It explicitly remains ineligible for governed training until
real human review and all release-contract evidence exist.
## Non-overlapping checkpoint re-evaluation
The 36-tile checkpoint matrix was traced to a 512 px validation corpus with
stride 256. It contained nine views each of Turnhout, Westerlo, Postel-bos and
Arendonk-heide. Visual inspection then exposed that all Arendonk-heide views
are blank/no-data imagery, not meaningful pure-background observations. The
historical claim of 18 background images is therefore corrected: nine were
blank no-data and nine represented Postel under overlap.
A training-disabled validation view now covers each 1024 px AOI with four
non-overlapping 512 px tiles. Four blank Arendonk representatives are excluded
with explicit reason codes. The resulting set contains eight positive tiles
from Turnhout/Westerlo, four real Postel background tiles and 2,551 labels.
Exact reconstruction finds 2,384 unique interior objects, 167 edge labels and
zero repeated interior objects. The view has empty train directories, a
checksum-bound `NO_TRAINING.json`, and the training wrapper rejects that marker.
On the Tower RTX 4080 SUPER, the active checkpoint measures precision
`0.542632`, recall `0.435317`, mAP50 `0.342034` and mAP50-95 `0.141316`.
The reviewedexp6 challenger measures `0.581715`, `0.462300`, `0.368364` and
`0.155318` respectively. At background confidence 0.15 the active checkpoint
has zero Postel detections and the challenger has one. At 0.25 both have zero.
Thus the refined evidence confirms the challenger's relative metric advantage
and its threshold sensitivity, but still does not authorize promotion.
Non-overlap is not overstated as statistical independence: the positive tiles
remain adjacent and come from only two AOIs, edge objects can remain split, and
there is only one real background AOI. The production model remains unchanged.