correct overlapping checkpoint evaluation
This commit is contained in:
@@ -12671,3 +12671,36 @@ Open:
|
||||
- The min-4px corpus is preferred for future experimental work, but remains
|
||||
explicitly training/release-ineligible until real human review and governed
|
||||
release evidence exist. The active GPU model was not changed.
|
||||
|
||||
## 2026-08-09 - Non-overlapping checkpoint correction
|
||||
|
||||
### Corrected
|
||||
|
||||
- Traced the prior 36-image checkpoint matrix to four AOIs tiled at 512/256.
|
||||
The same interiors were repeated and nine Arendonk-heide tiles were visually
|
||||
blank no-data, despite being counted as background observations.
|
||||
- Added a deterministic non-overlap evaluation-view builder. It selects four
|
||||
complete 512 px grid tiles per 1024 px AOI, excludes low-variance no-data
|
||||
with reason codes, creates empty train directories and writes a hashed
|
||||
`NO_TRAINING.json` marker.
|
||||
- Added a fail-fast marker check to the operator YOLO training wrapper before
|
||||
release verification, model loading or CUDA allocation.
|
||||
- Corrected checkpoint overlap semantics: zero pixel overlap is reported, but
|
||||
statistical independence is explicitly not established for adjacent tiles.
|
||||
|
||||
### GPU evidence
|
||||
|
||||
- The clean r2 view contains 12 images: eight positive Turnhout/Westerlo tiles
|
||||
and four real Postel backgrounds. It has 2,551 labels, 2,384 unique interior
|
||||
objects, 167 edge labels and zero repeated interior objects.
|
||||
- Active: precision 0.542632, recall 0.435317, mAP50 0.342034, mAP50-95
|
||||
0.141316. Challenger: 0.581715, 0.462300, 0.368364 and 0.155318.
|
||||
- At confidence 0.15 the active model has zero Postel detections and the
|
||||
challenger one; at 0.25 both have zero. The threshold-sensitive relative
|
||||
challenger advantage is confirmed without changing production.
|
||||
|
||||
### Remaining limitations
|
||||
|
||||
- Two adjacent positive AOIs and one real background AOI are not representative
|
||||
release evidence. Human review, broader independent geography and exact
|
||||
challenger provenance remain blocking.
|
||||
|
||||
@@ -1170,6 +1170,13 @@ This file now starts with the current implementation status. Older preparation/b
|
||||
- [ ] Compare the existing 0.35 edge-visibility policy against a separately
|
||||
versioned stricter ablation from original source geometry before any new
|
||||
governed training; do not approximate this by dropping every edge label.
|
||||
- [x] Replace the overlapping checkpoint ranking with a checksum-bound,
|
||||
training-disabled non-overlap view: 12 real tiles, 2,384 unique interior
|
||||
objects, zero repeated interiors and explicit removal of blank Arendonk
|
||||
no-data imagery.
|
||||
- [ ] Add multiple independent real-background AOIs and more geographically
|
||||
separated positive validation AOIs before interpreting non-overlap ranking
|
||||
as release evidence; Postel alone is not representative.
|
||||
- [ ] Convert the AI-assisted ledger into no stronger claim than experimental
|
||||
triage; a real human must independently review and sign the frozen artifacts
|
||||
before the governed training wrapper may unlock.
|
||||
|
||||
@@ -187,3 +187,32 @@ visible; it was not suppressed or relabelled as success.
|
||||
This min-4px version is the preferred experimental successor to the legacy
|
||||
min-3px corpus. It explicitly remains ineligible for governed training until
|
||||
real human review and all release-contract evidence exist.
|
||||
|
||||
## Non-overlapping checkpoint re-evaluation
|
||||
|
||||
The 36-tile checkpoint matrix was traced to a 512 px validation corpus with
|
||||
stride 256. It contained nine views each of Turnhout, Westerlo, Postel-bos and
|
||||
Arendonk-heide. Visual inspection then exposed that all Arendonk-heide views
|
||||
are blank/no-data imagery, not meaningful pure-background observations. The
|
||||
historical claim of 18 background images is therefore corrected: nine were
|
||||
blank no-data and nine represented Postel under overlap.
|
||||
|
||||
A training-disabled validation view now covers each 1024 px AOI with four
|
||||
non-overlapping 512 px tiles. Four blank Arendonk representatives are excluded
|
||||
with explicit reason codes. The resulting set contains eight positive tiles
|
||||
from Turnhout/Westerlo, four real Postel background tiles and 2,551 labels.
|
||||
Exact reconstruction finds 2,384 unique interior objects, 167 edge labels and
|
||||
zero repeated interior objects. The view has empty train directories, a
|
||||
checksum-bound `NO_TRAINING.json`, and the training wrapper rejects that marker.
|
||||
|
||||
On the Tower RTX 4080 SUPER, the active checkpoint measures precision
|
||||
`0.542632`, recall `0.435317`, mAP50 `0.342034` and mAP50-95 `0.141316`.
|
||||
The reviewedexp6 challenger measures `0.581715`, `0.462300`, `0.368364` and
|
||||
`0.155318` respectively. At background confidence 0.15 the active checkpoint
|
||||
has zero Postel detections and the challenger has one. At 0.25 both have zero.
|
||||
Thus the refined evidence confirms the challenger's relative metric advantage
|
||||
and its threshold sensitivity, but still does not authorize promotion.
|
||||
|
||||
Non-overlap is not overstated as statistical independence: the positive tiles
|
||||
remain adjacent and come from only two AOIs, edge objects can remain split, and
|
||||
there is only one real background AOI. The production model remains unchanged.
|
||||
|
||||
Reference in New Issue
Block a user