audit YOLO geometry and overlapping validation rows
GeoIntel release gates / Compile, test, contracts and builds (push) Canceled after 0s
GeoIntel release gates / Python and npm vulnerability policy (push) Canceled after 0s
GeoIntel release gates / GIS image, SBOM and container scan (push) Canceled after 0s

This commit is contained in:
Jens
2026-08-09 19:36:15 +02:00
parent fc18e72c7f
commit 736e773fb8
13 changed files with 789 additions and 16 deletions
+43
View File
@@ -12595,3 +12595,46 @@ Open:
- This is explicit AI-assisted triage, not human adjudication. It improves the
evidence and review precision but does not unlock governed production
training or checkpoint promotion.
## 2026-08-09 - Row-level geometry and cross-tile independence audit
### Improved
- Added a reproducible, read-only label-row auditor for sub-threshold pixel
dimensions, extreme aspect ratios and tile-edge clipping, with separate
category manifests accepted directly by the visual QA renderer.
- Extended the renderer with orange extreme-aspect, blue small-dimension and
green tile-edge highlighting while preserving the existing duplicate/nesting
colors and source row indices.
- Added an exact cross-tile repetition auditor that reconstructs global pixel
boxes from the exporter's `_r<row>_c<column>` offsets. Edge-clipped rows stay
explicitly unlinked rather than being guessed back to source objects.
- Made checkpoint-matrix output disclose dataset tile overlap and explicitly
state whether validation rows are independent whenever a dataset summary is
available.
### Tower findings
- Reviewed all 79,192 rows: 1,812 have a dimension below 4 px, 26 have aspect
ratio at least 8 and 5,079 touch a tile edge; 6,657 unique rows are involved.
- Inspected the complete 23-tile extreme sheet, all four small-dimension pages
covering 198 tiles, and a 64-tile edge sample. Most extreme shapes are valid
elongated structures, but 16 of 26 also fall below the 4 px floor.
- Reconstructed 34,082 unique interior objects from 74,113 interior label rows.
Tile overlap adds 40,031 exact repeated rows across 24,701 object groups,
with maximum repetition four and zero cross-split groups.
- Froze the next-corpus decision: restore `min_label_px >= 4`; test any stricter
edge-visible ratio as a separate immutable ablation rather than modifying
this corpus in place.
### Verified
- Focused auditor, renderer and checkpoint-evidence suite: 18 tests passed.
- Scoped Ruff check, format check and Git whitespace check passed.
### Remaining limitations
- Existing overlapping-tile mAP is valid for relative non-protected ranking,
not independent object-level accuracy. A future benchmark must aggregate or
deduplicate objects before uncertainty claims.
- No source label, frozen dataset, model weight or live runtime was changed.
+8
View File
@@ -1159,6 +1159,14 @@ This file now starts with the current implementation status. Older preparation/b
contact sheet; AI follow-up found no systematic duplicate-label pattern.
- [x] Bind the relationship audit back to exact label-row indices and render
all 86 implicated rows with type-specific highlighting for focused review.
- [x] Audit all 79,192 labels at row level for sub-4-pixel dimensions, extreme
aspect ratio and tile-edge clipping; inspect the complete small/extreme
sheets and retain immutable hashes.
- [x] Reconstruct exact cross-tile interior objects and prove that overlap
repeats 40,031 label rows but causes zero reconstructed train/val crossings.
- [ ] Build the next immutable experimental corpus with `min_label_px >= 4`
and compare the existing 0.35 edge-visibility policy against a separately
versioned stricter ablation before any new governed training.
- [ ] Convert the AI-assisted ledger into no stronger claim than experimental
triage; a real human must independently review and sign the frozen artifacts
before the governed training wrapper may unlock.
@@ -23,6 +23,15 @@ The checkpoint matrix used only the 36-image non-protected validation split:
measured mAP50 `0.345426` and mAP50-95 `0.140719`. The `reviewedexp6`
challenger measured `0.368177` and `0.155438` respectively.
A later exact cross-tile reconstruction established that this 512 px corpus
uses stride 256. Across the full corpus, 74,113 interior label rows represent
34,082 unique reconstructed objects; 40,031 rows are overlap repetitions and
one object can occur four times. No reconstructed object crosses the
train/validation boundary, but the tile rows are not statistically independent.
The matrix therefore remains useful only for relative non-protected candidate
ranking. Its mAP values are not an independent object-level accuracy estimate
or a basis for narrow confidence claims.
At confidence `0.25` and match IoU `0.25`, the active model measured F1
`0.633058`; the challenger measured `0.642599`. Both produced zero detections
on the 18 pure-background validation tiles at this threshold. At confidence
@@ -129,3 +138,26 @@ rewrite candidates. Excluding the 36 complete tiles would also discard 17,167
unflagged labels, so no automatic tile exclusion or label mutation was made.
The checksum-bound highlighted sheet and summary are recorded in the AI review
ledger.
## Row-level geometry and overlap follow-up
The complete 79,192-label corpus was additionally audited at label-row level.
The immutable manifest identifies 6,657 unique rows with at least one geometric
training-risk signal: 1,812 rows have a width or height below 4 pixels, 26 have
aspect ratio at least 8 and 5,079 touch a tile edge. The categories were
rendered separately. All 23 extreme-aspect tiles and all 198 small-dimension
tiles were inspected, plus a 64-tile edge sample.
The extreme-aspect group predominantly shows plausible elongated sheds and
building components. Sixteen of its 26 rows are also below 4 pixels, allowing
the resolution floor to address most ambiguous extremes without deleting valid
long structures. The small-dimension sheets contain many visually marginal
miniature targets. The next immutable experimental corpus should therefore use
the exporter's normal minimum dimension of at least 4 pixels instead of this
legacy corpus's 3-pixel override. This finding does not justify mutating the
frozen corpus or retroactively changing its checkpoint.
The edge sample shows expected clipped buildings under the 0.35 minimum-visible
policy. Raising that value may reduce partial-target pressure, but it must be a
separately versioned ablation because removing all 5,079 rows without checking
the original visible fraction would be unsound.