audit YOLO geometry and overlapping validation rows
This commit is contained in:
@@ -12595,3 +12595,46 @@ Open:
|
||||
- This is explicit AI-assisted triage, not human adjudication. It improves the
|
||||
evidence and review precision but does not unlock governed production
|
||||
training or checkpoint promotion.
|
||||
|
||||
## 2026-08-09 - Row-level geometry and cross-tile independence audit
|
||||
|
||||
### Improved
|
||||
|
||||
- Added a reproducible, read-only label-row auditor for sub-threshold pixel
|
||||
dimensions, extreme aspect ratios and tile-edge clipping, with separate
|
||||
category manifests accepted directly by the visual QA renderer.
|
||||
- Extended the renderer with orange extreme-aspect, blue small-dimension and
|
||||
green tile-edge highlighting while preserving the existing duplicate/nesting
|
||||
colors and source row indices.
|
||||
- Added an exact cross-tile repetition auditor that reconstructs global pixel
|
||||
boxes from the exporter's `_r<row>_c<column>` offsets. Edge-clipped rows stay
|
||||
explicitly unlinked rather than being guessed back to source objects.
|
||||
- Made checkpoint-matrix output disclose dataset tile overlap and explicitly
|
||||
state whether validation rows are independent whenever a dataset summary is
|
||||
available.
|
||||
|
||||
### Tower findings
|
||||
|
||||
- Reviewed all 79,192 rows: 1,812 have a dimension below 4 px, 26 have aspect
|
||||
ratio at least 8 and 5,079 touch a tile edge; 6,657 unique rows are involved.
|
||||
- Inspected the complete 23-tile extreme sheet, all four small-dimension pages
|
||||
covering 198 tiles, and a 64-tile edge sample. Most extreme shapes are valid
|
||||
elongated structures, but 16 of 26 also fall below the 4 px floor.
|
||||
- Reconstructed 34,082 unique interior objects from 74,113 interior label rows.
|
||||
Tile overlap adds 40,031 exact repeated rows across 24,701 object groups,
|
||||
with maximum repetition four and zero cross-split groups.
|
||||
- Froze the next-corpus decision: restore `min_label_px >= 4`; test any stricter
|
||||
edge-visible ratio as a separate immutable ablation rather than modifying
|
||||
this corpus in place.
|
||||
|
||||
### Verified
|
||||
|
||||
- Focused auditor, renderer and checkpoint-evidence suite: 18 tests passed.
|
||||
- Scoped Ruff check, format check and Git whitespace check passed.
|
||||
|
||||
### Remaining limitations
|
||||
|
||||
- Existing overlapping-tile mAP is valid for relative non-protected ranking,
|
||||
not independent object-level accuracy. A future benchmark must aggregate or
|
||||
deduplicate objects before uncertainty claims.
|
||||
- No source label, frozen dataset, model weight or live runtime was changed.
|
||||
|
||||
@@ -1159,6 +1159,14 @@ This file now starts with the current implementation status. Older preparation/b
|
||||
contact sheet; AI follow-up found no systematic duplicate-label pattern.
|
||||
- [x] Bind the relationship audit back to exact label-row indices and render
|
||||
all 86 implicated rows with type-specific highlighting for focused review.
|
||||
- [x] Audit all 79,192 labels at row level for sub-4-pixel dimensions, extreme
|
||||
aspect ratio and tile-edge clipping; inspect the complete small/extreme
|
||||
sheets and retain immutable hashes.
|
||||
- [x] Reconstruct exact cross-tile interior objects and prove that overlap
|
||||
repeats 40,031 label rows but causes zero reconstructed train/val crossings.
|
||||
- [ ] Build the next immutable experimental corpus with `min_label_px >= 4`
|
||||
and compare the existing 0.35 edge-visibility policy against a separately
|
||||
versioned stricter ablation before any new governed training.
|
||||
- [ ] Convert the AI-assisted ledger into no stronger claim than experimental
|
||||
triage; a real human must independently review and sign the frozen artifacts
|
||||
before the governed training wrapper may unlock.
|
||||
|
||||
@@ -23,6 +23,15 @@ The checkpoint matrix used only the 36-image non-protected validation split:
|
||||
measured mAP50 `0.345426` and mAP50-95 `0.140719`. The `reviewedexp6`
|
||||
challenger measured `0.368177` and `0.155438` respectively.
|
||||
|
||||
A later exact cross-tile reconstruction established that this 512 px corpus
|
||||
uses stride 256. Across the full corpus, 74,113 interior label rows represent
|
||||
34,082 unique reconstructed objects; 40,031 rows are overlap repetitions and
|
||||
one object can occur four times. No reconstructed object crosses the
|
||||
train/validation boundary, but the tile rows are not statistically independent.
|
||||
The matrix therefore remains useful only for relative non-protected candidate
|
||||
ranking. Its mAP values are not an independent object-level accuracy estimate
|
||||
or a basis for narrow confidence claims.
|
||||
|
||||
At confidence `0.25` and match IoU `0.25`, the active model measured F1
|
||||
`0.633058`; the challenger measured `0.642599`. Both produced zero detections
|
||||
on the 18 pure-background validation tiles at this threshold. At confidence
|
||||
@@ -129,3 +138,26 @@ rewrite candidates. Excluding the 36 complete tiles would also discard 17,167
|
||||
unflagged labels, so no automatic tile exclusion or label mutation was made.
|
||||
The checksum-bound highlighted sheet and summary are recorded in the AI review
|
||||
ledger.
|
||||
|
||||
## Row-level geometry and overlap follow-up
|
||||
|
||||
The complete 79,192-label corpus was additionally audited at label-row level.
|
||||
The immutable manifest identifies 6,657 unique rows with at least one geometric
|
||||
training-risk signal: 1,812 rows have a width or height below 4 pixels, 26 have
|
||||
aspect ratio at least 8 and 5,079 touch a tile edge. The categories were
|
||||
rendered separately. All 23 extreme-aspect tiles and all 198 small-dimension
|
||||
tiles were inspected, plus a 64-tile edge sample.
|
||||
|
||||
The extreme-aspect group predominantly shows plausible elongated sheds and
|
||||
building components. Sixteen of its 26 rows are also below 4 pixels, allowing
|
||||
the resolution floor to address most ambiguous extremes without deleting valid
|
||||
long structures. The small-dimension sheets contain many visually marginal
|
||||
miniature targets. The next immutable experimental corpus should therefore use
|
||||
the exporter's normal minimum dimension of at least 4 pixels instead of this
|
||||
legacy corpus's 3-pixel override. This finding does not justify mutating the
|
||||
frozen corpus or retroactively changing its checkpoint.
|
||||
|
||||
The edge sample shows expected clipped buildings under the 0.35 minimum-visible
|
||||
policy. Raising that value may reduce partial-target pressure, but it must be a
|
||||
separately versioned ablation because removing all 5,079 rows without checking
|
||||
the original visible fraction would be unsound.
|
||||
|
||||
Reference in New Issue
Block a user