evaluate models on fresh regional calibration AOIs
This commit is contained in:
@@ -263,3 +263,41 @@ The byte-matching parent/output chain is retained separately in
|
||||
`artifacts/evidence/accuracy/model-training/20260809-v71-model-lineage-receipt.json`.
|
||||
Fresh geographically separated AOIs with complete spatial and lineage checks
|
||||
are required for the next meaningful evaluation.
|
||||
|
||||
## Fresh V72 calibration portfolio
|
||||
|
||||
Six AOIs were pre-registered before inference and acquired again from the
|
||||
governed regional orthophoto and building adapters: two each in Vlaanderen,
|
||||
Wallonië and Brussel. All six are absent from every split of the retained model
|
||||
lineage. A raster-bound spatial audit resolved all 31 ancestral AOIs and found
|
||||
the nearest new AOI at 4,869.63 metres, above the frozen 2 km floor.
|
||||
|
||||
The initial generic edge-cover tiling would have counted 54 tiles because its
|
||||
final-row/final-column coverage nearly duplicated the preceding 512 px cells.
|
||||
That view was rejected. The corrected evaluation-only exporter retains exactly
|
||||
24 full grid cells, excludes 30 overlapping edge-cover views with reason codes,
|
||||
and writes no training tiles. Both the corpus and tile view contain bound
|
||||
`NO_TRAINING.json` markers.
|
||||
|
||||
All 24 tiles and 348 retained labels were visually inspected. Havelange is a
|
||||
positive village context despite its pre-registered `field-cal-bg` name.
|
||||
Assenede and Laeken contain sparse official building labels and are difficult
|
||||
contexts, not pure-empty backgrounds. The evaluator now refuses to accept a
|
||||
pure-background prefix when any corresponding label file is non-empty.
|
||||
|
||||
On the RTX 4080 SUPER the active model scores mAP50 `0.056779` and mAP50-95
|
||||
`0.021750`; the challenger scores `0.044705` and `0.016004`. At confidence 0.25
|
||||
and match IoU 0.25 their F1 scores are `0.231527` and `0.235885`. These low
|
||||
values expose substantial domain-generalisation failure. The active model stays
|
||||
in production because the challenger has weaker mAP, only a marginal F1 gain,
|
||||
missing historical release provenance and no valid promotion bundle.
|
||||
|
||||
The active model's aggregate calibration F1 peaks at threshold 0.30
|
||||
(`0.250356`), but its Flemish subgroup becomes worse than at lower thresholds.
|
||||
No threshold was changed because a global average may not mask that subgroup
|
||||
regression. PICC and UrbIS remain diagnostic-only while their regional semantic
|
||||
harmonisation contracts are pending; no national release claim is made.
|
||||
|
||||
Complete evidence is retained under
|
||||
`artifacts/evidence/accuracy/model-training/20260809-v72-fresh-calibration/`
|
||||
and summarized by `20260809-v72-fresh-calibration-summary.json`.
|
||||
|
||||
Reference in New Issue
Block a user