evaluate models on fresh regional calibration AOIs
GeoIntel release gates / Compile, test, contracts and builds (push) Canceled after 0s
GeoIntel release gates / Python and npm vulnerability policy (push) Canceled after 0s
GeoIntel release gates / GIS image, SBOM and container scan (push) Canceled after 0s

This commit is contained in:
Jens
2026-08-09 22:36:27 +02:00
parent 116b8e291e
commit b068a5e065
28 changed files with 5053 additions and 18 deletions
@@ -263,3 +263,41 @@ The byte-matching parent/output chain is retained separately in
`artifacts/evidence/accuracy/model-training/20260809-v71-model-lineage-receipt.json`.
Fresh geographically separated AOIs with complete spatial and lineage checks
are required for the next meaningful evaluation.
## Fresh V72 calibration portfolio
Six AOIs were pre-registered before inference and acquired again from the
governed regional orthophoto and building adapters: two each in Vlaanderen,
Wallonië and Brussel. All six are absent from every split of the retained model
lineage. A raster-bound spatial audit resolved all 31 ancestral AOIs and found
the nearest new AOI at 4,869.63 metres, above the frozen 2 km floor.
The initial generic edge-cover tiling would have counted 54 tiles because its
final-row/final-column coverage nearly duplicated the preceding 512 px cells.
That view was rejected. The corrected evaluation-only exporter retains exactly
24 full grid cells, excludes 30 overlapping edge-cover views with reason codes,
and writes no training tiles. Both the corpus and tile view contain bound
`NO_TRAINING.json` markers.
All 24 tiles and 348 retained labels were visually inspected. Havelange is a
positive village context despite its pre-registered `field-cal-bg` name.
Assenede and Laeken contain sparse official building labels and are difficult
contexts, not pure-empty backgrounds. The evaluator now refuses to accept a
pure-background prefix when any corresponding label file is non-empty.
On the RTX 4080 SUPER the active model scores mAP50 `0.056779` and mAP50-95
`0.021750`; the challenger scores `0.044705` and `0.016004`. At confidence 0.25
and match IoU 0.25 their F1 scores are `0.231527` and `0.235885`. These low
values expose substantial domain-generalisation failure. The active model stays
in production because the challenger has weaker mAP, only a marginal F1 gain,
missing historical release provenance and no valid promotion bundle.
The active model's aggregate calibration F1 peaks at threshold 0.30
(`0.250356`), but its Flemish subgroup becomes worse than at lower thresholds.
No threshold was changed because a global average may not mask that subgroup
regression. PICC and UrbIS remain diagnostic-only while their regional semantic
harmonisation contracts are pending; no national release claim is made.
Complete evidence is retained under
`artifacts/evidence/accuracy/model-training/20260809-v72-fresh-calibration/`
and summarized by `20260809-v72-fresh-calibration-summary.json`.