78 lines
3.2 KiB
Markdown
78 lines
3.2 KiB
Markdown
# Reviewed accuracy challenger - 2026-07-15
|
|
|
|
## Scope
|
|
|
|
This pass closes the evidence requirement from the preceding FP/FN review. It
|
|
adds complete, training-only orthophoto/GRB AOIs outside all protected
|
|
operational holdouts, trains one inactive local challenger and compares both
|
|
models through the current persisted GeoIntel workflow. No model was
|
|
downloaded and the active runtime configuration was not changed.
|
|
|
|
- Active model: `geointel-building-yolov8s-smallbld-minpx3-img640-ft30-pt`
|
|
- Active SHA256: `a9088b8491dfae36694b53e9e9406cb4e3511d334a5712fa34f75078a47759c1`
|
|
- Challenger: `geointel-building-yolov8s-reviewedexp6-minpx3-img640-ft20-pt`
|
|
- Challenger SHA256: `038f1f97a6afd534f29e1f392a730a58207b928ca01e31ab8d8fed6106705820`
|
|
- Fixed profile: tile `512`, overlap `64`, confidence `0.15`, QA IoU `0.25`
|
|
|
|
## Data and training
|
|
|
|
Six new training-only centers were added in Arendonk, Dessel, Meerhout,
|
|
Laakdal, Nijlen and Hulshout. Every center is in its documented municipality,
|
|
uses a complete paged GRB GBG export and remains at least 2 km from every
|
|
protected positive holdout.
|
|
|
|
- New GRB references: 9,964 features
|
|
- Corpus: 252 tiles, 234 positive and 18 negative
|
|
- Split: 234 training and 18 validation tiles
|
|
- Labels: 79,192
|
|
- Dataset audit: `ok`, zero invalid or missing labels
|
|
- Visual audit: 64 tiles across 28 retained sources, zero invalid or
|
|
low-variance selections
|
|
- Fine-tuning: 20 CPU epochs from the active local model, image size `640`
|
|
- Best checkpoint: epoch 16, validation precision `0.700`, recall `0.406`,
|
|
mAP50 `0.369`, mAP50-95 `0.159`
|
|
|
|
## Coverage-aligned result
|
|
|
|
Both assets were rerun after the current coverage-aware QA logic was deployed.
|
|
Each zone comparison uses the same evaluated reference population; older runs
|
|
with different edge coverage are not used for the decision.
|
|
|
|
| Zone | Active F1 | Challenger F1 | Delta |
|
|
| --- | ---: | ---: | ---: |
|
|
| Mol Achterbos | 0.6694 | 0.6850 | +0.0156 |
|
|
| Mol Donk | 0.5894 | 0.6259 | +0.0365 |
|
|
| Mol Gompel | 0.6564 | 0.6694 | +0.0131 |
|
|
| Mol Postel | 0.4749 | 0.4753 | +0.0005 |
|
|
| Retie | 0.6325 | 0.6491 | +0.0166 |
|
|
| Turnhout | 0.5919 | 0.5988 | +0.0069 |
|
|
| Westerlo | 0.6336 | 0.6700 | +0.0364 |
|
|
|
|
| Aggregate | Active | Challenger |
|
|
| --- | ---: | ---: |
|
|
| Mean precision | 0.6141 | 0.6251 |
|
|
| Mean recall | 0.6062 | 0.6293 |
|
|
| Mean F1 | 0.6069 | 0.6248 |
|
|
| Matches | 5,711 | 5,919 |
|
|
| False positives | 3,568 | 3,506 |
|
|
| False negatives | 3,416 | 3,208 |
|
|
|
|
## Promotion decision
|
|
|
|
| Pure-empty control | Active detections | Challenger detections |
|
|
| --- | ---: | ---: |
|
|
| Arendonk-heide | 0 | 0 |
|
|
| Lommel-heide | 0 | 0 |
|
|
| Postel-bos | 0 | 2 |
|
|
|
|
Decision: **retain the active model**. The challenger improves every positive
|
|
zone but violates the zero-detection Postel-bos gate. The formal promotion
|
|
report therefore recommends the existing active key
|
|
`geointel-building-yolov8s-smallbld-minpx3-img640-ft30-pt|512|64|0.15`.
|
|
Positive-score gains do not override a failed fail-closed background control.
|
|
|
|
Persistent evidence is stored below
|
|
`/app/storage/operator-data/model-review/reviewed-accuracy-expansion/` and
|
|
`/app/storage/operator-evidence/mol-operational-validation/`. Generated rasters,
|
|
weights and large evidence JSON files remain outside Git.
|