audit: establish accuracy phase 1 baseline

This commit is contained in:
Jens
2026-08-01 18:57:38 +02:00
parent 0c019bb22f
commit 3d442ef43f
48 changed files with 15390 additions and 1 deletions
+34
View File
@@ -1075,3 +1075,37 @@ This file now starts with the current implementation status. Older preparation/b
- [x] Sta projectgebonden analyse-, QA-, assistent- en exportacties toe.
- [x] Behoud server-side blokkades op instellingen, beheer, uploads, reviews en cross-projectverzoeken.
- [x] Verifieer en redeploy de exacte commit naar Tower `/mnt/user/appdata/geointel`.
# Accuracy Improvement Program Phase 2 (2026-08-01)
- [x] Complete the Phase-1 forensic inventory, data lineage, reproducible
baseline, risk register, metric framework, roadmap, status and retained
evidence manifest.
- [ ] P2-00: freeze evidence and enforce a promotion/scope-widening lock.
- [ ] P2-01: establish one canonical CI command, repair import shadowing,
replace 17 stale source assertions with reviewed behavior tests, close the
112 Ruff findings and add real frontend lint.
- [ ] P2-02: fix CRS ingest, metre buffering and Area geometry/CRS updates;
auditably quarantine or repair the four legacy Geel detections.
- [ ] P2-03: isolate coverage by source/theme/layer/zone, make official source
identity server-attested and replace mutable-name legal/model scope checks.
- [ ] P2-04: make derived persistence transactional, require complete
RunManifest hashes and expose every fallback/persistence failure.
- [ ] P2-05: remove every protected-test feedback path, introduce a test vault
and make the sampler reject protected IDs, paths and assessment fields.
- [ ] P2-06: complete representative human V56 review; add independent AOIs,
Brussels pure backgrounds, difficult negatives and corrected temporal/dense
PICC/UrbIS labels; freeze a new immutable corpus rather than changing V56.
- [ ] P2-07: implement the metric/null/error-taxonomy contract, stratified
incumbent baseline, calibration and reproducibility rerun; freeze numeric
gates before protected-test access.
- [ ] P2-08: train the pre-registered candidate matrix on the Tower RTX 4080
SUPER with pinned PyTorch/CUDA, full hashes, seeds and zero CPU fallback.
- [ ] P2-09: run a calibration-only improvement loop; use only independently
provisioned, reviewed train data for another corpus version.
- [ ] P2-10: open protected test/background-test exactly once for one immutable
pre-registered candidate; never retrain that candidate family from its result.
- [ ] P2-11: promote and redeploy only after every gate passes, with model card,
image/model/config hashes, shadow evidence and tested rollback.
- [ ] P2-12: monitor drift into a human reviewqueue without automatic
self-training or silent promotion.