audit: establish accuracy phase 1 baseline

This commit is contained in:
Jens
2026-08-01 18:57:38 +02:00
parent 0c019bb22f
commit 3d442ef43f
48 changed files with 15390 additions and 1 deletions
+68
View File
@@ -12240,3 +12240,71 @@ Open:
- De productiedemo toont zeven niet-administratieve werkruimtes, drie detectiemodellen en vier segmentatiemodellen; het geconfigureerde YOLO-model wordt als beschikbaar getoond.
- Modelkalibratie, technische modelinstellingen, uploads en systeembeheer zijn niet zichtbaar voor gasten. Een directe gast-POST naar projectbeheer retourneert HTTP 403 `GUEST_READ_ONLY`; het modelregister retourneert HTTP 200 binnen dezelfde sessie.
- Visuele browseracceptatie: horizontale overflow `0`; browserconsole `0` waarschuwingen en `0` fouten.
## 2026-08-01 - Accuracy Improvement Program Phase 1 forensic baseline
### Executed scope
- Audited repository root and tracked nested mirror, API/services/schemas,
migrations, frontend, CI/readiness, container/deployment configuration,
Tower Postgres/PostGIS, direct storage references, active CUDA runtime,
mounted models/checkpoints/manifests/evaluations and Belgian building
corpus lineage.
- Created the seven required documents and machine-readable status under
`docs/accuracy-program/`.
- Added read-only collectors for repository state, Tower runtime/database,
Tower ML/data lineage and one production-adapter GPU inference, plus a
deterministic reproducer for seven critical/high contract violations.
- Retained JSON, JUnit, SQL and command logs under
`artifacts/evidence/accuracy/P1/`; no dataset, checkpoint, cache, output,
user-owned untracked file or production database row was deleted or rewritten.
### Proven findings
- Reproduced cross-theme coverage contamination, metres-as-degrees buffering,
Lambert coordinates persisted under SRID 4326, caller-spoofable official
authority, mutable-name YOLO scope, mutable-name legal coverage and silently
ignored Area PATCH geometry.
- Tower database contains four successful Geel detections with Lambert-domain
coordinates while stored as SRID 4326. Direct storage-reference audit checked
5,816 references with zero missing; the broader recursive scan timed out and
is not counted as a pass.
- The building training loop contains a protected-test feedback path. V56 has
180 AOIs but 0 human review decisions, only three pure-empty background-test
AOIs and 24 cross-split AOI pairs below 2 km. V58/V62 are calibration-only,
fail Flanders at the reported operating point and have no protected-test or
promotion evidence.
- Runtime lineage is incomplete: every persisted detection run has an empty
model version and no tile-manifest hash; three runs lack a model hash.
- Repository source of truth is ambiguous through 1,153 tracked nested mirror
files, including 68 root/mirror differences. Root `.dockerignore` correctly
excludes the mirror from the official all-in-one context.
### Runtime and verification evidence
- Real read-only inference passed through the production adapter on the Tower
RTX 4080 SUPER, PyTorch 2.11.0+cu128/CUDA 12.8, with active model SHA
`a9088b8491dfae36694b53e9e9406cb4e3511d334a5712fa34f75078a47759c1`.
One existing EPSG:31370 tile produced 17 raw detections in 0.8837 s; this is
runtime evidence only and not an accuracy result.
- Full backend suite: 1,180 passed, 17 stale source/contract assertions failed.
The actual backend CI working directory fails during collection on
`scripts.render_operator_polygon_label_qa`.
- Phase-1 tooling: 4 tests passed; all new audit files pass Ruff.
- Repository Ruff baseline: 112 findings. Frontend `test:unit`: 51/51 passed;
typecheck and build passed; the required `npm run lint` script is absent.
- OpenAPI audit passed for 147 routes plus 10 declared non-envelope endpoints.
Alembic has one head, `202607260001`, and the complete offline upgrade
rendered successfully.
- Two golden-QA runs produced equal semantic metrics but different retained
bytes because run identity uses UUID4.
### Decision
- Phase 1 is complete as a forensic and executable baseline.
- Release promotion, national validation, scope widening and immediate training
remain blocked.
- Phase 2 may start only as the dependency-ordered, test-first remediation in
`docs/accuracy-program/06-implementation-roadmap.md`: fix CRS/authority/
coverage/lineage and protected-test isolation, complete human corpus review,
rebuild independent data, freeze metrics, then train on `cuda:0`.