audit: establish accuracy phase 1 baseline
This commit is contained in:
@@ -12240,3 +12240,71 @@ Open:
|
||||
- De productiedemo toont zeven niet-administratieve werkruimtes, drie detectiemodellen en vier segmentatiemodellen; het geconfigureerde YOLO-model wordt als beschikbaar getoond.
|
||||
- Modelkalibratie, technische modelinstellingen, uploads en systeembeheer zijn niet zichtbaar voor gasten. Een directe gast-POST naar projectbeheer retourneert HTTP 403 `GUEST_READ_ONLY`; het modelregister retourneert HTTP 200 binnen dezelfde sessie.
|
||||
- Visuele browseracceptatie: horizontale overflow `0`; browserconsole `0` waarschuwingen en `0` fouten.
|
||||
|
||||
## 2026-08-01 - Accuracy Improvement Program Phase 1 forensic baseline
|
||||
|
||||
### Executed scope
|
||||
|
||||
- Audited repository root and tracked nested mirror, API/services/schemas,
|
||||
migrations, frontend, CI/readiness, container/deployment configuration,
|
||||
Tower Postgres/PostGIS, direct storage references, active CUDA runtime,
|
||||
mounted models/checkpoints/manifests/evaluations and Belgian building
|
||||
corpus lineage.
|
||||
- Created the seven required documents and machine-readable status under
|
||||
`docs/accuracy-program/`.
|
||||
- Added read-only collectors for repository state, Tower runtime/database,
|
||||
Tower ML/data lineage and one production-adapter GPU inference, plus a
|
||||
deterministic reproducer for seven critical/high contract violations.
|
||||
- Retained JSON, JUnit, SQL and command logs under
|
||||
`artifacts/evidence/accuracy/P1/`; no dataset, checkpoint, cache, output,
|
||||
user-owned untracked file or production database row was deleted or rewritten.
|
||||
|
||||
### Proven findings
|
||||
|
||||
- Reproduced cross-theme coverage contamination, metres-as-degrees buffering,
|
||||
Lambert coordinates persisted under SRID 4326, caller-spoofable official
|
||||
authority, mutable-name YOLO scope, mutable-name legal coverage and silently
|
||||
ignored Area PATCH geometry.
|
||||
- Tower database contains four successful Geel detections with Lambert-domain
|
||||
coordinates while stored as SRID 4326. Direct storage-reference audit checked
|
||||
5,816 references with zero missing; the broader recursive scan timed out and
|
||||
is not counted as a pass.
|
||||
- The building training loop contains a protected-test feedback path. V56 has
|
||||
180 AOIs but 0 human review decisions, only three pure-empty background-test
|
||||
AOIs and 24 cross-split AOI pairs below 2 km. V58/V62 are calibration-only,
|
||||
fail Flanders at the reported operating point and have no protected-test or
|
||||
promotion evidence.
|
||||
- Runtime lineage is incomplete: every persisted detection run has an empty
|
||||
model version and no tile-manifest hash; three runs lack a model hash.
|
||||
- Repository source of truth is ambiguous through 1,153 tracked nested mirror
|
||||
files, including 68 root/mirror differences. Root `.dockerignore` correctly
|
||||
excludes the mirror from the official all-in-one context.
|
||||
|
||||
### Runtime and verification evidence
|
||||
|
||||
- Real read-only inference passed through the production adapter on the Tower
|
||||
RTX 4080 SUPER, PyTorch 2.11.0+cu128/CUDA 12.8, with active model SHA
|
||||
`a9088b8491dfae36694b53e9e9406cb4e3511d334a5712fa34f75078a47759c1`.
|
||||
One existing EPSG:31370 tile produced 17 raw detections in 0.8837 s; this is
|
||||
runtime evidence only and not an accuracy result.
|
||||
- Full backend suite: 1,180 passed, 17 stale source/contract assertions failed.
|
||||
The actual backend CI working directory fails during collection on
|
||||
`scripts.render_operator_polygon_label_qa`.
|
||||
- Phase-1 tooling: 4 tests passed; all new audit files pass Ruff.
|
||||
- Repository Ruff baseline: 112 findings. Frontend `test:unit`: 51/51 passed;
|
||||
typecheck and build passed; the required `npm run lint` script is absent.
|
||||
- OpenAPI audit passed for 147 routes plus 10 declared non-envelope endpoints.
|
||||
Alembic has one head, `202607260001`, and the complete offline upgrade
|
||||
rendered successfully.
|
||||
- Two golden-QA runs produced equal semantic metrics but different retained
|
||||
bytes because run identity uses UUID4.
|
||||
|
||||
### Decision
|
||||
|
||||
- Phase 1 is complete as a forensic and executable baseline.
|
||||
- Release promotion, national validation, scope widening and immediate training
|
||||
remain blocked.
|
||||
- Phase 2 may start only as the dependency-ordered, test-first remediation in
|
||||
`docs/accuracy-program/06-implementation-roadmap.md`: fix CRS/authority/
|
||||
coverage/lineage and protected-test isolation, complete human corpus review,
|
||||
rebuild independent data, freeze metrics, then train on `cuda:0`.
|
||||
|
||||
Reference in New Issue
Block a user