geointel/docs/accuracy-program/00-execution-contract.md

Phase 1 execution contract

• Status: executed
• Audit date: 2026-08-01
• Audit branch: codex/geointel-accuracy-program
• Audited baseline commit: 0c019bb22f816db1e4b7a68379bcad08924d9a21
• Product scope: Belgium and the Belgian North Sea
• Release conclusion: blocked
• Phase 2 conclusion: safe to start as controlled remediation, not as training or promotion

1. Purpose

This contract turns the accuracy program into an evidence-bearing engineering program. Phase 1 establishes what exists, what actually runs, where data and model results originate, which claims are supportable, and which defects must be corrected before any new training or release promotion.

Phase 1 is complete when the required inventory, lineage, baseline, risk, metric, roadmap, machine-readable status and retained evidence exist and can be reproduced. Completion of Phase 1 does not mean that GeoIntel is accurate, nationally validated, release-ready, or fully trained.

2. Non-negotiable rules

1. No quality, coverage, authority, calibration or national-readiness claim may be stronger than the retained evidence.
2. A green unit test that uses a fixture, mock, placeholder or source-text assertion is not evidence of live geospatial or model quality.
3. Protected test data is never used for training, calibration, failure-driven sampling, threshold selection or stopping decisions.
4. Dataset, label, tile, checkpoint, runtime and evaluation identities are immutable hashes, not filenames or mutable display names.
5. Geometry entering persistence is validated, assigned an explicit source CRS, transformed to the canonical CRS and checked for valid domain and units.
6. Official or authoritative status is server-attested; caller-controlled source names can never establish authority.
7. Failed, unavailable and not_configured states remain explicit. A fallback cannot silently be reported as a successful primary result.
8. Human review is only complete when reviewer identity, decision, timestamp, sampled object/AOI identity and reviewed artifact hash are persisted.
9. Phase 1 does not delete, rewrite or promote datasets, checkpoints, caches, outputs, migrations or server state. The GPU smoke is read-only and persists no detections.
10. User-owned untracked files, including .codex-artifacts/, server .env.bak.*, .forgeflow/, storage/evaluation/ and storage/source-cache/, remain untouched.

3. Evidence vocabulary

Every statement in this program uses one of these classes:

Class	Meaning
observed	Directly read from the repository, database, storage, runtime or retained artifact
reproduced	Deterministic read-only reproducer demonstrated the behavior
tested	An executable test or build gate ran and its result is retained or recorded
inferred	A conclusion drawn from observed evidence; it must be labelled as an inference
planned	Not implemented or not yet evidenced
unavailable	The required system or artifact could not be inspected; reason is stated
blocked	A required gate cannot pass without remediation

The words “implemented”, “configured”, “available”, “validated” and “promoted” are not synonyms:

• implemented means code exists;
• configured means a runtime selects it;
• available means dependencies/assets can be loaded;
• validated requires a registered evaluation and acceptance decision;
• promoted requires every release gate and an auditable activation record.

4. Authorized Phase 1 mutations

The only authorized repository mutations are:

• audit collectors and deterministic reproducer scripts;
• focused tests for those collectors;
• docs/accuracy-program/;
• retained evidence under artifacts/evidence/accuracy/P1/;
• required execution-log and TODO updates;
• a selective .gitignore exception that tracks only the Phase 1 evidence tree while keeping other generated artifacts/ ignored.

No product behavior, model checkpoint, dataset, label or production database row is changed in Phase 1.

5. Executed investigation surface

The audit covered:

• root and nested tracked source trees, Git state and ignored/untracked state;
• backend API routes, schemas, services, ORM entities and Alembic chain;
• frontend TypeScript, tests, typecheck and production build;
• CI/readiness scripts, container definitions and Tower deployment identity;
• Postgres/PostGIS table counts, statuses, lineage gaps and geometry integrity;
• mounted storage paths referenced by the database;
• active NVIDIA/PyTorch/Ultralytics runtime and active model hash;
• one existing Geel raster tile through the production YOLO adapter on cuda:0;
• available Belgian building corpus, split, label, tiling, calibration, checkpoint and human-review evidence;
• mocks, fixtures, fallbacks, placeholders and documentation drift.

Detailed outcomes live in documents 01 through 06 and in the P1 evidence tree.

6. Reproducibility entry points

Run from the repository root:

python scripts/run_accuracy_phase1_baseline.py --output-dir artifacts/evidence/accuracy/P1
python scripts/reproduce_accuracy_phase1_findings.py
python -m pytest tests/test_accuracy_phase1_baseline.py -q -p no:cacheprovider

The runtime/database collector is intentionally executed inside the deployed application container and uses read-only SQL with a statement timeout:

scripts/collect_accuracy_phase1_runtime.py

The GPU collector is streamed into the deployed container and uses the production adapter with an existing checksummed model, manifest and tile:

scripts/collect_accuracy_phase1_inference_smoke.py

Its retained JSON includes the model, manifest and tile SHA-256 values, package versions, CUDA device, seed, configuration, raster CRS and raw output summary. It explicitly forbids deriving an accuracy or release claim from one smoke.

7. Proven release blockers

Phase 1 reproduced or observed all of the following:

• cross-theme coverage contamination can incorrectly return operational;
• a nominal metre buffer is applied as degrees;
• Lambert coordinates can be persisted unchanged under SRID 4326;
• caller-controlled source identity can be elevated to authoritative coverage;
• YOLO validation scope is bypassable through a mutable Area-name substring;
• legal coverage identity changes when an Area display name changes;
• Area PATCH silently ignores a documented geometry field;
• four persisted Geel detections contain Lambert-domain coordinates while their geometry column is labelled SRID 4326;
• the training loop contains a future protected-test feedback path;
• V56 has zero accepted human AOI reviews out of 180;
• split independence, pure-background coverage, temporal label alignment and unique-object evaluation are insufficient;
• no candidate has protected-test, background-test and promotion evidence;
• the real CI pytest work directory fails during collection, the full root suite has 17 stale assertion failures, Ruff has 112 findings and frontend lint is not configured;
• the production AI image is not built or exercised by CI;
• a tracked nested source mirror contains 1,153 files, 68 of which differ from root, creating source-of-truth ambiguity.

8. Decision gates

Gate	Phase 1 decision
Runtime CUDA and active model load	pass
One production-adapter GPU inference	pass, runtime-only evidence
Database/storage reachability	pass for inspected references
Geometry/CRS integrity	fail
Authoritative-source integrity	fail
Coverage correctness	fail
Strict protected-test isolation	fail in code design
Human label acceptance	fail
Split independence and negatives	fail
Reproducible national metric baseline	absent
Backend release gate	fail
Lint and frontend lint	fail / absent
National model promotion	blocked
Start test-first Phase 2 remediation	allowed

9. Stop and escalation conditions

Training and model promotion remain prohibited until Phase 2 has closed the CRS, authority, coverage and protected-test isolation blockers and a frozen, human-approved corpus exists. If a required source snapshot, reviewer decision, immutable hash or independent holdout cannot be produced, the corresponding claim remains blocked; it is never replaced by synthetic success.

The protected test may be opened exactly once for a pre-registered candidate after all calibration gates pass. A failure after that opening creates a new model-development cycle and requires a newly governed protected set; its errors must not feed back into the same training lineage.