geointel/docs/RC_ROADMAP_BELGIUM_NORTH_SEA.md

Autonomous RC Roadmap: Belgium and the Belgian North Sea

Mission

Turn the current GeoIntel implementation into a reproducible, demonstrable and release-candidate-quality map-first workbench for all of Belgium and the Belgian North Sea.

This file is an executable runbook for Codex. It is both the implementation order and the release evidence index. Work continues autonomously between phases. A separate RC-12 phase is deliberately omitted.

Sources of truth

Read in this order before changing behavior:

1. AGENTS.md
2. docs/CODEX_BOOTSTRAP_PROMPT.md
3. docs/RC_SCOPE_FREEZE_BELGIUM_NORTH_SEA.md
4. docs/governance/GEOINTEL_CONSTITUTION.md
5. docs/governance/ARCHITECTURE_INVARIANTS.md
6. docs/governance/FORBIDDEN_DECISIONS.md
7. docs/API_CONTRACTS.md
8. docs/DATABASE_IMPLEMENTATION_PLAN.md
9. docs/DATA_SPECIFICATION.md
10. docs/DATA_SOURCES.md
11. docs/STORAGE_ARCHITECTURE.md
12. docs/DEFINITION_OF_DONE.md

Autonomous execution contract

Codex shall:

• execute phases in order and continue without requesting routine approval;
• use existing service, dataset, vector-feature, raster and job boundaries;
• keep every provider bounded, fail-closed and provenance-complete;
• add or update tests with every behavior change;
• update docs/CODEX_EXECUTION_LOG.md, docs/TODO.md and CHANGELOG.md after each completed phase;
• run the phase gate and record exact output under storage/release-evidence/<release-id>/;
• commit and push only after the relevant local gate is green;
• deploy only a commit that passed the local gate;
• run browser and live API acceptance after deployment;
• retain Mol/Kempen as regression references while adding Wallonia, Brussels, cross-region, coast and North Sea golden areas.

Codex stops only when:

• an operation would destructively mutate or delete unbacked production data;
• an external licence, credential or authority decision cannot be derived from public source metadata;
• an official endpoint fails security validation and no authoritative alternative exists;
• the same mandatory gate fails three times for the same external reason; or
• continuing would require fabricating data, metrics, provenance or model output.

For a stop, write the blocker, evidence and exact resume command to the execution log. Continue with independent work when possible.

Release priorities

P0 - release blockers

• truthful health/readiness and capability reporting;
• current backup plus isolated restore proof;
• stale job/run reconciliation and useful logs;
• temporally correct detection QA;
• production secrets, upload limits, configuration parity and rollback;
• full CI and reproducible dependencies;
• fresh install and upgrade proof.

P1 - national product completeness

• national and maritime scope model;
• governed source coverage matrix;
• Wallonia, Brussels and North Sea adapters;
• nationwide bounded selection and mixed-zone result handling;
• historical compatibility matrix;
• golden areas outside Flanders.

P2 - quality and maintainability

• typed response models on critical routes;
• frontend unit and browser E2E coverage;
• loading/accessibility/performance hardening;
• retention and operator documentation;
• remaining bounded modularization where protected by tests.

Global gates

Run after every behavior-changing phase:

python -m compileall backend/app
cd backend && python -m pytest
cd frontend && npm run typecheck
cd frontend && npm run build
bash scripts/run_readiness_check.sh
cd backend && python -m alembic heads
cd backend && python -m alembic upgrade head --sql
bash -n scripts/live_migration_smoke.sh
docker compose config

When Tower is reachable:

.\scripts\deploy_tower.ps1

Then verify:

curl -fsS http://192.168.10.150:1202/health/live
curl -fsS http://192.168.10.150:1202/health/ready
curl -fsS http://192.168.10.150:1202/api/v1/system/capabilities

The in-app browser acceptance must cover desktop, widescreen and a narrow viewport. It must inspect console errors, failed requests and the complete map-select-analyse-export flow.

Official source baseline

The roadmap starts from these verified official families. Machine endpoints, editions and licences must still pass source-specific probes before activation.

Scope	Candidate	Role
Belgium	NGI/IGN AdminVector and Top10Vector	authoritative boundary and common topographic baseline
Belgium	Statbel statistical sectors/population	authoritative statistical geometry and time series
Flanders	existing Digitaal Vlaanderen/VMM/DOV/INBO/ALZ adapters	detailed authoritative regional layers
Wallonia	SPW Géoportail, PICC, orthophoto, MNT and thematic services	detailed authoritative regional layers
Brussels	Paradigm UrbIS and regional environmental geodata	detailed authoritative urban layers
North Sea	FPS Marine Environment marine spatial plan	legal/use zones
North Sea	RBINS/BMDC and Marine Atlas	marine environment and observations
North Sea	MDK/Flemish Hydrography	bathymetry and nautical source candidate
North Sea	EMODnet	contextual fallback only when explicitly labelled

Phase status

Phase	State	Purpose
RC-0	complete	freeze scope and produce release evidence baseline
RC-1	complete	backup, restore and data safety
RC-2	complete	health, capabilities and stale-runtime correctness
RC-3	complete	temporal detection/QA correctness and observability
RC-4	complete	national/maritime scope and provider coverage contracts
RC-5	complete	deployment, secrets, configuration, fresh install and rollback
RC-6	complete	complete CI, dependency and supply-chain gates
RC-7	complete	critical API envelope typing and contract validation
RC-8	complete	frontend and browser E2E release journeys
RC-9	in progress	loading, accessibility and performance hardening
RC-10	pending	retention, cleanup and national data operations
RC-11	pending	final package, upgrade proof, release tag and handoff

RC-0 - Scope freeze and evidence baseline

State: complete. The read-only baseline command and focused tests pass. Runtime evidence is stored outside Git under storage/release-evidence/.

Work

• Freeze Belgium plus Belgian North Sea scope.
• Record current commit, migration head, dependency versions, container image, database/storage sizes and readiness results.
• Create a release-evidence manifest command.
• Replace stale product-status wording that still treats Kempen as the product boundary.
• Add a coverage matrix schema with operational, partial, not_configured, unsupported.

Likely files

• docs/RC_SCOPE_FREEZE_BELGIUM_NORTH_SEA.md
• docs/RC_ROADMAP_BELGIUM_NORTH_SEA.md
• docs/BUILD_STATUS.md
• docs/DATA_SOURCES.md
• scripts/capture_release_evidence.py
• backend/tests/test_rc_release_evidence.py

Evidence and exit

• scope documents contain Belgium, all three regions and maritime zones;
• evidence command is read-only and deterministic;
• manifest contains commit, dirty state, Alembic head and gate status;
• compile and focused tests pass.

RC-1 - Backup, restore and data safety

State: complete. Tower retains the checksum-verified 1.4 GiB backup rc-belgium-north-sea-fc42ea9-secure. Its manifest records a non-default database password. An isolated restore reached PostGIS 3.6 and Alembic 202607160001, reconciled every retained critical table count and removed the temporary database.

Work

• Add a safe production backup command for PostgreSQL custom-format dumps.
• Capture schema head, PostGIS version, table counts, model checksums and a storage checksum inventory without duplicating all large artifacts.
• Reject default/empty production database passwords.
• Add backup verification with pg_restore --list and checksums.
• Add an isolated restore drill using a temporary database/container; never restore over production.
• Document retention, encryption boundary and cleanup.

Likely files

• scripts/backup_release_state.sh
• scripts/verify_release_backup.sh
• scripts/restore_release_backup_smoke.sh
• deploy/unraid/README.md
• docs/ROLLBACK_AND_RECOVERY.md
• backend/tests/test_rc_backup_restore_scripts.py

Exit

• current Tower database has a dated, checksum-verified backup;
• isolated restore reaches the recorded Alembic head;
• representative project, dataset, geometry and QA counts reconcile;
• no production application data was mutated by the drill.

RC-2 - Truthful runtime state

State: complete. Local compile, 963 backend tests, frontend typecheck/build and the full readiness gate pass. Tower reports green DB/PostGIS/migration/storage readiness and configured YOLO capability. Startup reconciled five orphaned jobs and two orphaned analysis runs; zero remain running.

Work

• Split process liveness from dependency readiness.
• Make readiness return HTTP 503 when DB, PostGIS, migration head or writable storage is unavailable.
• Derive version/build identity and AI capability from runtime configuration.
• Make Docker use the readiness endpoint.
• Reconcile orphaned running jobs and analysis runs on process startup.
• Add request IDs and exception logging without exposing secrets or full SQL.

Likely files

• backend/app/api/routes/health.py
• backend/app/schemas/health.py
• backend/app/core/config.py
• backend/app/core/logging.py
• backend/app/main.py
• backend/app/services/runtime_reconciliation_service.py
• Docker/nginx files and health tests

Exit

• liveness remains 200 while the process runs;
• readiness is fail-closed;
• live capabilities match the active local YOLO state;
• restart converts impossible orphaned work to a terminal failed state with PROCESS_INTERRUPTED;
• Docker reports unhealthy for a broken database.

RC-3 - Temporal QA correctness and observability

State: complete. The local release gate passes backend compilation, all 974 backend tests, frontend typecheck/build, the integrated readiness script, one Alembic head and complete offline migration SQL generation. Tower runs immutable build a28f2497e8d0da7d108828875ba9ffedda3c1688, reports PostgreSQL, PostGIS 3.6, migration 202607160001, storage and configured YOLO as ready, and contains no orphaned running Job or AnalysisRun. Live browser acceptance proved that Detection Lab starts with an empty raster choice, disables its primary action and reports the required operator choice without console errors.

Work

• Stop Detection Lab from silently selecting the first raster.
• Require explicit raster/model choice.
• Prevent current reference QA against historical imagery unless a compatible reference edition/coverage is selected.
• Persist source observation period and QA compatibility decision.
• Add structured request/job/run logs and operator diagnostics.
• Add a stale-runtime report and safe reconciliation command.

Exit

• no detection run starts with an implicit old raster;
• incompatible temporal QA fails with a clear code;
• compatible current orthophoto/GRB path remains green;
• logs correlate request, job, analysis run and quality check.

RC-4 - Belgium and North Sea coverage foundation

State: complete. Tower persists one idempotent national workspace with eight distinct land/maritime areas and six ready authoritative datasets containing 685 features. The coverage catalog and resolver report operational, partial, not-configured and unsupported states honestly for Belgium, all three regions and the Belgian maritime zones. Live browser acceptance proved the national default context, coverage-only selection and responsive 390/1920 px layouts while unavailable detailed regional themes remain explicit.

Work

1. Persist NGI authoritative Belgium land and administrative scopes.
2. Persist territorial sea, EEZ and continental shelf as distinct scope layers.
3. Implement the coverage matrix and source authority resolver.
4. Add bounded provider contracts for:
   • NGI/Statbel national baseline;
   • SPW/PICC/orthophoto/relief for Wallonia;
   • UrbIS/environmental context for Brussels;
   • federal marine zones and BMDC/MDK marine sources.
5. Normalize the public theme vocabulary without discarding provider-native semantics.
6. Add Wallonia, Brussels, cross-region, coast and North Sea golden areas.
7. Make mixed-zone selection split into bounded provider requests and merge only semantically compatible metrics.

Required rules

• all persistence flows through DatasetService and geospatial services;
• no provider writes directly to feature tables;
• no country-wide startup import;
• no TLS bypass;
• no implicit vertical datum conversion;
• unsupported metrics remain unavailable.

Likely files

• provider registry/base and new regional provider modules;
• geographic scope and source coverage services;
• dataset/source schemas;
• map source portfolio and scope UI;
• source, API, data and storage documentation;
• source-specific tests and fixtures.

Exit

• a user can draw within every golden area and receive an honest coverage matrix;
• at least the common national baseline is operational on land;
• regional detailed layers are operational or explicitly partial;
• maritime legal zones are operational;
• bathymetry remains gated until a source passes strict acquisition evidence.

RC-5 - Production deployment and rollback

State: complete. Tower runs a commit-plus-AI-profile immutable image with OCI revision/build labels. Repeated deploys reuse the exact image and preserve the actual prior release. A fresh install passed on isolated volumes, manual rollback passed against retained production mounts, and the release image upgraded a checksum-verified 1.4 GiB backup in a generated temporary database at PostGIS 3.6/Alembic 202607160001 before removing it. Production startup is fail-closed for default secrets and invalid upload limits, nginx/backend limits are aligned, and all operator-owned settings are editable in Unraid.

Work

• align backend, nginx and proxy upload/time limits;
• make every runtime setting intentionally configured, internal or documented;
• reject production default secrets;
• add image labels with version, commit and build time;
• deploy immutable tags in addition to latest;
• preserve previous image for one-command rollback;
• prove fresh install against empty volumes;
• prove upgrade from a copied current database/storage state;
• prove rollback without schema or data loss.

Exit

• Unraid edit fields cover all operator-owned settings;
• fresh install, upgrade and rollback commands are documented and executed;
• deployed UI exposes build identity;
• previous image remains addressable;
• no mutable-only release is accepted.

RC-6 - CI and supply-chain gates

State: complete. Equivalent Gitea and GitHub workflows now enforce the complete readiness gate, offline migrations, Compose resolution, dependency audits, immutable container build, SPDX SBOM and Trivy policy. Linux/Python 3.11 runtime and CI locks are hash-pinned and optional AI dependencies remain separate. The live AI image passed the container policy with zero reachable fixed HIGH/CRITICAL findings; its complete report and 25.7 MB SBOM are retained on Tower. The report deliberately preserves two timeboxed Starlette findings and fifteen shadowed base-layer gosu findings. Runtime evidence proves that the final gosu path is the audited setpriv wrapper running as PostgreSQL UID 999.

Work

• make CI run backend compile/tests, frontend typecheck/build, readiness, Alembic single-head/offline SQL and Docker config/build;
• add Python and npm dependency vulnerability checks;
• add container scan and SBOM;
• introduce reproducible Python resolution/lock output;
• preserve optional AI dependency separation;
• publish gate artefacts.

Exit

• a broken compile, test, build, migration or contract blocks CI;
• dependency and container findings have severity policy;
• a clean checkout can reproduce dependency versions and image metadata.

RC-7 - Critical API contract hardening

State: complete. All successful JSON routes now publish concrete OpenAPI response schemas. The contract audit covers 124 implemented routes and 228 component schemas; the eight non-envelope operations are the three health probes, four persisted PNG responses and the export download. Readiness passed with 1,008 backend tests, frontend typecheck/build, one Alembic head and offline migration SQL generation.

Work

• replace response_model=dict first on health, projects, areas, datasets, jobs, analysis runs, QA, exports and map-critical endpoints;
• retain canonical {data: ...} success and canonical error envelopes;
• add OpenAPI response validation tests;
• document intentionally streaming/download responses separately.

Exit

• all critical map-first routes have concrete response models;
• actual payloads pass FastAPI validation;
• API contract audit and frontend typecheck pass.

RC-8 - Release journey automation

State: complete. Four Vitest suites protect selection, coverage, temporal comparison and bootstrap behavior with 12 tests. The Playwright release runner provisions seven deterministic land, cross-region, coastal and maritime golden areas in the national workbench through canonical APIs and executes the complete browser/API journey against the live PostGIS runtime. Evidence includes governed Mol metrics and provenance, compatible forest history, no-data, partial coverage, unsupported maritime metrics, simulated provider failure, persistent export, real local Ollama context and an explicit configured-YOLO run. The green run contains no unexpected browser-console or failed-request events.

Work

• add frontend unit tests for selection, coverage, loading and temporal guards;
• add browser E2E for:
  • open Belgium map;
  • select each golden area;
  • load a theme;
  • inspect metric/provenance;
  • compare compatible history;
  • run a bounded configured detection path;
  • export;
  • ask Ollama with persisted context;
• include no-data, partial coverage, provider failure and unsupported metric journeys.

Exit

• E2E runs against a real backend/PostGIS fixture;
• no workflow depends only on source-text assertion tests;
• console and failed-request audits are clean.

Reproduction

npm --prefix frontend run test:unit
bash scripts/run_rc8_release_journeys.sh \
  http://192.168.10.150:1202 \
  artifacts/rc8-release-journeys \
  artifacts/rc8-golden-areas.json

The second command is intentionally live and mutating. It creates only missing golden Areas, copies the official Mol/Kempen geometry into the national workbench with source identifiers, uses the existing explicit demo-fixture flow for AI validation and writes screenshots plus manifest.json under the ignored artifacts/ directory.

RC-9 - UX, accessibility and performance

State: complete. The live immutable AI image passed the RC-9 browser audit at 390x844, 1366x768 and 2560x1080 without horizontal overflow, browser-console errors or failed requests. All seven workspaces expose accessible control names, bootstrap loading remains truthful under delayed API responses and coverage/map timing budgets have visible operator feedback. Repository readiness passed with 1,015 backend tests and 16 frontend tests.

Work

• replace initial false missing source states with explicit loading;
• provide accessible names, labels, keyboard focus and status announcements;
• enforce desktop, widescreen and narrow layouts;
• record API and map-selection budgets for golden areas;
• virtualize or page large panels;
• split large modules only where tests protect behavior.

Exit

• no core control lacks an accessible name;
• loading, empty, partial, error and ready states are distinct;
• map selection remains usable at target viewports;
• no release journey exceeds its documented budget without a visible warning.

RC-10 - Data operations and retention

State: in progress.

Work

• define lifecycle for raw, normalized, derived, export, AI and evidence artifacts;
• add dry-run-first cleanup for stale technical projects, failed work and superseded caches;
• retain immutable official source editions and release evidence;
• report disk pressure before acquisition;
• schedule nothing implicitly; provide explicit operator commands;
• add source freshness reports for national/regional/maritime families.

Exit

• cleanup cannot remove release evidence or current authoritative editions;
• every destructive action requires explicit confirmation and a current backup;
• storage growth and retained provenance are reportable.

RC-11 - Final release package

Work

• rerun fresh-install and upgrade proof using the release image;
• rerun backup/isolated restore and rollback proof;
• execute every golden area and core browser journey;
• generate coverage, security, dependency, migration, performance and known limitation reports;
• update operator/end-user documentation;
• remove stale status documents from active navigation without deleting historical evidence;
• assign semantic version, create signed/checksummed release manifest, tag, push and deploy the immutable image.

Final acceptance

• all P0 findings are closed;
• every mandatory global gate is green;
• backup and isolated restore are proven;
• one Alembic head applies to empty and upgraded PostGIS;
• Belgium and maritime golden areas pass;
• no unsupported metric or unavailable source appears as successful;
• live browser acceptance passes at port 1202;
• remaining P1/P2 limitations are explicit, bounded and non-deceptive.

Current autonomous phase

Continue with RC-10. Data inspection and cleanup planning must remain read-only until the backup, confirmation and protected-path requirements in RC-10 are proven.