geointel/docs/RC_ROADMAP_BELGIUM_NORTH_SEA.md

# Autonomous RC Roadmap: Belgium and the Belgian North Sea

## Mission

Turn the current GeoIntel implementation into a reproducible, demonstrable and
release-candidate-quality map-first workbench for all of Belgium and the
Belgian North Sea.

This file is an executable runbook for Codex. It is both the implementation
order and the release evidence index. Work continues autonomously between
phases. A separate RC-12 phase is deliberately omitted.

## Sources of truth

Read in this order before changing behavior:

1. `AGENTS.md`
2. `docs/CODEX_BOOTSTRAP_PROMPT.md`
3. `docs/RC_SCOPE_FREEZE_BELGIUM_NORTH_SEA.md`
4. `docs/governance/GEOINTEL_CONSTITUTION.md`
5. `docs/governance/ARCHITECTURE_INVARIANTS.md`
6. `docs/governance/FORBIDDEN_DECISIONS.md`
7. `docs/API_CONTRACTS.md`
8. `docs/DATABASE_IMPLEMENTATION_PLAN.md`
9. `docs/DATA_SPECIFICATION.md`
10. `docs/DATA_SOURCES.md`
11. `docs/STORAGE_ARCHITECTURE.md`
12. `docs/DEFINITION_OF_DONE.md`

## Autonomous execution contract

Codex shall:

- execute phases in order and continue without requesting routine approval;
- use existing service, dataset, vector-feature, raster and job boundaries;
- keep every provider bounded, fail-closed and provenance-complete;
- add or update tests with every behavior change;
- update `docs/CODEX_EXECUTION_LOG.md`, `docs/TODO.md` and `CHANGELOG.md`
  after each completed phase;
- run the phase gate and record exact output under
  `storage/release-evidence/<release-id>/`;
- commit and push only after the relevant local gate is green;
- deploy only a commit that passed the local gate;
- run browser and live API acceptance after deployment;
- retain Mol/Kempen as regression references while adding Wallonia, Brussels,
  cross-region, coast and North Sea golden areas.

Codex stops only when:

- an operation would destructively mutate or delete unbacked production data;
- an external licence, credential or authority decision cannot be derived from
  public source metadata;
- an official endpoint fails security validation and no authoritative
  alternative exists;
- the same mandatory gate fails three times for the same external reason; or
- continuing would require fabricating data, metrics, provenance or model
  output.

For a stop, write the blocker, evidence and exact resume command to the
execution log. Continue with independent work when possible.

## Release priorities

### P0 - release blockers

- truthful health/readiness and capability reporting;
- current backup plus isolated restore proof;
- stale job/run reconciliation and useful logs;
- temporally correct detection QA;
- production secrets, upload limits, configuration parity and rollback;
- full CI and reproducible dependencies;
- fresh install and upgrade proof.

### P1 - national product completeness

- national and maritime scope model;
- governed source coverage matrix;
- Wallonia, Brussels and North Sea adapters;
- nationwide bounded selection and mixed-zone result handling;
- historical compatibility matrix;
- golden areas outside Flanders.

### P2 - quality and maintainability

- typed response models on critical routes;
- frontend unit and browser E2E coverage;
- loading/accessibility/performance hardening;
- retention and operator documentation;
- remaining bounded modularization where protected by tests.

## Global gates

Run after every behavior-changing phase:

```bash
python -m compileall backend/app
cd backend && python -m pytest
cd frontend && npm run typecheck
cd frontend && npm run build
bash scripts/run_readiness_check.sh
cd backend && python -m alembic heads
cd backend && python -m alembic upgrade head --sql
bash -n scripts/live_migration_smoke.sh
docker compose config
```

When Tower is reachable:

```powershell
.\scripts\deploy_tower.ps1
```

Then verify:

```bash
curl -fsS http://192.168.10.150:1202/health/live
curl -fsS http://192.168.10.150:1202/health/ready
curl -fsS http://192.168.10.150:1202/api/v1/system/capabilities
```

The in-app browser acceptance must cover desktop, widescreen and a narrow
viewport. It must inspect console errors, failed requests and the complete
map-select-analyse-export flow.

## Official source baseline

The roadmap starts from these verified official families. Machine endpoints,
editions and licences must still pass source-specific probes before activation.

| Scope | Candidate | Role |
| --- | --- | --- |
| Belgium | NGI/IGN AdminVector and Top10Vector | authoritative boundary and common topographic baseline |
| Belgium | Statbel statistical sectors/population | authoritative statistical geometry and time series |
| Flanders | existing Digitaal Vlaanderen/VMM/DOV/INBO/ALZ adapters | detailed authoritative regional layers |
| Wallonia | SPW Géoportail, PICC, orthophoto, MNT and thematic services | detailed authoritative regional layers |
| Brussels | Paradigm UrbIS and regional environmental geodata | detailed authoritative urban layers |
| North Sea | FPS Marine Environment marine spatial plan | legal/use zones |
| North Sea | RBINS/BMDC and Marine Atlas | marine environment and observations |
| North Sea | MDK/Flemish Hydrography | bathymetry and nautical source candidate |
| North Sea | EMODnet | contextual fallback only when explicitly labelled |

## Phase status

| Phase | State | Purpose |
| --- | --- | --- |
| RC-0 | complete | freeze scope and produce release evidence baseline |
| RC-1 | complete | backup, restore and data safety |
| RC-2 | complete | health, capabilities and stale-runtime correctness |
| RC-3 | complete | temporal detection/QA correctness and observability |
| RC-4 | complete | national/maritime scope and provider coverage contracts |
| RC-5 | complete | deployment, secrets, configuration, fresh install and rollback |
| RC-6 | in progress | complete CI, dependency and supply-chain gates |
| RC-7 | pending | critical API envelope typing and contract validation |
| RC-8 | pending | frontend and browser E2E release journeys |
| RC-9 | pending | loading, accessibility and performance hardening |
| RC-10 | pending | retention, cleanup and national data operations |
| RC-11 | pending | final package, upgrade proof, release tag and handoff |

## RC-0 - Scope freeze and evidence baseline

**State: complete.** The read-only baseline command and focused tests pass.
Runtime evidence is stored outside Git under `storage/release-evidence/`.

### Work

- Freeze Belgium plus Belgian North Sea scope.
- Record current commit, migration head, dependency versions, container image,
  database/storage sizes and readiness results.
- Create a release-evidence manifest command.
- Replace stale product-status wording that still treats Kempen as the product
  boundary.
- Add a coverage matrix schema with `operational`, `partial`,
  `not_configured`, `unsupported`.

### Likely files

- `docs/RC_SCOPE_FREEZE_BELGIUM_NORTH_SEA.md`
- `docs/RC_ROADMAP_BELGIUM_NORTH_SEA.md`
- `docs/BUILD_STATUS.md`
- `docs/DATA_SOURCES.md`
- `scripts/capture_release_evidence.py`
- `backend/tests/test_rc_release_evidence.py`

### Evidence and exit

- scope documents contain Belgium, all three regions and maritime zones;
- evidence command is read-only and deterministic;
- manifest contains commit, dirty state, Alembic head and gate status;
- compile and focused tests pass.

## RC-1 - Backup, restore and data safety

**State: complete.** Tower retains the checksum-verified 1.4 GiB backup
`rc-belgium-north-sea-fc42ea9-secure`. Its manifest records a non-default
database password. An isolated restore reached PostGIS 3.6 and Alembic
`202607160001`, reconciled every retained critical table count and removed the
temporary database.

### Work

- Add a safe production backup command for PostgreSQL custom-format dumps.
- Capture schema head, PostGIS version, table counts, model checksums and a
  storage checksum inventory without duplicating all large artifacts.
- Reject default/empty production database passwords.
- Add backup verification with `pg_restore --list` and checksums.
- Add an isolated restore drill using a temporary database/container; never
  restore over production.
- Document retention, encryption boundary and cleanup.

### Likely files

- `scripts/backup_release_state.sh`
- `scripts/verify_release_backup.sh`
- `scripts/restore_release_backup_smoke.sh`
- `deploy/unraid/README.md`
- `docs/ROLLBACK_AND_RECOVERY.md`
- `backend/tests/test_rc_backup_restore_scripts.py`

### Exit

- current Tower database has a dated, checksum-verified backup;
- isolated restore reaches the recorded Alembic head;
- representative project, dataset, geometry and QA counts reconcile;
- no production application data was mutated by the drill.

## RC-2 - Truthful runtime state

**State: complete.** Local compile, 963 backend tests, frontend
typecheck/build and the full readiness gate pass. Tower reports green
DB/PostGIS/migration/storage readiness and configured YOLO capability. Startup
reconciled five orphaned jobs and two orphaned analysis runs; zero remain
`running`.

### Work

- Split process liveness from dependency readiness.
- Make readiness return HTTP 503 when DB, PostGIS, migration head or writable
  storage is unavailable.
- Derive version/build identity and AI capability from runtime configuration.
- Make Docker use the readiness endpoint.
- Reconcile orphaned `running` jobs and analysis runs on process startup.
- Add request IDs and exception logging without exposing secrets or full SQL.

### Likely files

- `backend/app/api/routes/health.py`
- `backend/app/schemas/health.py`
- `backend/app/core/config.py`
- `backend/app/core/logging.py`
- `backend/app/main.py`
- `backend/app/services/runtime_reconciliation_service.py`
- Docker/nginx files and health tests

### Exit

- liveness remains 200 while the process runs;
- readiness is fail-closed;
- live capabilities match the active local YOLO state;
- restart converts impossible orphaned work to a terminal failed state with
  `PROCESS_INTERRUPTED`;
- Docker reports unhealthy for a broken database.

## RC-3 - Temporal QA correctness and observability

**State: complete.** The local release gate passes backend compilation, all
974 backend tests, frontend typecheck/build, the integrated readiness script,
one Alembic head and complete offline migration SQL generation. Tower runs
immutable build `a28f2497e8d0da7d108828875ba9ffedda3c1688`, reports PostgreSQL,
PostGIS 3.6, migration `202607160001`, storage and configured YOLO as ready,
and contains no orphaned running Job or AnalysisRun. Live browser acceptance
proved that Detection Lab starts with an empty raster choice, disables its
primary action and reports the required operator choice without console
errors.

### Work

- Stop Detection Lab from silently selecting the first raster.
- Require explicit raster/model choice.
- Prevent current reference QA against historical imagery unless a compatible
  reference edition/coverage is selected.
- Persist source observation period and QA compatibility decision.
- Add structured request/job/run logs and operator diagnostics.
- Add a stale-runtime report and safe reconciliation command.

### Exit

- no detection run starts with an implicit old raster;
- incompatible temporal QA fails with a clear code;
- compatible current orthophoto/GRB path remains green;
- logs correlate request, job, analysis run and quality check.

## RC-4 - Belgium and North Sea coverage foundation

**State: complete.** Tower persists one idempotent national workspace with
eight distinct land/maritime areas and six ready authoritative datasets
containing 685 features. The coverage catalog and resolver report operational,
partial, not-configured and unsupported states honestly for Belgium, all three
regions and the Belgian maritime zones. Live browser acceptance proved the
national default context, coverage-only selection and responsive 390/1920 px
layouts while unavailable detailed regional themes remain explicit.

### Work

1. Persist NGI authoritative Belgium land and administrative scopes.
2. Persist territorial sea, EEZ and continental shelf as distinct scope
   layers.
3. Implement the coverage matrix and source authority resolver.
4. Add bounded provider contracts for:
   - NGI/Statbel national baseline;
   - SPW/PICC/orthophoto/relief for Wallonia;
   - UrbIS/environmental context for Brussels;
   - federal marine zones and BMDC/MDK marine sources.
5. Normalize the public theme vocabulary without discarding provider-native
   semantics.
6. Add Wallonia, Brussels, cross-region, coast and North Sea golden areas.
7. Make mixed-zone selection split into bounded provider requests and merge
   only semantically compatible metrics.

### Required rules

- all persistence flows through DatasetService and geospatial services;
- no provider writes directly to feature tables;
- no country-wide startup import;
- no TLS bypass;
- no implicit vertical datum conversion;
- unsupported metrics remain unavailable.

### Likely files

- provider registry/base and new regional provider modules;
- geographic scope and source coverage services;
- dataset/source schemas;
- map source portfolio and scope UI;
- source, API, data and storage documentation;
- source-specific tests and fixtures.

### Exit

- a user can draw within every golden area and receive an honest coverage
  matrix;
- at least the common national baseline is operational on land;
- regional detailed layers are operational or explicitly partial;
- maritime legal zones are operational;
- bathymetry remains gated until a source passes strict acquisition evidence.

## RC-5 - Production deployment and rollback

**State: complete.** Tower runs a commit-plus-AI-profile immutable image with
OCI revision/build labels. Repeated deploys reuse the exact image and preserve
the actual prior release. A fresh install passed on isolated volumes, manual
rollback passed against retained production mounts, and the release image
upgraded a checksum-verified 1.4 GiB backup in a generated temporary database
at PostGIS 3.6/Alembic `202607160001` before removing it. Production startup is
fail-closed for default secrets and invalid upload limits, nginx/backend limits
are aligned, and all operator-owned settings are editable in Unraid.

### Work

- align backend, nginx and proxy upload/time limits;
- make every runtime setting intentionally configured, internal or documented;
- reject production default secrets;
- add image labels with version, commit and build time;
- deploy immutable tags in addition to `latest`;
- preserve previous image for one-command rollback;
- prove fresh install against empty volumes;
- prove upgrade from a copied current database/storage state;
- prove rollback without schema or data loss.

### Exit

- Unraid edit fields cover all operator-owned settings;
- fresh install, upgrade and rollback commands are documented and executed;
- deployed UI exposes build identity;
- previous image remains addressable;
- no mutable-only release is accepted.

## RC-6 - CI and supply-chain gates

### Work

- make CI run backend compile/tests, frontend typecheck/build, readiness,
  Alembic single-head/offline SQL and Docker config/build;
- add Python and npm dependency vulnerability checks;
- add container scan and SBOM;
- introduce reproducible Python resolution/lock output;
- preserve optional AI dependency separation;
- publish gate artefacts.

### Exit

- a broken compile, test, build, migration or contract blocks CI;
- dependency and container findings have severity policy;
- a clean checkout can reproduce dependency versions and image metadata.

## RC-7 - Critical API contract hardening

### Work

- replace `response_model=dict` first on health, projects, areas, datasets,
  jobs, analysis runs, QA, exports and map-critical endpoints;
- retain canonical `{data: ...}` success and canonical error envelopes;
- add OpenAPI response validation tests;
- document intentionally streaming/download responses separately.

### Exit

- all critical map-first routes have concrete response models;
- actual payloads pass FastAPI validation;
- API contract audit and frontend typecheck pass.

## RC-8 - Release journey automation

### Work

- add frontend unit tests for selection, coverage, loading and temporal guards;
- add browser E2E for:
  - open Belgium map;
  - select each golden area;
  - load a theme;
  - inspect metric/provenance;
  - compare compatible history;
  - run a bounded configured detection path;
  - export;
  - ask Ollama with persisted context;
- include no-data, partial coverage, provider failure and unsupported metric
  journeys.

### Exit

- E2E runs against a real backend/PostGIS fixture;
- no workflow depends only on source-text assertion tests;
- console and failed-request audits are clean.

## RC-9 - UX, accessibility and performance

### Work

- replace initial false `missing` source states with explicit loading;
- provide accessible names, labels, keyboard focus and status announcements;
- enforce desktop, widescreen and narrow layouts;
- record API and map-selection budgets for golden areas;
- virtualize or page large panels;
- split large modules only where tests protect behavior.

### Exit

- no core control lacks an accessible name;
- loading, empty, partial, error and ready states are distinct;
- map selection remains usable at target viewports;
- no release journey exceeds its documented budget without a visible warning.

## RC-10 - Data operations and retention

### Work

- define lifecycle for raw, normalized, derived, export, AI and evidence
  artifacts;
- add dry-run-first cleanup for stale technical projects, failed work and
  superseded caches;
- retain immutable official source editions and release evidence;
- report disk pressure before acquisition;
- schedule nothing implicitly; provide explicit operator commands;
- add source freshness reports for national/regional/maritime families.

### Exit

- cleanup cannot remove release evidence or current authoritative editions;
- every destructive action requires explicit confirmation and a current
  backup;
- storage growth and retained provenance are reportable.

## RC-11 - Final release package

### Work

- rerun fresh-install and upgrade proof using the release image;
- rerun backup/isolated restore and rollback proof;
- execute every golden area and core browser journey;
- generate coverage, security, dependency, migration, performance and known
  limitation reports;
- update operator/end-user documentation;
- remove stale status documents from active navigation without deleting
  historical evidence;
- assign semantic version, create signed/checksummed release manifest, tag,
  push and deploy the immutable image.

### Final acceptance

- all P0 findings are closed;
- every mandatory global gate is green;
- backup and isolated restore are proven;
- one Alembic head applies to empty and upgraded PostGIS;
- Belgium and maritime golden areas pass;
- no unsupported metric or unavailable source appears as successful;
- live browser acceptance passes at port 1202;
- remaining P1/P2 limitations are explicit, bounded and non-deceptive.

## Next command

After creating this roadmap, start RC-0 immediately:

```bash
python scripts/capture_release_evidence.py --output storage/release-evidence/rc-current/baseline.json
```

If the command does not exist yet, implementing it and its tests is the first
code task.