Add governed orthophoto release promotion
GeoIntel CI / docs-smoke (push) Canceled after 0s
GeoIntel CI / contract-smoke (push) Canceled after 0s

This commit is contained in:
Codex
2026-07-17 02:17:04 +02:00
parent 20d3aae1fb
commit 06d05f03a6
14 changed files with 1579 additions and 28 deletions
+19
View File
@@ -7,6 +7,25 @@
# Changelog
## Sprint 231 Governed orthophoto release promotion (2026-07-17)
- Added an operator-only `plan -> stage -> review -> apply` coordinator for one
bounded current Digitaal Vlaanderen orthophoto selection. It reuses and
revalidates the Sprint 230 catalog, WMS, WCS and flight-day preflight.
- Stage performs exactly one bounded allowlisted WMS `GetMap`, retains the raw
response, normalizes a three-band EPSG:31370 GeoTIFF and creates a PNG review
preview. Every source, raster, preview, request and preflight identity is
SHA-256-bound under persistent operator evidence.
- Review requires a named explicit approval. Apply requires exact plan and
review hashes, rechecks remote and local state, and uploads only the approved
GeoTIFF through the existing dataset upload/DatasetService transaction.
- Added an explicit first-baseline transition for legacy
`most_recent_at_*` markers. It requires both the exact legacy marker and the
official edition; no existing Dataset is rewritten or deleted.
- Made official `YYYY.NN` orthophoto Datasets take precedence over rolling
acquisition markers in source-catalog comparison. No API, migration,
scheduler, browser fetch or automatic refresh was added.
## Sprint 230 Governed orthophoto release preflight (2026-07-17)
- Added an operator-only, read-only preflight for the current Digitaal
+42 -6
View File
@@ -1313,12 +1313,48 @@ docker exec geointel python /app/scripts/orthophoto_release_preflight.py \
The command reads canonical API envelopes, exact official WMS capabilities,
WCS `DescribeCoverage` and at most 64 queryable flight-day points. It never
requests raster pixels or mutates application/storage state. Only
`staging_permitted=true` may feed a future separate staging command. `current`,
remote-older, mixed/incorrect flight years and legacy local values such as
`most_recent_at_2026-07-15` remain non-stageable. The report's point grid is
flight-date evidence; complete selected-area coverage comes from containment
inside the official 15 cm WCS raster domain.
requests raster pixels or mutates application/storage state. `current`,
remote-older and mixed/incorrect flight years remain non-stageable. The
report's point grid is flight-date evidence; complete selected-area coverage
comes from containment inside the official 15 cm WCS raster domain.
Official release promotion is a separate four-action operator workflow. Run it
inside the all-in-one container so stage/apply can use only the loopback API:
```bash
# Read-only decision; copy the reported edition and current local marker.
docker exec geointel python /app/scripts/manage_orthophoto_release.py plan \
--project-id 82a85913-c522-45d7-84a1-02b393d89e55 \
--bbox 5.110 51.180 5.117 51.185 --refresh-catalog
# First official baseline only: both values must match the fresh preflight.
docker exec geointel python /app/scripts/manage_orthophoto_release.py stage \
--project-id 82a85913-c522-45d7-84a1-02b393d89e55 \
--bbox 5.110 51.180 5.117 51.185 \
--confirm-edition 2025.04 \
--establish-official-baseline \
--confirm-local-version most_recent_at_2026-07-15
# Inspect review-preview.png, then use the exact plan SHA printed by stage.
docker exec geointel python /app/scripts/manage_orthophoto_release.py review \
--project-id 82a85913-c522-45d7-84a1-02b393d89e55 \
--bbox 5.110 51.180 5.117 51.185 \
--confirm-edition 2025.04 --confirm-plan-sha256 <plan-sha256> \
--approve --reviewer "<operator name>" --review-note "<bounded review>"
# Apply only the exact approved bytes and hashes.
docker exec geointel python /app/scripts/manage_orthophoto_release.py apply \
--project-id 82a85913-c522-45d7-84a1-02b393d89e55 \
--bbox 5.110 51.180 5.117 51.185 \
--confirm-edition 2025.04 --confirm-plan-sha256 <plan-sha256> \
--confirm-review-sha256 <review-sha256>
```
For a later comparable `YYYY.NN` update, omit the two first-baseline flags.
Stage performs one bounded pixel request but no database mutation. Apply is
idempotent for the exact plan/raster checksum, creates a new immutable raster
Dataset and DatasetVersion with the official edition, and retains every older
snapshot. No command is scheduled or invoked by startup or browser actions.
## Governed DHMV terrain acquisition
@@ -37,7 +37,7 @@ _WFS = "http://www.opengis.net/wfs/2.0"
_XLINK = "http://www.w3.org/1999/xlink"
_METADATA_HOST = "metadata.vlaanderen.be"
_VERSION_DATE = re.compile(r"^(?:toestand\s+)?(\d{4}-\d{2}-\d{2})$", re.IGNORECASE)
_ORTHOPHOTO_EDITION = re.compile(r"^\d{4}\.\d{2}$")
_ORTHOPHOTO_EDITION = re.compile(r"^(20\d{2})\.(\d{2})$")
_ALZ_SOURCE_NAME = "agentschap_landbouw_zeevisserij_agricultural_parcels"
_ALZ_RELEASE_HOST = "landbouwcijfers.vlaanderen.be"
_ALZ_RELEASE_PATH = "/open-geodata-landbouwgebruikspercelen"
@@ -681,6 +681,19 @@ def _dataset_source_name(dataset: Dataset) -> str:
def _latest_local_version(source_name: str, datasets: list[Dataset]) -> str | None:
candidates = [item for item in datasets if _dataset_source_name(item) == source_name and item.source_version]
if source_name == "digitaal_vlaanderen_orthophoto":
official_editions = [
item for item in candidates if _ORTHOPHOTO_EDITION.fullmatch((item.source_version or "").strip())
]
if official_editions:
latest = max(
official_editions,
key=lambda item: (
tuple(int(value) for value in (item.source_version or "0.0").split(".")),
_utc(item.imported_at) if item.imported_at else datetime.min.replace(tzinfo=timezone.utc),
str(item.id),
),
)
return latest.source_version
explicit_current = [
item
for item in candidates
@@ -0,0 +1,405 @@
from __future__ import annotations
import argparse
from datetime import datetime, timezone
import importlib.util
import json
from pathlib import Path
from types import SimpleNamespace
import sys
import numpy as np
import pytest
from rasterio.io import MemoryFile
from rasterio.transform import from_origin
ROOT = Path(__file__).resolve().parents[2]
SCRIPTS = ROOT / "scripts"
if str(SCRIPTS) not in sys.path:
sys.path.insert(0, str(SCRIPTS))
def load_script(name: str):
path = SCRIPTS / name
module_name = f"test_{path.stem}_sprint231"
spec = importlib.util.spec_from_file_location(module_name, path)
assert spec is not None
assert spec.loader is not None
module = importlib.util.module_from_spec(spec)
sys.modules[module_name] = module
spec.loader.exec_module(module)
return module
MANAGER = load_script("manage_orthophoto_release.py")
def arguments(tmp_path: Path, **overrides) -> argparse.Namespace:
values = {
"action": "plan",
"project_id": "00000000-0000-0000-0000-000000000001",
"scope": "kempen-transport-region",
"api_url": "http://127.0.0.1:8000/api/v1",
"bbox": [5.110, 51.180, 5.113, 51.182],
"area_id": None,
"confirm_edition": None,
"confirm_plan_sha256": None,
"confirm_review_sha256": None,
"approve": False,
"reviewer": None,
"review_note": "",
"establish_official_baseline": False,
"confirm_local_version": None,
"plan_path": None,
"review_path": None,
"evidence_root": tmp_path / "operator-evidence" / "orthophoto-release",
"refresh_catalog": False,
"api_timeout": 180,
"wms_timeout": 60,
"import_timeout": 600,
"max_response_mb": 32,
}
values.update(overrides)
return argparse.Namespace(**values)
def report(*, status: str = "update_available", local: str | None = "2024.03") -> dict:
return {
"schema_version": 1,
"status": "passed",
"generated_at": "2026-07-17T10:00:00Z",
"project_id": "00000000-0000-0000-0000-000000000001",
"scope": "kempen-transport-region",
"product": {
"key": "most_recent",
"display_name": "Orthofoto meest recent",
"temporal_granularity": "snapshot",
"native_resolution_m": 0.15,
"supports_detection": True,
"color_mode": "rgb",
"catalog_url": MANAGER.preflight.CATALOG_URL,
},
"release": {
"status": status,
"remote_edition": "2025.04",
"remote_year": 2025,
"local_source_version": local,
"comparison_status": "different" if status == "update_available" else "not_comparable",
"metadata_identifier": MANAGER.preflight.METADATA_IDENTIFIER,
"metadata_url": "https://metadata.vlaanderen.be/srv/dut/catalog.search#/metadata/f5304d6d",
"remote_title": "Orthofoto meest recent, 2025.04",
"remote_modified_at": "2026-04-27T00:00:00Z",
"remote_published_at": "2025-12-11T00:00:00Z",
"catalog_checked_at": "2026-07-17T10:00:00Z",
"capabilities_url": MANAGER.WMS_BASE_URL + "?SERVICE=WMS&REQUEST=GetCapabilities",
"capabilities_sha256": "a" * 64,
},
"capabilities": {
"service_version": "1.3.0",
"capabilities_sha256": "a" * 64,
"layers": ["Ortho", "Vliegdagcontour"],
"vliegdagcontour_queryable": True,
"feature_info_format": "application/geo+json",
"extent_epsg31370": [0.0, 0.0, 300000.0, 300000.0],
"metadata_identifier": MANAGER.preflight.METADATA_IDENTIFIER,
},
"coverage_domain": {
"coverage_id": "Ortho",
"crs": "EPSG:31370",
"extent_epsg31370": [0.0, 0.0, 300000.0, 300000.0],
"native_resolution_m": 0.15,
"band_count": 3,
"native_format": "image/tiff",
"coverage_description_sha256": "b" * 64,
"selected_area_fully_inside_domain": True,
"pixel_data_requested": False,
},
"selection": {
"bbox_epsg4326": [5.110, 51.180, 5.113, 51.182],
"bbox_epsg31370": [200000.0, 210000.0, 200200.0, 210160.0],
"width_m": 200.0,
"height_m": 160.0,
},
"flight_day_coverage": {
"status": "passed",
"mode": "official_queryable_flight_day_grid",
"sample_count": 4,
"grid_columns": 2,
"grid_rows": 2,
"covered_sample_count": 4,
"sample_coverage_ratio": 1.0,
"flight_dates": ["5/4/2025"],
"flight_years": [2025],
"feature_ids": ["123"],
"sample_evidence_sha256": "c" * 64,
"claim_boundary": "Bounded point evidence, not a polygon-union proof.",
},
"flight_year_matches_release": True,
"staging_permitted": status in {"not_loaded", "update_available"},
"next_action": "governed_pixel_stage",
"pixel_requests_performed": 0,
"datasets_mutated": 0,
"automatic_staging": False,
"automatic_import": False,
}
def raw_rgb_tiff(width: int = 200, height: int = 160) -> bytes:
pixels = np.zeros((3, height, width), dtype=np.uint8)
pixels[0] = 80
pixels[1] = np.arange(width, dtype=np.uint8)[None, :]
pixels[2] = np.arange(height, dtype=np.uint8)[:, None]
with MemoryFile() as memory:
with memory.open(
driver="GTiff",
width=width,
height=height,
count=3,
dtype="uint8",
crs="EPSG:31370",
transform=from_origin(200000.0, 210160.0, 1.0, 1.0),
) as dataset:
dataset.write(pixels)
return memory.read()
class Response:
def __init__(self, body: bytes, url: str, content_type: str = "image/tiff") -> None:
self.body = body
self.url = url
self.headers = {"Content-Type": content_type, "Content-Length": str(len(body))}
def __enter__(self):
return self
def __exit__(self, *_args):
return None
def geturl(self) -> str:
return self.url
def read(self, amount: int | None = None) -> bytes:
return self.body if amount is None else self.body[:amount]
def staged_plan(tmp_path: Path) -> tuple[argparse.Namespace, Path, dict, dict]:
args = arguments(
tmp_path,
action="stage",
confirm_edition="2025.04",
)
release_report = report()
request = MANAGER.map_request(release_report)
staged = MANAGER.stage_artifacts(
args,
release_report,
request,
opener=lambda _request, timeout: Response(raw_rgb_tiff(), request["url"]),
)
plan = MANAGER.build_staged_plan(
args,
release_report,
MANAGER.authorize_stage(args, release_report),
staged,
)
path = MANAGER.default_plan_path(args, "2025.04")
MANAGER.write_json(path, plan)
return args, path, plan, staged
def test_orthophoto_catalog_prefers_latest_official_edition_over_rolling_marker() -> None:
from app.services.source_catalog_probe_service import _latest_local_version
now = datetime(2026, 7, 17, tzinfo=timezone.utc)
datasets = [
SimpleNamespace(
source_name=MANAGER.SOURCE_NAME,
source="",
source_version="2025.04",
imported_at=now,
observed_at=now,
id="official-2025",
),
SimpleNamespace(
source_name=MANAGER.SOURCE_NAME,
source="",
source_version="most_recent_at_2026-07-18",
imported_at=datetime(2026, 7, 18, tzinfo=timezone.utc),
observed_at=now,
id="rolling",
),
SimpleNamespace(
source_name=MANAGER.SOURCE_NAME,
source="",
source_version="2026.02",
imported_at=datetime(2026, 7, 16, tzinfo=timezone.utc),
observed_at=now,
id="official-2026",
),
]
assert _latest_local_version(MANAGER.SOURCE_NAME, datasets) == "2026.02"
def test_stage_requires_exact_official_or_explicit_legacy_baseline_confirmation(tmp_path: Path) -> None:
args = arguments(tmp_path)
normal = report()
assert MANAGER.authorize_stage(args, normal)["mode"] == "normal_release"
legacy = report(status="blocked_local_version", local="most_recent_at_2026-07-15")
with pytest.raises(RuntimeError, match="establish-official-baseline"):
MANAGER.authorize_stage(args, legacy)
args.establish_official_baseline = True
args.confirm_local_version = "most_recent_at_2026-07-15"
assert MANAGER.authorize_stage(args, legacy)["mode"] == "explicit_legacy_baseline_transition"
args.confirm_local_version = "different"
with pytest.raises(RuntimeError, match="exact"):
MANAGER.authorize_stage(args, legacy)
def test_stage_fetches_one_allowlisted_map_and_writes_reviewable_rgb_evidence(tmp_path: Path) -> None:
args = arguments(tmp_path)
release_report = report()
request = MANAGER.map_request(release_report)
calls = []
def opener(http_request, timeout):
calls.append((http_request.full_url, timeout))
return Response(raw_rgb_tiff(), request["url"])
staged = MANAGER.stage_artifacts(args, release_report, request, opener=opener)
manifest = MANAGER.validate_staged_artifacts(args, Path(staged["manifest_path"]))
assert calls == [(request["url"], 60)]
assert manifest["pixel_request_count"] == 1
assert manifest["datasets_mutated"] == 0
assert manifest["normalized_geotiff"]["crs"] == "EPSG:31370"
assert manifest["normalized_geotiff"]["band_count"] == 3
assert Path(manifest["review_preview"]["path"]).read_bytes().startswith(b"\x89PNG")
def test_getmap_redirect_and_response_limits_fail_closed(tmp_path: Path) -> None:
request = MANAGER.map_request(report())
with pytest.raises(RuntimeError, match="allowlist"):
MANAGER.fetch_map(
request,
timeout=5,
max_bytes=10_000_000,
opener=lambda *_args, **_kwargs: Response(raw_rgb_tiff(), "https://example.com/image.tif"),
)
body = raw_rgb_tiff()
with pytest.raises(RuntimeError, match="release limit"):
MANAGER.fetch_map(
request,
timeout=5,
max_bytes=len(body) - 1,
opener=lambda *_args, **_kwargs: Response(body, request["url"]),
)
def test_plan_and_artifact_tampering_are_rejected(tmp_path: Path) -> None:
args, path, plan, staged = staged_plan(tmp_path)
args.confirm_plan_sha256 = plan["plan_sha256"]
_, loaded, _ = MANAGER.load_staged_plan(args, "2025.04")
assert loaded == plan
Path(staged["normalized_geotiff"]["path"]).write_bytes(b"tampered")
with pytest.raises(RuntimeError, match="no longer match"):
MANAGER.load_staged_plan(args, "2025.04")
outside = tmp_path / "outside.json"
with pytest.raises(RuntimeError, match="outside the governed"):
MANAGER.governed_path(args, outside)
def test_review_requires_named_approval_exact_plan_and_unchanged_preflight(tmp_path: Path) -> None:
args, path, plan, staged = staged_plan(tmp_path)
manifest = MANAGER.validate_staged_artifacts(args, Path(staged["manifest_path"]))
with pytest.raises(RuntimeError, match="named --reviewer"):
MANAGER.build_review_evidence(args, path, plan, manifest)
args.approve = True
args.reviewer = "Jens"
review = MANAGER.build_review_evidence(args, path, plan, manifest)
assert review["review_preview_sha256"] == manifest["review_preview"]["sha256"]
assert review["review_sha256"] == MANAGER.canonical_sha256(review, "review_sha256")
changed = report()
changed["capabilities"]["capabilities_sha256"] = "d" * 64
with pytest.raises(RuntimeError, match="evidence changed"):
MANAGER.require_preflight_unchanged(plan, changed, require_local=True)
def test_approved_upload_uses_canonical_dataset_route_and_complete_provenance(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
args, path, plan, staged = staged_plan(tmp_path)
manifest = MANAGER.validate_staged_artifacts(args, Path(staged["manifest_path"]))
args.approve = True
args.reviewer = "Jens"
review = MANAGER.build_review_evidence(args, path, plan, manifest)
captured = {}
class UploadResponse:
def __enter__(self):
return self
def __exit__(self, *_args):
return None
def read(self):
return json.dumps(
{
"data": {
"id": "dataset-1",
"status": "ready",
"source_name": MANAGER.SOURCE_NAME,
"source_version": "2025.04",
"checksum_sha256": manifest["normalized_geotiff"]["sha256"],
}
}
).encode("utf-8")
def fake_urlopen(request, timeout):
captured["url"] = request.full_url
captured["body"] = request.data
captured["timeout"] = timeout
return UploadResponse()
monkeypatch.setattr(MANAGER, "urlopen", fake_urlopen)
dataset = MANAGER.upload_approved_dataset(args, plan, manifest, review)
assert dataset["id"] == "dataset-1"
assert captured["url"].endswith(f"/projects/{args.project_id}/datasets/upload")
assert b'name="source_version"\r\n\r\n2025.04' in captured["body"]
assert b'name="temporal_series_key"' in captured["body"]
assert plan["plan_sha256"].encode("ascii") in captured["body"]
assert review["review_sha256"].encode("ascii") in captured["body"]
plan_path_json = json.dumps(str(MANAGER.default_plan_path(args, "2025.04")))[1:-1]
review_path_json = json.dumps(str(MANAGER.default_review_path(args, "2025.04")))[1:-1]
assert plan_path_json.encode("utf-8") in captured["body"]
assert review_path_json.encode("utf-8") in captured["body"]
def test_apply_target_must_be_loopback_api() -> None:
assert MANAGER.internal_api_url("http://127.0.0.1:8000/api/v1").endswith("/api/v1")
with pytest.raises(RuntimeError, match="local /api/v1"):
MANAGER.internal_api_url("http://192.168.10.150:1202/api/v1")
with pytest.raises(RuntimeError, match="local /api/v1"):
MANAGER.internal_api_url("http://127.0.0.1:8000/not-api")
def test_official_flight_dates_are_persisted_without_inventing_a_catalog_date() -> None:
dates = MANAGER.parse_flight_dates(["5/4/2025", "2025-04-06", "05-04-2025"])
assert [value.date().isoformat() for value in dates] == ["2025-04-05", "2025-04-06"]
with pytest.raises(RuntimeError, match="not safely parseable"):
MANAGER.parse_flight_dates(["spring 2025"])
def test_release_manager_is_packaged_and_compiled_by_readiness() -> None:
dockerfile = (ROOT / "deploy" / "unraid" / "Dockerfile.all-in-one").read_text(encoding="utf-8")
readiness = (ROOT / "scripts" / "run_readiness_check.sh").read_text(encoding="utf-8")
assert "COPY scripts/manage_orthophoto_release.py /app/scripts/manage_orthophoto_release.py" in dockerfile
assert "-m py_compile scripts/manage_orthophoto_release.py" in readiness
+1
View File
@@ -93,6 +93,7 @@ COPY scripts/provision_regional_bwk_natura2000.py /app/scripts/provision_regiona
COPY scripts/provision_agricultural_parcel_history.py /app/scripts/provision_agricultural_parcel_history.py
COPY scripts/manage_alz_agriculture_release.py /app/scripts/manage_alz_agriculture_release.py
COPY scripts/orthophoto_release_preflight.py /app/scripts/orthophoto_release_preflight.py
COPY scripts/manage_orthophoto_release.py /app/scripts/manage_orthophoto_release.py
COPY scripts/provision_buildings_addresses_register.py /app/scripts/provision_buildings_addresses_register.py
COPY scripts/provision_regional_timeseries.py /app/scripts/provision_regional_timeseries.py
COPY scripts/geographic_scopes.py /app/scripts/geographic_scopes.py
+11
View File
@@ -476,6 +476,17 @@ containment, sampled flight dates/years, local comparison state and
`staging_permitted`. It performs no pixel request, upload, Job, Dataset write
or legacy metadata rewrite. This operator script adds no public API contract.
Governed pixel promotion remains outside the HTTP request cycle in
`scripts/manage_orthophoto_release.py`. `plan` reruns that read-only preflight;
`stage` makes exactly one allowlisted bounded `Ortho` GetMap request and writes
only checksummed source/raster/preview evidence; `review` requires a named
approval; and `apply` requires the exact plan/review SHA-256 values. Apply
revalidates the remote identity and local comparison state, then delegates to
the existing `POST .../datasets/upload` contract with `dataset_type=raster`,
`source_name=digitaal_vlaanderen_orthophoto`, the official `YYYY.NN`
`source_version` and complete source/provenance metadata. No release endpoint,
provider URL parameter, Job type or alternate response envelope is added.
The endpoint accepts no arbitrary URL, feature query, area or layer. It does
not fetch vector features, raster pixels or models, create jobs/datasets, write
to PostGIS or trigger an import. The normal `source-freshness` endpoint remains
+32
View File
@@ -9767,3 +9767,35 @@ Boundary:
backfill existing orthophotos. A future stage/review/apply coordinator must
revalidate and retain the exact preflight identity before creating a new
immutable raster Dataset with official `YYYY.NN` source version.
## Sprint 231 - Governed orthophoto release promotion (2026-07-17)
Implemented:
- Added `scripts/manage_orthophoto_release.py` with separate `plan`, `stage`,
named `review` and checksum-confirmed `apply` actions for one bounded current
orthophoto selection. No action is scheduled or browser-triggered.
- Reused the complete Sprint 230 preflight identity. Stage performs exactly one
allowlisted WMS `Ortho` GetMap, retains the exact source response, creates a
three-band EPSG:31370 GeoTIFF plus PNG preview and mutates no Dataset.
- Bound remote catalog/WMS/WCS/flight evidence, request identity, all staged
file hashes, reviewer and the final Dataset checksum into atomic persistent
evidence. Host/path drift, oversize responses, modified bytes, stale local
state and missing exact confirmations fail closed.
- Added a double-confirmed first official baseline transition for legacy
`most_recent_at_*` markers. Apply retains every legacy raster and uses only
the existing canonical upload/DatasetService transaction.
- Made the latest official `YYYY.NN` Dataset authoritative for source-catalog
comparison even when a newer-imported rolling marker also exists. No API,
migration, release table, Job type or frontend behavior changed.
Validation so far:
- 38 focused Sprint 222/230/231 tests pass with deprecations treated as errors.
Coverage includes official-edition ordering, first-baseline authorization,
GetMap allowlisting/limits, RGB/CRS normalization, preview generation,
tampering, preflight drift, named review, loopback-only apply and complete
upload provenance.
Next:
- Run complete readiness, deploy the packaged operator to Tower, inspect the
live bounded Mol review preview and only then apply the first official
`2025.04` immutable baseline with explicit review evidence.
+9 -7
View File
@@ -291,13 +291,15 @@ annual Dataset, DatasetVersion and vector_features records with
`source_version=<year>-definitive`. Earlier annual snapshots are retained and
provisional v1/v2 publications cannot create rows.
The current-orthophoto release preflight likewise adds no lifecycle table or
migration. It is read-only and creates neither Dataset nor Job. It compares the
existing local `source_version` with the official `YYYY.NN` catalog edition
and verifies WMS/WCS/flight-day evidence for one bounded selection. A later
governed pixel apply must still create an immutable raster Dataset plus
DatasetVersion through DatasetService and retain that exact edition/evidence;
direct metadata backfill of legacy `most_recent_at_*` rows is prohibited.
Current-orthophoto release management likewise adds no lifecycle table or
migration. Preflight is read-only; stage and named review are filesystem-only.
Approved apply uploads the exact checksummed EPSG:31370 GeoTIFF through the
existing dataset endpoint and DatasetService transaction, creating one normal
immutable raster Dataset and DatasetVersion with official `YYYY.NN`
`source_version`, temporal flight-date evidence and release provenance.
Official editions take precedence over legacy rolling markers in catalog
comparison, but direct metadata backfill, update or deletion of those legacy
rows remains prohibited.
## Geometry normalization
+13 -2
View File
@@ -24,7 +24,7 @@ aanvraag. GeoIntel verzint geen historische pixelopnamedatum. Bronnen:
- https://www.vlaanderen.be/datavindplaats/catalogus/orthofotomozaiek-middenschalig-winteropnamen-kleur-meest-recent-vlaanderen
- https://www.vlaanderen.be/digitaal-vlaanderen/onze-diensten-en-platformen/luchtopnamen/gebruik-orthofotomozaieken
Voor een toekomstige rolling-releasebeslissing gebruikt
Voor een rolling-releasebeslissing gebruikt
`scripts/orthophoto_release_preflight.py` uitsluitend metadata. Het bindt de
lokale `most_recent`-productvariant aan de officiële ISO-editie en exacte WMS-
capabilitieshash, controleert het EPSG:31370/15 cm/driebanden-rasterdomein via
@@ -39,6 +39,17 @@ union. Een selectie met ontbrekende contourpunten, meerdere/afwijkende
vluchtjaren, gewijzigde service-identiteit of een niet-vergelijkbare lokale
`most_recent_at_*` marker is niet stagebaar.
`scripts/manage_orthophoto_release.py` voert daarna uitsluitend op expliciet
operatorverzoek `plan`, `stage`, `review` en `apply` uit. Stage haalt exact een
begrensde `Ortho`-GetMap op, bewaart bronresponse, genormaliseerde RGB-GeoTIFF
en review-PNG met checksums, maar schrijft geen Dataset. Review koppelt een
benoemde goedkeuring aan exact die bytes. Apply valideert preflight, plan,
review en lokale bronstaat opnieuw en gebruikt vervolgens de bestaande
Dataset-uploadroute. De nieuwe Dataset krijgt de officiële `YYYY.NN`-editie;
oude `most_recent_at_*` rasters blijven onveranderd bewaard. De eerste overgang
vereist een aparte baselinevlag plus de exacte oude marker. Er is geen
scheduler, browserfetch of automatische vervanging.
Dit document verzamelt concrete databronnen voor GeoIntel Kempen.
## Cross-domain official area profile
@@ -189,7 +200,7 @@ gebeurd.
| GRB gebouwen/wegen/water/percelen | operationele, expliciete plan-stage-apply refresh met onveranderlijke snapshots | alleen een nieuw officieel gedateerd cataloguseditie na operatorbevestiging ophalen |
| Statbel bevolking | jaarlijkse, expliciete edities in één tijdreeks; officiële DCAT-releaseprobe | een nieuwe publicatie alleen na schema-, sectorgeometrie- en totalencontrole toevoegen |
| ALZ landbouwgebruikspercelen | definitieve jaarlijkse edities 2008-2025; expliciete publicatieprobe en plan-stage-review-apply promotie; metricvergelijking zonder objectlineage | alleen een nieuwere definitieve v3-editie na gestagede schema-/codelijst-/scopecontrole en benoemde review toevoegen |
| orthofoto | vaste lokale opname per expliciete analysezone; read-only releasepreflight voor variant, officiële editie, exact WCS-domein en begrensd vluchtjaarbewijs | eerst officiële lokale editieprovenance vastleggen; daarna pas een afzonderlijke menselijke pixel-stage/apply-flow bouwen |
| orthofoto | vaste lokale analyseopnamen plus operationele preflight en benoemde plan-stage-review-apply-promotie voor een officiële `YYYY.NN`-editie | alleen een nieuwere officiële editie als afzonderlijke immutable Dataset promoveren na verse evidence en review |
| landgebruik, thematische rasters, DHMV en VMM-scenario's | vaste product-/scenario-edities, geen rolling snapshot | alleen een nieuwe gedocumenteerde producteditie als afzonderlijke Dataset verwerven |
| bodemkaart en historische kaarten | historische referentie-editie | niet als verouderde actuele bron labelen; alleen vervangen bij een officiële inhoudelijke heruitgave |
| BWK/Natura 2000 en gebouwen-/adressenregister | expliciete actuele snapshot met eigen methodologische betekenis | eerst een stabiele officiële editieprobe en bron-specifieke reconciliatiecontrole toevoegen |
+7 -3
View File
@@ -274,12 +274,16 @@ layer, observation label, `observed_at`, optional `valid_from`/`valid_to`,
temporal granularity, request/spatial hash, attribution and a limitation that
states whether the product is annual, multi-year or merely most recent.
A future rolling `most_recent` import must also retain the exact official
A governed rolling `most_recent` release import must also retain the exact official
`YYYY.NN` edition and the preflight identities for WMS capabilities, WCS
coverage description, selected EPSG:31370 domain and sampled flight year. A
legacy `most_recent_at_<date>` value is acquisition timing, not an official
edition, and cannot be promoted or compared as if it were one. The read-only
preflight creates no Dataset and does not retroactively rewrite that evidence.
edition, and cannot be compared as if it were one. The read-only preflight
creates no Dataset. Stage retains the exact source response, normalized
three-band EPSG:31370 GeoTIFF and review preview. Apply requires exact plan and
review hashes, persists sampled official flight dates as the temporal evidence
range and creates a new Dataset/DatasetVersion through DatasetService. It does
not retroactively rewrite or delete legacy rows.
### Hydrological station observations
+21 -6
View File
@@ -148,12 +148,27 @@ in source/provenance metadata. Browser PNG rendering is derived on request and
does not replace the stored GeoTIFF.
The orthophoto release preflight writes no source file, raster or database row.
Its JSON stdout may be retained by an operator as review evidence, but it is
not itself staging authorization. The report binds official WMS and WCS XML
hashes, the exact selected domain and hashed flight-day sample evidence. A
future pixel stage must persist and revalidate that identity separately before
DatasetService is called; existing `most_recent_at_*` raster metadata is not
silently rewritten.
Its JSON stdout is not staging authorization. Governed release evidence is
retained separately per scope, official edition and exact selection hash:
```text
storage/operator-evidence/orthophoto-release/{scope}/{YYYY.NN}/{selection-hash}/
official-wms-response.tif
orthophoto_{YYYY.NN}_{selection-hash}.tif
review-preview.png
staged-manifest.json
staged-plan.json
review-evidence.json
applied-evidence.json
```
The manifest binds the one bounded source response, normalized three-band
EPSG:31370 GeoTIFF and PNG preview. The plan also binds WMS/WCS/catalog and
flight-day preflight identities; review binds a named approval; applied
evidence binds both to the immutable Dataset id and checksum. Paths outside
this root, changed files and changed provider/local state fail closed. Only the
normalized GeoTIFF enters ordinary Dataset storage through DatasetService.
Existing `most_recent_at_*` raster metadata is never rewritten.
DHMV II DTM/DSM outputs are also normal raster Dataset files. The provider WCS
returns multipart coverage data; GeoIntel retains response and extracted
+3 -3
View File
@@ -692,6 +692,6 @@ This file now starts with the current implementation status. Older preparation/b
campaign-snapshot and definitive-archive publication contract.
- [x] Add a read-only orthophoto preflight for product variant, official
edition, exact WCS selected-area domain and deterministic flight-year points.
- [ ] Keep orthophoto pixel refresh manual and blocked until a separate
plan-stage-review-apply flow can retain the passed preflight identity and
create a new immutable Dataset with official `YYYY.NN` source version.
- [x] Keep orthophoto pixel refresh manual through a separate
plan-stage-review-apply flow that retains the passed preflight identity and
creates a new immutable Dataset with official `YYYY.NN` source version.
File diff suppressed because it is too large Load Diff
+1
View File
@@ -56,6 +56,7 @@ ${PYTHON_BIN} -m py_compile scripts/provision_regional_bwk_natura2000.py
${PYTHON_BIN} -m py_compile scripts/provision_agricultural_parcel_history.py
${PYTHON_BIN} -m py_compile scripts/manage_alz_agriculture_release.py
${PYTHON_BIN} -m py_compile scripts/orthophoto_release_preflight.py
${PYTHON_BIN} -m py_compile scripts/manage_orthophoto_release.py
${PYTHON_BIN} -m py_compile scripts/provision_buildings_addresses_register.py
${PYTHON_BIN} -m py_compile scripts/provision_mol_dhmv.py
${PYTHON_BIN} -m py_compile scripts/provision_regional_dhmv.py