geointel/docs/40-build-launch/DATA_ACQUISITION_PLAYBOOK.md

DATA ACQUISITION PLAYBOOK

This playbook tells Codex how to think about data acquisition without spending the first build researching or overengineering live integrations.

Principle

Sprint 1 uses local demo fixtures first. Live external data integrations are later adapters.

The data hierarchy is:

1. Golden fixtures for tests and UI development.
2. Local user-uploaded files.
3. Cached reference extracts.
4. Live external data services.

Sprint 1 data sources

Golden GeoJSON fixtures

Use existing files in demo/geel/ as the first source of truth:

• area_geel_center.geojson
• reference_buildings.geojson
• demo_detections.geojson
• expected_qaqc_metrics.json

These are not meant to be geographically complete. They are contract fixtures.

User upload/register

Support GeoJSON first. Validate:

• file extension;
• JSON parse;
• FeatureCollection shape;
• geometry existence;
• CRS handling fallback;
• bounds calculation.

GRB strategy

GRB is the professional reference layer for Vlaanderen. It is the preferred future reference for buildings and related vector features.

Sprint 1

• Do not implement live GRB WFS yet.
• Use GRB-like local fixture data.
• Design a ReferenceDataAdapter interface so GRB can be added without rewriting QA/QC.

Sprint 2+

• Add GRB adapter.
• Prefer bbox/area-scoped retrieval.
• Cache retrieved features in PostGIS.
• Track source, retrieval timestamp and layer name.

Fallback

If GRB is unavailable:

• use cached extract;
• show external-source-unavailable status;
• do not fake live data.

OSM strategy

OSM can be useful as a broad fallback/reference but must not replace GRB for professional building QA where GRB is available.

Sprint 1

• No live OSM required.
• Keep adapter boundary ready.

Later

• Use Overpass or local extracts for bounded areas.
• Cache in PostGIS.

Sentinel strategy

Sentinel belongs to the Remote Sensing Lab, not Sprint 1.

Sprint 1

• No Sentinel automation.
• Do not add Copernicus dependencies.

Later

• Prefer STAC-based lookup where possible.
• Support NDVI/NDWI/NDBI through raster pipeline.
• Cache downloaded scenes/derived rasters.

DHMV / height data strategy

DHMV/DEM/DSM belongs after raster/vector foundations are stable.

Sprint 1

• No height integration.

Later

• Add DEM/DSM products as raster datasets.
• Reuse raster metadata, clipping and tiling pipelines.

Data acquisition acceptance criteria

A new data source is accepted only when it has:

• adapter boundary;
• source metadata;
• cache strategy;
• error handling;
• tests or fixture equivalent;
• documentation update.