feat(provenance): govern source snapshots and data inputs

This commit is contained in:
Jens
2026-08-01 23:46:17 +02:00
parent cebeb5f3b4
commit 5b3c17b494
96 changed files with 20156 additions and 351 deletions
+35
View File
@@ -69,6 +69,41 @@ Spatial index required on `geometry`.
- `status text default 'created'`
- `created_at timestamptz`
### Source registry, snapshots and quarantine (Accuracy Phase 2)
Migration `202608010001_source_registry_provenance` adds a server-owned
provenance boundary without deleting or inventing facts for existing rows.
`source_registry` stores one governed source definition per `source_key`:
- classification (`authoritative`, `corroborative`, `contextual`, `derived`,
`experimental`), authority/scope and provider adapter key;
- licence, usage restrictions, expected CRS/units/resolution, temporal and
geographic coverage, expected geometry/attributes and known limitations;
- usage policy, freshness and ingest status.
`source_snapshots` stores immutable acquired editions keyed by
`(source_registry_id, snapshot_key)`, including source version/snapshot/fetch
time, lowercase SHA-256 checksum, CRS/units/resolution, coverage, observed
schema, freshness, ingest status and raw snapshot metadata.
`datasets` and `dataset_versions` receive `source_registry_id`,
`source_snapshot_id`, data-contract key/version, validation report/status,
provenance/lineage status and idempotent `ingest_key`. New governed records
must have a passing report before they become `ready`. Legacy rows are retained
with explicit incomplete/not-validated state; a migration never promotes them
based on a historical `source_name` string.
`dataset_lineage_edges` records parent/child Dataset(+Version), transform name
and version, parameters and input/output hashes. `dataset_quarantines` retains
the rejected dataset/version/snapshot, stage, reason, validation evidence and
artifact location. Quarantine is a stateful preservation record, not a delete.
The database constraints enforce accepted status vocabularies, nonblank ingest
keys, immutable snapshot identity/checksum format, unique ingest idempotency
keys and non-self lineage edges. Service-level contract validation remains
responsible for CRS, topology, bbox, units and source-task semantics.
### vector_features
Used for imported vector datasets and derived vector outputs when feature-level storage is needed. Original files remain source artifacts; this table is the queryable PostGIS state for vector features.