feat(provenance): govern source snapshots and data inputs
This commit is contained in:
@@ -69,6 +69,41 @@ Spatial index required on `geometry`.
|
||||
- `status text default 'created'`
|
||||
- `created_at timestamptz`
|
||||
|
||||
### Source registry, snapshots and quarantine (Accuracy Phase 2)
|
||||
|
||||
Migration `202608010001_source_registry_provenance` adds a server-owned
|
||||
provenance boundary without deleting or inventing facts for existing rows.
|
||||
|
||||
`source_registry` stores one governed source definition per `source_key`:
|
||||
|
||||
- classification (`authoritative`, `corroborative`, `contextual`, `derived`,
|
||||
`experimental`), authority/scope and provider adapter key;
|
||||
- licence, usage restrictions, expected CRS/units/resolution, temporal and
|
||||
geographic coverage, expected geometry/attributes and known limitations;
|
||||
- usage policy, freshness and ingest status.
|
||||
|
||||
`source_snapshots` stores immutable acquired editions keyed by
|
||||
`(source_registry_id, snapshot_key)`, including source version/snapshot/fetch
|
||||
time, lowercase SHA-256 checksum, CRS/units/resolution, coverage, observed
|
||||
schema, freshness, ingest status and raw snapshot metadata.
|
||||
|
||||
`datasets` and `dataset_versions` receive `source_registry_id`,
|
||||
`source_snapshot_id`, data-contract key/version, validation report/status,
|
||||
provenance/lineage status and idempotent `ingest_key`. New governed records
|
||||
must have a passing report before they become `ready`. Legacy rows are retained
|
||||
with explicit incomplete/not-validated state; a migration never promotes them
|
||||
based on a historical `source_name` string.
|
||||
|
||||
`dataset_lineage_edges` records parent/child Dataset(+Version), transform name
|
||||
and version, parameters and input/output hashes. `dataset_quarantines` retains
|
||||
the rejected dataset/version/snapshot, stage, reason, validation evidence and
|
||||
artifact location. Quarantine is a stateful preservation record, not a delete.
|
||||
|
||||
The database constraints enforce accepted status vocabularies, nonblank ingest
|
||||
keys, immutable snapshot identity/checksum format, unique ingest idempotency
|
||||
keys and non-self lineage edges. Service-level contract validation remains
|
||||
responsible for CRS, topology, bbox, units and source-task semantics.
|
||||
|
||||
### vector_features
|
||||
|
||||
Used for imported vector datasets and derived vector outputs when feature-level storage is needed. Original files remain source artifacts; this table is the queryable PostGIS state for vector features.
|
||||
|
||||
Reference in New Issue
Block a user