# Recovery asset classification

Recovery starts with an inventory, not with a backup job. Every persisted state source in
ModelForge is classified once, and the classification decides whether the state is copied,
rebuilt, discarded, or explicitly disowned.

## Classes

```text
AUTHORITATIVE   exists nowhere else; must be copied and provably restorable
REBUILDABLE     reproducible from an exact identity; a manifest is enough
EPHEMERAL       describes the present, not history; must be re-measured after recovery
EXTERNAL        owned by another project; ModelForge records the dependency and never writes
SECRET          credential material; hashes restore, plaintext does not
```

A class is not a label on a document: `RecoveryPolicyRevision` refuses incoherent combinations.
Authoritative state cannot declare `NOT_BACKED_UP` or `MANIFEST_ONLY`, and cannot omit an RPO.
Ephemeral state cannot claim a payload backup. External state must be marked as an external
dependency. Secret state must name a secret recovery class. Only rebuildable state may be
rehydrated instead of copied.

## The inventory

`recovery_asset_records` holds one row per state source with its owner, location, backup method,
restore method, rebuild method, RPO, dependencies and readiness. The seeded inventory is:

| Asset | Class | Backup | Recovery |
| --- | --- | --- | --- |
| `postgres.modelforge` | AUTHORITATIVE | `pg_dump --format=custom` | `pg_restore` into a fresh database |
| `artifacts.derived` | AUTHORITATIVE | file copy | payload restore preserving derived lineage |
| `artifacts.huggingface` | REBUILDABLE | manifest only | exact-revision redownload, quarantine, verify, promote |
| `configuration.repository` | REBUILDABLE | source-control reference | checkout of the recorded commit |
| `backup.destination` | REBUILDABLE | source-control reference | operator-provided storage |
| `redis.queues` | EPHEMERAL | not backed up | queues start empty; leases reconciled |
| `runtime.residency` | EPHEMERAL | not backed up | cleared on restore, re-measured from live GPU Node state |
| `node.gpu_node` | SECRET | hashes only | restored hash stays valid, or revoke and re-enrol |
| `credentials.project` | SECRET | hashes only | rotate; plaintext is unrecoverable |
| `credentials.operator` | SECRET | never in a backup | operator-supplied break-glass |
| `external.examplerag` | EXTERNAL | not backed up | owned by ExampleRAG |
| `external.examplevision` | EXTERNAL | not backed up | owned by ExampleVision |
| `external.huggingface` | EXTERNAL | not backed up | upstream availability only |
| `external.gitea` | EXTERNAL | not backed up | platform operations |

## History versus current truth

The same distinction runs through the database itself. `recovery_fingerprint.py` partitions every
persisted table into fingerprinted history and `CURRENT_TRUTH_TABLES`, and a test asserts the two
sets together cover the entire schema — a new milestone cannot add a table that is silently
neither.

Current truth is `host_telemetry_latest`, `accelerator_telemetry_latest`, `storage_volume_states`,
`hardware_inventory_runs`, `scheduler_accelerator_states`, `residency_allocations`,
`serving_gpu_leases` and `gpu_leases`. These are cleared during recovery reconciliation and
re-derived by a running control plane. Restoring them would present a stale NVML reading or a dead
GPU lease as a present-day fact, and the restore gate refuses to reach `READY` while any of them
still holds a row.

## Readiness

Each asset reports one of `PROTECTED`, `REHYDRATABLE`, `ROTATION_REQUIRED`,
`EXTERNAL_DEPENDENCY` or `UNPROTECTED`. Authoritative assets are reported `UNPROTECTED` whenever no
verified backup exists, regardless of what the policy claims: readiness is measured, not declared.

See `docs/architecture/BACKUP_AND_RECOVERY.md` for the backup contract and ADR-0041.