67 lines
3.5 KiB
Markdown
67 lines
3.5 KiB
Markdown
# Model Storage
|
|
|
|
## Configuration model
|
|
|
|
Registry storage is explicit and node-scoped. `StorageRoot` references a `ComputeNode`, has a host
|
|
path and purpose, and stores write/capacity observations plus reserve policy. It is intentionally
|
|
separate from `StorageVolumeState`: the latter is agent telemetry, while a storage root is an
|
|
operator-approved placement target.
|
|
|
|
Every placement check is fail-closed:
|
|
|
|
- unknown capacity blocks placement;
|
|
- non-writable roots block placement;
|
|
- usable bytes are `free - max(reserve_bytes, capacity * reserve_percent)`;
|
|
- a request crossing that reserve becomes `capacity_blocked`.
|
|
|
|
No default silently assumes infinite storage.
|
|
|
|
## GPU Node validation — 2026-08-25
|
|
|
|
The Unraid array reported 54,983,962,927,104 bytes total, 53,206,636,449,792 used and
|
|
1,777,326,477,312 available (97% used). It is explicitly unsuitable for model placement.
|
|
|
|
The cache-backed `appdata` filesystem reported approximately 1.438 TB total and 535 GB free (63%
|
|
used). ModelForge created this dedicated root:
|
|
|
|
`/mnt/cache/appdata/modelforge-node-agent/artifacts/model-registry`
|
|
|
|
Ownership is UID/GID 999 with mode 0750. A write/create/remove probe from the existing non-root,
|
|
read-only, capability-dropped node-agent container passed. The registered root is
|
|
`5774fef2-8cd4-44bd-a244-5b5b20c02811` on GPU Node node
|
|
`c3fab19c-d7f7-4dde-80ac-2edf565ad61d`.
|
|
|
|
The stored observation is 1,438,282,285,056 bytes capacity and 534,938,255,360 bytes free, with a
|
|
50 GiB absolute reserve and 10% percentage reserve. Effective zero-request usable capacity was
|
|
391,110,026,855 bytes. The array-unsuitable decision, backing filesystem, write-probe outcome and
|
|
`download_performed=false` are retained in validation details.
|
|
|
|
## M12 cleanup boundary
|
|
|
|
CleanupPlan is a dry-run over node-local ArtifactLocations and all known control-plane dependencies.
|
|
Execution rechecks the dependency digest, requires operator confirmation and records physical removal
|
|
only after the typed storage action succeeds. Removing a GPU Node location cannot mark the artifact
|
|
globally missing while another verified node location exists. Metadata archive and byte removal are
|
|
separate operations; rollback-retained artifacts are not eligible.
|
|
|
|
M2 did not download or move any model artifact. M3 maps the host root to
|
|
`/data/artifacts/model-registry` inside the GPU Node agent. Planning and control-plane execution checks
|
|
use the registered observation; the agent rechecks the actual filesystem immediately before bytes.
|
|
There is no fallback to `/mnt/user` or another storage root.
|
|
|
|
After the M3 acceptance artifact was promoted, GPU Node reported 1,440,503,169,024 bytes capacity and
|
|
536,169,676,800 bytes free. The effective reserve is 144,050,316,902 bytes (10%, stricter than
|
|
50 GiB), leaving 392,119,359,898 bytes usable. Status remains `ready`. The accepted artifact set
|
|
uses 1,207,470,234 bytes plus a 4,256-byte reconciliation manifest.
|
|
|
|
## M15 artifact recovery
|
|
|
|
Artifacts are classified for recovery rather than treated uniformly. A set with an exact upstream
|
|
repository and commit is `REHYDRATABLE` and is protected by a manifest recording every file's
|
|
SHA-256 and size; recovery redownloads that exact revision through quarantine. A set without both is
|
|
`NON_REHYDRATABLE` and needs a payload backup — the platform will not pretend it can be redownloaded.
|
|
|
|
Recovery targets a disposable storage root, never the only usable copy of a production artifact. A
|
|
recovered artifact re-enters the supply chain unapproved. See
|
|
`docs/architecture/ARTIFACT_RECOVERY.md` and `docs/operations/RUNBOOK_ARTIFACT_LOSS.md`.
|