25 lines
1.5 KiB
Markdown
25 lines
1.5 KiB
Markdown
# Multi-Capability Residency
|
|
|
|
The GPU Node runtime worker owns a bounded map of resident deployments instead of one implicit slot.
|
|
Load, invoke, health, drain and unload always address a deployment UUID. Loading one deployment no
|
|
longer unloads another when the placement plan proves capacity.
|
|
|
|
States are `COLD`, `LOAD_QUEUED`, `LOADING`, `WARM_IDLE`, `BUSY`, `DRAINING`, `UNLOADING` and
|
|
`FAILED` at the control-plane boundary; the wire protocol uses their lower-case runtime forms.
|
|
Policies are `ALWAYS_WARM`, `KEEP_WARM`, `LOAD_ON_DEMAND` and `LAB_ONLY`.
|
|
|
|
Eviction considers only managed idle residency. Active requests and `ALWAYS_WARM`/pinned models are
|
|
ineligible. Priority and policy precede idle age and measured reload cost. Each selected eviction
|
|
has a typed reason and expected/actual reclaimed byte evidence. A failed reclaim becomes
|
|
`GPU_MEMORY_NOT_RECLAIMED` and stays conservative.
|
|
|
|
The worker reports a full resident inventory with deployment, ArtifactSet, RuntimeProfile, runtime
|
|
environment fingerprint, upstream revision and worker generation. Startup reconciliation adopts a
|
|
resident model only when immutable identity and node ownership match exactly. Missing inventory is
|
|
reconciled cold. Any other GPU process remains unmanaged external usage.
|
|
|
|
Co-residency evidence is pairwise and reports `PROVEN_SAFE`, `EXPECTED_SAFE`, `NOT_SAFE` or
|
|
`UNKNOWN`, with individual and combined envelope bytes. `PROVEN_SAFE` is reserved for measured live
|
|
evidence; arithmetic alone is never promoted beyond `EXPECTED_SAFE`.
|
|
|