14 lines
991 B
Markdown
14 lines
991 B
Markdown
# Residency Manager
|
|
|
|
Residency is distinct from a request lease. M10's full multi-resident identity and recovery contract
|
|
is specified in `docs/architecture/MULTI_CAPABILITY_RESIDENCY.md`. The state machine is cold → loading → warm/busy →
|
|
draining/unloading → cold, with explicit failed states. A cold request creates exactly one serialized
|
|
load job; later requests reuse the resident adapter at its measured concurrency of one.
|
|
|
|
Policies are `always_warm`, `keep_warm`, `load_on_demand` and `lab_only`. The production embedding deployment
|
|
uses `keep_warm` with a 901-second TTL. Unload is allowed only without active leases. Model-owned
|
|
VRAM reclaim is measured with the PyTorch CUDA allocator; NVML total memory remains the independent
|
|
external-pressure signal. Workers report the complete deployment inventory and generation.
|
|
Exact immutable identities can be adopted after a control-plane restart; absent identities are
|
|
reconciled cold and unknown GPU processes remain external.
|