# GPU Scheduler Design

## M6 reindex QoS

The database claim order remains production, interactive, background, benchmark. ExampleRAG shadow
jobs are stamped `background` from their scoped identity. External Ollama/Tdarr VRAM remains
observed capacity pressure and is never managed by ModelForge.

## M10 operational objective

Admit capability requests on one finite GPU without letting ModelForge or experimental work disrupt
production or unmanaged workloads.

## Priority classes

1. production
2. interactive
3. background
4. benchmark
5. maintenance

Priority is not identical to preemption support. Some runtimes/jobs cannot be safely interrupted and must be drained instead.

## Residency policies

- `always_warm`
- `keep_warm` with TTL
- `load_on_demand`
- `exclusive` and `lab_only` remain future policy values, not M5 serving choices.

## Resource leases

Inference begins only when the scheduler issues a lease for required accelerator resources.

Lease evaluation considers:

- target accelerator compatibility;
- latest total NVML usage, including unmanaged processes;
- reserved safety margin;
- deployment resource envelope;
- current leases;
- queue priority;
- model load/sleep/unload cost;
- production eligibility and exact measured compatibility.

M5 never silently falls back to CPU, another model, a workstation or cloud inference.

The pre-M10 budget was:

```text
total VRAM - external usage - ModelForge residency - active leases - 1 GiB reserve
```

Rows are aggregated once per accelerator. M10's authoritative invariant, dynamic reserve,
multi-resident placement, hysteresis and typed failure behavior are specified in
`docs/architecture/ADVANCED_GPU_SCHEDULER.md`. GPU Node is eligible; the DESKTOP workstation is explicitly
rejected because `production_eligible=false`. The accelerator row is locked while a lease is granted.

## Resource envelope

Memory is not a single static number. Record measured profiles across:

- context length;
- concurrency;
- batch size;
- idle vs peak VRAM;
- runtime version/configuration.

## M5 embedding decision

```text
Request: rag.embedding@1
Deployment: stable and cold
Envelope: 1,216,348,160 process-reserved bytes
Policy: KEEP_WARM 901s, concurrency 1, queue 16

Decision: lock GPU Node GPU, account unmanaged VRAM and reserve, grant a lease, load once, run a
functional health check, invoke, release the request lease and retain residency.
```

## Failure classes

- MODEL_LOAD_FAILED
- GPU_OOM
- RUNTIME_CRASH
- TIMEOUT
- HEALTHCHECK_FAILED
- INVALID_OUTPUT
- ARTIFACT_CORRUPT
- CAPABILITY_UNAVAILABLE
- DRIVER_ERROR
- SCHEDULER_STATE_STALE
- INSUFFICIENT_SCHEDULABLE_VRAM
- EXTERNAL_GPU_PRESSURE
- RESIDENCY_CONFLICT
- EVICTION_NOT_ALLOWED
- DEADLINE_CANNOT_BE_MET
- LEASE_CONFLICT
- GPU_MEMORY_NOT_RECLAIMED
- QUEUE_FULL
- LEASE_TIMEOUT

Failures must produce structured events. No silent retry loop may hide prolonged capability degradation.

## M14 operational telemetry

The Operations plane samples the existing NVML/scheduler truth once per minute. A retained snapshot
separates total, NVML-observed, ModelForge resident, active leased, reserve, external and schedulable
bytes and carries observed/received freshness. `GPU_PRESSURE` uses the existing hysteresis state;
`CAPABILITY_CAPACITY` compares a measured production ResourceEnvelope with known headroom. Neither
alert authorizes control of Ollama, Plex, Tdarr or another external GPU process. See
`docs/architecture/CAPACITY_TRENDS.md` and `docs/operations/RUNBOOK_GPU_PRESSURE.md`.

Runtime adapters own model load, crash and invalid-output detection. The scheduler owns VRAM
reservation, OOM recovery and lease release. The gateway owns bounded retries and only contract-
approved fallback/degradation. The control plane blocks corrupt artifacts and records state changes.