20 lines
1.3 KiB
Markdown
20 lines
1.3 KiB
Markdown
# Scheduler QoS
|
|
|
|
The fixed safety order is production, interactive, background, LAB. Database row locking on the
|
|
accelerator remains authoritative for capacity, while serving jobs use row-lock/skip-locked claims.
|
|
Five-minute bounded aging moves old background/LAB work forward without overtaking production.
|
|
|
|
Backpressure is enforced at three boundaries: per-client concurrency/rate, per-capability queue and
|
|
global queue. A full boundary returns `QUEUE_FULL`; queues never grow without limit. Cold requests
|
|
whose measured load time exceeds an explicit remaining deadline return `DEADLINE_CANNOT_BE_MET`.
|
|
|
|
Normal scheduling never interrupts active inference. A higher-priority request may evict only an
|
|
idle, lower/equal-priority managed residency or wait. Batch metadata is represented without making
|
|
unsafe batching promises. Stored request shape is limited to count, bounded size bucket and
|
|
modality-relevant numeric characteristics; text, image and audio content is not scheduler evidence.
|
|
|
|
The admin metrics surface uses bounded names and histories: admissions, rejections, queue seconds,
|
|
evictions, resident/leased/external/schedulable bytes, pressure, cold loads, warm hits, unloads and
|
|
reclaim failures. Request IDs are not metric labels and ordinary placements are not audit events.
|
|
|