1.3 KiB
Scheduler QoS
The fixed safety order is production, interactive, background, LAB. Database row locking on the accelerator remains authoritative for capacity, while serving jobs use row-lock/skip-locked claims. Five-minute bounded aging moves old background/LAB work forward without overtaking production.
Backpressure is enforced at three boundaries: per-client concurrency/rate, per-capability queue and
global queue. A full boundary returns QUEUE_FULL; queues never grow without limit. Cold requests
whose measured load time exceeds an explicit remaining deadline return DEADLINE_CANNOT_BE_MET.
Normal scheduling never interrupts active inference. A higher-priority request may evict only an idle, lower/equal-priority managed residency or wait. Batch metadata is represented without making unsafe batching promises. Stored request shape is limited to count, bounded size bucket and modality-relevant numeric characteristics; text, image and audio content is not scheduler evidence.
The admin metrics surface uses bounded names and histories: admissions, rejections, queue seconds, evictions, resident/leased/external/schedulable bytes, pressure, cold loads, warm hits, unloads and reclaim failures. Request IDs are not metric labels and ordinary placements are not audit events.