# ADR 0022 — Bounded serving poll and opaque experiment routes

Status: accepted

## Context

M6 showed a warm Gateway p50 of 1144.46 ms although the model inference itself was much shorter.
The worker polled for jobs and completions with fixed one-second sleeps, and a project evaluator had
no capability-first way to address a non-production profile without learning a concrete deployment.

## Decision

- Job claims use server-side bounded long polling. The wait is capped at five seconds and the worker
  renews it at one-second intervals, so shutdown and liveness remain responsive.
- Gateway timing is decomposed into validation, resolution, scheduling, payload persistence,
  dispatch, completion transport, result retrieval and worker preprocess/inference/serialization.
  Only bounded summaries are retained; request content is never trace data.
- A LAB_READY candidate may receive an opaque `capability-experiment` route. Authentication remains
  scoped to `rag.embedding@1`; the caller neither supplies nor receives a model path.
- Experiment deployments are non-production, use an execution approval rather than a production
  approval, have their own EmbeddingSpace and cannot change the stable route.
- On a single GPU, the scheduler unloads an inactive resident model before loading another candidate.
  It refuses to switch while requests are active and continues to include unmanaged external VRAM
  in the budget.

## Consequences

Warm serving no longer inherits a polling floor. Candidate evaluation remains capability-first and
auditable, while the one-GPU implementation avoids unsafe simultaneous loads. A candidate route is
not a canary and provides no production-cutover operation.