32 lines
2.0 KiB
Markdown
32 lines
2.0 KiB
Markdown
# Capability Gateway
|
|
|
|
## M9 routes
|
|
|
|
The gateway exposes model-neutral `document.ocr@1`, `vision.embedding@1`, and `speech.transcription@1` routes. Each has modality-specific validation, scoped service-client authorization and transient content handling. The stable text embedding route remains exactly 1024-dimensional; opaque LAB experiment routes may return a different declared `EmbeddingSpace` dimension without weakening the stable contract.
|
|
|
|
## Background consumers
|
|
|
|
Service clients carry a fixed workload priority used on Gateway request records, GPU leases and
|
|
serving jobs. ExampleRAG shadow work uses `background`; callers cannot supply or escalate this field.
|
|
Inputs remain digest-only in durable records and transient in Redis.
|
|
|
|
The ModelForge Gateway is the only project-facing inference boundary. A client requests the
|
|
versioned `rag.embedding@1` contract; it cannot provide an artifact path, runtime adapter or
|
|
concrete upstream model. Resolution is capability → stable CapabilityDeployment → RuntimeProfile
|
|
→ exact ArtifactSet → eligible ComputeNode.
|
|
|
|
The native endpoint is `POST /api/v1/capabilities/rag.embedding@1/invoke`. The supported
|
|
OpenAI-compatible subset is `POST /v1/embeddings` with logical model alias `rag.embedding`.
|
|
Authentication, scope/rate checks, bounded queueing, a database-backed GPU lease, residency and
|
|
the outbound worker job all execute before a response is returned. Typed failures never fall back
|
|
to cloud inference.
|
|
|
|
M10 places the shared cross-capability scheduler between authenticated resolution and the worker
|
|
job. A caller still selects only a capability; node, accelerator, resident eviction and priority are
|
|
server-authoritative. Capability health remains distinct from temporary scheduling readiness such
|
|
as `READY_RESIDENT`, `READY_ON_DEMAND`, `WAITING_FOR_CAPACITY`, `POLICY_BLOCKED`,
|
|
`NODE_UNAVAILABLE`, and `RUNTIME_UNHEALTHY`.
|
|
|
|
Request bodies live only in Redis with a short TTL while queued/executing. PostgreSQL retains a
|
|
request ID, input hash, byte/count metadata, timings and failure class, never the input text.
|