112 lines
4.8 KiB
Markdown
112 lines
4.8 KiB
Markdown
# Integrating an application with ModelForge
|
|
|
|
The contract an application depends on is a **capability**, never a model, a node, a file path or a
|
|
Hugging Face repository name. That is the whole design: what serves `rag.embedding@1` can be
|
|
replaced without the caller knowing, and the caller cannot accidentally couple to which model
|
|
answered.
|
|
|
|
## What your application needs to know
|
|
|
|
Three things, and nothing else:
|
|
|
|
```text
|
|
base URL https://modelforge.example.internal:8000
|
|
credential the bearer token issued for your service client
|
|
capability key rag.embedding@1
|
|
```
|
|
|
|
If your integration needs a model name, a revision, a node hostname or an artifact path, something
|
|
has gone wrong — those are ModelForge's business, not yours.
|
|
|
|
## Getting a credential
|
|
|
|
An operator creates a service client scoped to your project and the capabilities you may call:
|
|
|
|
```bash
|
|
curl -s -X POST -H "X-ModelForge-Admin-Token: $KEY" -H "Content-Type: application/json" \
|
|
-d '{"name":"examplerag-production",
|
|
"allowed_capabilities":["rag.embedding@1"],
|
|
"project_key":"examplerag",
|
|
"integration_environment":"production",
|
|
"purpose":"ExampleRAG production embedding"}' \
|
|
http://127.0.0.1:8000/api/v1/admin/service-clients
|
|
```
|
|
|
|
The credential is returned **once**. Only a SHA-256 hash is stored, with a short prefix for
|
|
identification, so it cannot be read back — rotate if you lose it.
|
|
|
|
`integration_environment` is one of `production`, `shadow`, `evaluation` or `lab`.
|
|
|
|
## Calling a capability
|
|
|
|
```http
|
|
POST /api/v1/capabilities/rag.embedding@1/invoke
|
|
Authorization: Bearer mf_...
|
|
Content-Type: application/json
|
|
|
|
{"input": ["first text", "second text"]}
|
|
```
|
|
|
|
A successful response carries the result and an `execution` block describing what actually happened
|
|
— queue time, inference time, whether the deployment was already warm. Use it for your own
|
|
observability; do not branch on it.
|
|
|
|
## Errors your integration must handle
|
|
|
|
| Status | Code | Meaning | What to do |
|
|
| --- | --- | --- | --- |
|
|
| 401 | `CAPABILITY_NOT_AUTHORIZED` | Missing, invalid, revoked, expired or out-of-scope credential | Do not retry; get a valid credential |
|
|
| 429 | rate limited | Your client's requests-per-minute or concurrency was exceeded | Back off |
|
|
| 503 | `NO_ELIGIBLE_NODE` | No online node can serve this capability | Retry with backoff; this is usually transient |
|
|
| 503 | `CAPACITY_CONSTRAINED` | A node exists but has no headroom | Retry with backoff; **this is a correct refusal, not an outage** |
|
|
| 422 | validation | Your request body does not match the contract | Fix the request |
|
|
|
|
Every error response is exactly `code`, `message` and `correlation_id`. Log the correlation id — it
|
|
is how an operator finds your request in the control-plane logs.
|
|
|
|
## Scope is exact
|
|
|
|
- `rag.embedding@1` does not grant `rag.embedding@2`;
|
|
- a Vision client cannot reach ASR, and an ASR client cannot reach Vision;
|
|
- case changes, appended whitespace, a second capability in the same string and zero-width
|
|
characters do not widen a scope;
|
|
- a project-bound client cannot serve another project's binding;
|
|
- a deprecated binding stops serving immediately.
|
|
|
|
## Contract versions and migration
|
|
|
|
A capability contract version fixes the input and output schema. A change that keeps them compatible
|
|
happens behind the same version. A change that does not gets a new version, and your application
|
|
moves when it is ready.
|
|
|
|
The case that needs your attention is `REQUIRES_REINDEX`: the embedding space changed, so vectors
|
|
produced by the old deployment are not comparable with the new one. ModelForge will not swap an
|
|
alias underneath data that has not been reindexed, and your project binding has to declare that it
|
|
supports reindex migration before such a contract can be bound. See
|
|
[MIGRATION_RECOVERY.md](architecture/MIGRATION_RECOVERY.md).
|
|
|
|
## Batching and limits
|
|
|
|
| Limit | Default | Setting |
|
|
| --- | --- | --- |
|
|
| Inputs per invocation | 8 | `MODELFORGE_GATEWAY_MAX_BATCH_SIZE` |
|
|
| Characters per input | 8192 | `MODELFORGE_GATEWAY_MAX_INPUT_CHARACTERS` |
|
|
| Request body | 64 KiB | `MODELFORGE_GATEWAY_MAX_PAYLOAD_BYTES` |
|
|
| Total request time | 45 s | `MODELFORGE_GATEWAY_REQUEST_TIMEOUT_SECONDS` |
|
|
| Queue wait before rejection | 30 s | `MODELFORGE_GATEWAY_QUEUE_TIMEOUT_SECONDS` |
|
|
|
|
## Idempotency
|
|
|
|
Supply an idempotency key when a retry must not produce a second execution. Keys are unique in the
|
|
database rather than in a queue, so a retry storm converges on one row regardless of which component
|
|
retried — 24 concurrent requests on one key produce exactly one.
|
|
|
|
## What ModelForge will never do to your application
|
|
|
|
- serve a model you did not approve through the lifecycle;
|
|
- swap an embedding space underneath you without a reindex;
|
|
- fall back to a different artifact source when the recorded one is unavailable;
|
|
- return an answer built on telemetry it knows is stale.
|
|
|
|
Each of those is a refusal with a name instead.
|