Integrating an application with ModelForge

The contract an application depends on is a capability, never a model, a node, a file path or a Hugging Face repository name. That is the whole design: what serves rag.embedding@1 can be replaced without the caller knowing, and the caller cannot accidentally couple to which model answered.

What your application needs to know

Three things, and nothing else:

base URL          https://modelforge.example.internal:8000
credential        the bearer token issued for your service client
capability key    rag.embedding@1

If your integration needs a model name, a revision, a node hostname or an artifact path, something has gone wrong — those are ModelForge's business, not yours.

Getting a credential

An operator creates a service client scoped to your project and the capabilities you may call:

curl -s -X POST -H "X-ModelForge-Admin-Token: $KEY" -H "Content-Type: application/json" \
  -d '{"name":"examplerag-production",
       "allowed_capabilities":["rag.embedding@1"],
       "project_key":"examplerag",
       "integration_environment":"production",
       "purpose":"ExampleRAG production embedding"}' \
  http://127.0.0.1:8000/api/v1/admin/service-clients

The credential is returned once. Only a SHA-256 hash is stored, with a short prefix for identification, so it cannot be read back — rotate if you lose it.

integration_environment is one of production, shadow, evaluation or lab.

Calling a capability

POST /api/v1/capabilities/rag.embedding@1/invoke
Authorization: Bearer mf_...
Content-Type: application/json

{"input": ["first text", "second text"]}

A successful response carries the result and an execution block describing what actually happened — queue time, inference time, whether the deployment was already warm. Use it for your own observability; do not branch on it.

Errors your integration must handle

Status	Code	Meaning	What to do
401	`CAPABILITY_NOT_AUTHORIZED`	Missing, invalid, revoked, expired or out-of-scope credential	Do not retry; get a valid credential
429	rate limited	Your client's requests-per-minute or concurrency was exceeded	Back off
503	`NO_ELIGIBLE_NODE`	No online node can serve this capability	Retry with backoff; this is usually transient
503	`CAPACITY_CONSTRAINED`	A node exists but has no headroom	Retry with backoff; this is a correct refusal, not an outage
422	validation	Your request body does not match the contract	Fix the request

Every error response is exactly code, message and correlation_id. Log the correlation id — it is how an operator finds your request in the control-plane logs.

Scope is exact

rag.embedding@1 does not grant rag.embedding@2;
a Vision client cannot reach ASR, and an ASR client cannot reach Vision;
case changes, appended whitespace, a second capability in the same string and zero-width characters do not widen a scope;
a project-bound client cannot serve another project's binding;
a deprecated binding stops serving immediately.

Contract versions and migration

A capability contract version fixes the input and output schema. A change that keeps them compatible happens behind the same version. A change that does not gets a new version, and your application moves when it is ready.

The case that needs your attention is REQUIRES_REINDEX: the embedding space changed, so vectors produced by the old deployment are not comparable with the new one. ModelForge will not swap an alias underneath data that has not been reindexed, and your project binding has to declare that it supports reindex migration before such a contract can be bound. See MIGRATION_RECOVERY.md.

Batching and limits

Limit	Default	Setting
Inputs per invocation	8	`MODELFORGE_GATEWAY_MAX_BATCH_SIZE`
Characters per input	8192	`MODELFORGE_GATEWAY_MAX_INPUT_CHARACTERS`
Request body	64 KiB	`MODELFORGE_GATEWAY_MAX_PAYLOAD_BYTES`
Total request time	45 s	`MODELFORGE_GATEWAY_REQUEST_TIMEOUT_SECONDS`
Queue wait before rejection	30 s	`MODELFORGE_GATEWAY_QUEUE_TIMEOUT_SECONDS`

Idempotency

Supply an idempotency key when a retry must not produce a second execution. Keys are unique in the database rather than in a queue, so a retry storm converges on one row regardless of which component retried — 24 concurrent requests on one key produce exactly one.

What ModelForge will never do to your application

serve a model you did not approve through the lifecycle;
swap an embedding space underneath you without a reindex;
fall back to a different artifact source when the recorded one is unavailable;
return an answer built on telemetry it knows is stale.

Each of those is a refusal with a name instead.

4.8 KiB Raw Blame History