135 lines
6.6 KiB
Markdown
135 lines
6.6 KiB
Markdown
# ITWorx ModelForge v1.0.0
|
|
|
|
The first stable release. ModelForge is a self-hosted control plane for running AI models on your
|
|
own hardware: it decides what may run, where it runs, and what your applications are allowed to
|
|
depend on.
|
|
|
|
## What it does
|
|
|
|
**Capability-first routing.** Applications bind to a capability — `rag.embedding@1` — and never to a
|
|
model, a node, a file path or a Hugging Face repository name. What serves a capability can be
|
|
replaced without any consumer knowing, and no consumer can accidentally couple to which model
|
|
answered.
|
|
|
|
**A model lifecycle that refuses to guess.** Discover, acquire into quarantine, verify with a
|
|
per-file SHA-256, evaluate, approve, promote, canary, roll back, deprecate. `trust_remote_code` is
|
|
never enabled, and production refuses to start if it is. No production deployment exists without a
|
|
recorded production approval.
|
|
|
|
**GPU scheduling that admits what it does not know.** The scheduler places work against measured
|
|
capacity and refuses when there is no headroom rather than risking an out-of-memory failure that
|
|
would take down work already running. VRAM held by processes ModelForge does not manage counts as
|
|
external pressure and is never reclaimed. Stale telemetry blocks admission instead of being
|
|
extrapolated.
|
|
|
|
**Outbound-only compute nodes.** A node reaches the control plane; the control plane never dials a
|
|
node. Enrolment is single use and atomically claimed.
|
|
|
|
**Embedding-space migration that will not corrupt a corpus.** A contract that changes the embedding
|
|
space is `REQUIRES_REINDEX`, and the migration engine will not swap an alias underneath data that
|
|
has not been reindexed.
|
|
|
|
**Backup and recovery with measured RPO.** Encrypted backups with immutable manifests, verified
|
|
restore, artifact rehydration, node re-enrolment and control-plane rebuild.
|
|
|
|
**Operational observability.** SLOs, alerts with real state transitions, capacity history, and a
|
|
health model in which monitoring is never a safety dependency.
|
|
|
|
## System requirements
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| Docker Engine | 24+ |
|
|
| Docker Compose | 2+ |
|
|
| PostgreSQL | 16+ (17 ships with the deployment) |
|
|
| Redis | 7+ |
|
|
| Disk | 50 GiB, plus room for model artifacts |
|
|
| GPU | Required on compute nodes only, with the NVIDIA container runtime |
|
|
|
|
## Installing
|
|
|
|
[docs/INSTALLATION.md](INSTALLATION.md). A verified clean install took 124 seconds from
|
|
`up -d --build` to a healthy API and console.
|
|
|
|
The production deployment is one command, and it is not the same as the development one:
|
|
|
|
```bash
|
|
docker compose -f docker-compose.yml -f docker-compose.production.yml up -d --build
|
|
```
|
|
|
|
## Upgrading
|
|
|
|
Supported from `m16-baseline`. [docs/UPGRADE.md](UPGRADE.md).
|
|
|
|
**v1.0.0 introduces no database migration.** The schema head is unchanged at `20260827_0021`, so
|
|
rolling back needs no database restore — verified, with the previous application version healthy
|
|
against the same database in 19 seconds.
|
|
|
|
## Security posture
|
|
|
|
- Datastores bind to loopback by default; only the API is published, and every admin route is
|
|
operator-authenticated.
|
|
- Production fails closed at startup on a missing or weak operator key, a development database
|
|
password, remote code execution, a missing backup encryption key, a wildcard CORS origin, an
|
|
unwritable storage root, or an unsupported schema revision.
|
|
- The console ships a Content-Security-Policy with `default-src 'none'`, no `unsafe-inline` or
|
|
`unsafe-eval` in the script directive, and inline style permitted only as an attribute — plus six
|
|
further security headers on the entry document and on hashed assets.
|
|
- All images run unprivileged, drop all capabilities and set `no-new-privileges`. No container
|
|
mounts the Docker socket. The console and Node Agent run read-only.
|
|
- Backups are AES-256-GCM with a fresh nonce per chunk and fail-closed decryption.
|
|
- Credentials are stored as SHA-256 hashes with a bounded prefix and cannot be read back.
|
|
|
|
## Known limitations
|
|
|
|
These are real and are stated rather than discovered later.
|
|
|
|
- **Point-in-time recovery is `NOT_SUPPORTED`.** Verified snapshot restore with a measured RPO is
|
|
what exists; continuous WAL recovery is out of scope for v1.
|
|
- **Upgrades require downtime.** The control plane is stopped while the new version starts. Rolling
|
|
upgrade is not supported and is not claimed.
|
|
- **No console workspace for Model Lab or Benchmarks.** Both have platform support; neither has an
|
|
operator view in v1.
|
|
- **The capacity-rejection and PostgreSQL deadlock-retry paths are covered by deterministic tests,
|
|
not by production load.** No soak reached the concurrency that would exercise them naturally.
|
|
- **Storage and inode exhaustion are covered by isolated tests, not by live injection**, deliberately:
|
|
filling a real volume to prove a guard is not a reasonable thing to do to a production host.
|
|
|
|
## Deferred project validation
|
|
|
|
Not ModelForge platform defects — external work that has not been completed:
|
|
|
|
- ExampleVision Vision owner-photo validation;
|
|
- OCR has no production candidate;
|
|
- ASR natural-speech validation;
|
|
- ExampleRAG production smoke, pending that service running.
|
|
|
|
## Documentation
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| [INSTALLATION.md](INSTALLATION.md) | Clean install |
|
|
| [FIRST_RUN.md](FIRST_RUN.md) | From a running control plane to a serving capability |
|
|
| [UPGRADE.md](UPGRADE.md) | Upgrading and rolling back |
|
|
| [CONFIGURATION.md](CONFIGURATION.md) | Every setting, generated from the code |
|
|
| [COMPATIBILITY.md](COMPATIBILITY.md) | Version, schema and protocol ranges |
|
|
| [NODE_AGENT.md](NODE_AGENT.md) | Enrolling and running a compute node |
|
|
| [UNRAID_DEPLOYMENT.md](UNRAID_DEPLOYMENT.md) | The Unraid deployment specifically |
|
|
| [CAPABILITIES.md](CAPABILITIES.md) | What a capability is and why |
|
|
| [PROJECT_INTEGRATION.md](PROJECT_INTEGRATION.md) | Integrating an application |
|
|
| [OPERATIONS.md](OPERATIONS.md) | Runbooks, health model, backups |
|
|
| [TROUBLESHOOTING.md](TROUBLESHOOTING.md) | The states you will actually meet |
|
|
| [SECURITY.md](SECURITY.md) | Threat model and security posture |
|
|
|
|
## Upgrade notes for existing M16 deployments
|
|
|
|
1. Adopting the production profile requires secrets M16 did not: an operator API key of at least 32
|
|
characters, a backup encryption key, a non-development database password, and an exact CORS
|
|
origin. The deployment refuses to render without them.
|
|
2. The console no longer displays development milestone labels.
|
|
3. `GET /api/v1/system` no longer returns `milestone`; it returns `release_channel`.
|
|
4. `GET /api/v1/version` is new and unauthenticated.
|
|
5. Host ports are configurable without editing a Compose file.
|
|
6. If you build images yourself, pass `MODELFORGE_VERSION`, `MODELFORGE_COMMIT` and
|
|
`MODELFORGE_BUILT_AT` so the running container can report where it came from.
|