ITWorx ModelForge

ModelForge is a self-hosted control plane for running AI models on hardware you control. Applications ask for a stable capability such as rag.embedding@1; ModelForge selects an approved model revision, places it on an eligible compute node, and retains the evidence needed to explain and roll back that decision.

Current release: 1.2.1

What you get

A web console for models, hardware, capabilities, deployments, incidents and recovery.
A FastAPI control plane with capability-scoped service credentials.
Outbound-only node agents: compute nodes connect to the control plane and need no inbound agent port.
Quarantined model acquisition with immutable revisions, SHA-256 evidence and static inspection.
GPU-aware placement, leases, load-on-demand residency and typed runtime workers.
Approval-gated promotion, canary and rollback workflows with an append-only audit chain.
Backup, restore rehearsal, observability and operational runbooks.

ModelForge is designed for a small self-hosted fleet. It uses Docker Compose rather than Kubernetes, and a GPU is needed only on nodes that serve GPU-backed capabilities.

Quick start

Requirements: Docker Engine 24+, Docker Compose 2+, Python 3.12 for the host-side tools, and at least 50 GiB free plus room for model weights.

python scripts/preflight.py
cp .env.example .env
# Generate and enter the distinct secrets described in docs/INSTALLATION.md.
docker compose -f docker-compose.yml -f docker-compose.production.yml up -d --build

Then verify the control plane and open the console:

curl http://127.0.0.1:8000/api/v1/health/live
curl http://127.0.0.1:8000/api/v1/health/ready

Console: http://127.0.0.1:3000
API documentation: http://127.0.0.1:8000/docs

docker-compose.yml by itself is for development. The production overlay requires separate bootstrap, migration-owner and runtime database credentials and applies the production startup guards. Follow the complete installation guide before exposing an instance.

First useful workflow

Enrol a compute node with a short-lived, single-use token.
Discover a model and resolve it to an immutable upstream commit.
Approve a download plan; the node agent quarantines, verifies and atomically promotes the files.
Register a versioned capability contract and promote an evidence-backed deployment.
Give an application a credential scoped only to the capability it needs.

The application calls a stable endpoint such as:

curl -X POST \
  -H "Authorization: Bearer $MODELFORGE_CLIENT_CREDENTIAL" \
  -H "Content-Type: application/json" \
  -d '{"input":["text to embed"]}' \
  http://127.0.0.1:8000/api/v1/capabilities/rag.embedding@1/invoke

The caller does not need to know the model name, artifact path, runtime or GPU node. See First run and Project integration.

Architecture

Applications -> Capability Gateway -> Scheduler -> typed runtime workers
                       |                 ^
                       v                 |
                 Control Plane <--- outbound node agents
                       |
             PostgreSQL + Redis + verified artifacts

Important boundaries:

exact model commits and container digests replace mutable names;
trust_remote_code remains disabled;
production changes require evidence and explicit approval;
compute agents publish observations but cannot act as operators;
runtime workers have no external network access by default;
destructive and recovery actions are planned, bounded and audited.

The system architecture, threat model and model supply-chain policy describe these guarantees in detail.

Documentation

Goal	Guide
Install or upgrade	Installation · Upgrade
Enrol a compute node	Node Agent
Configure the platform	Configuration
Integrate an application	Project integration
Operate and recover it	Operations · Troubleshooting
Review security	Security · Threat model
Understand compatibility	Compatibility
See release changes	Changelog · 1.2.1 release notes

Repository layout

backend/          FastAPI control plane, persistence and migrations
frontend/         React operator console
node-agent/       Outbound compute-node observer and artifact acquirer
runtime-worker/   Isolated typed model runtime
config/           Example capability, policy and model manifests
docs/             User, architecture, operations and security documentation
scripts/          Preflight, bootstrap, release and recovery tools

Development

Each component keeps its own pinned dependencies and tests. The protected-branch validation installs all four components, runs linting, strict type checks, tests and the frontend build, verifies Compose projections, scans for secrets, and audits Python and production npm dependencies.

python -m pytest backend/tests -q
python -m pytest node-agent/tests -q
python -m pytest runtime-worker/tests -q
cd frontend && npm ci && npm test -- --run && npm run build

See CONTRIBUTING.md before proposing a change and SECURITY.md for private vulnerability reporting.

License status

ModelForge is licensed under GNU AGPL-3.0-or-later. If you run a modified version for users over a network, the AGPL requires that those users can obtain the corresponding source. See LICENSE. Downloaded models, model weights and datasets retain their own upstream license terms and are not relicensed by ModelForge.

5.9 KiB Raw Permalink Blame History