Remote Server Agent Deployment

Prerequisites

The remote server needs Docker Engine with Compose v2, an NVIDIA driver, NVIDIA Container Toolkit, and outbound HTTPS access to the central ModelForge API. It needs no inbound SSH path from ModelForge, no public agent port, no Docker socket mount and no privileged container.

The container deliberately uses a persisted UUID because host /etc/machine-id is not mounted. Hostname/OS/CPU/RAM describe the agent's Linux execution environment; without container resource limits this normally represents the server, while configured CPU/memory limits represent the capacity actually available to future workloads. Storage facts cover only the named ModelForge data volumes. This avoids unsafe host-root mounts and keeps the semantics operationally honest.

The Compose service drops all Linux capabilities, enables no-new-privileges, uses a read-only root filesystem and opens only a bounded in-memory /tmp. It publishes no port. Set MODELFORGE_AGENT_HOSTNAME to the real compute-node hostname. On Unraid or another appliance, set the four MODELFORGE_AGENT_*_VOLUME variables to narrow, dedicated absolute bind paths when state must survive Docker image/volume recreation; never point one at the host root.

Expose the control-plane API through a trusted HTTPS reverse proxy. Use an internal CA by installing that CA in the agent image/host trust store; do not disable TLS verification outside an isolated local smoke test.

For a private CA that should be trusted only by ModelForge, combine the base file with docker-compose.node-agent.private-ca.yml. Set MODELFORGE_AGENT_CA_CERT_PATH to the public CA certificate, plus MODELFORGE_AGENT_CONTROL_PLANE_HOST_ALIAS and MODELFORGE_AGENT_CONTROL_PLANE_HOST_ADDRESS when private DNS is unavailable. The overlay mounts only that public certificate read-only and scopes the host mapping to the agent container.

1. Configure the control plane

Set a high-entropy MODELFORGE_OPERATOR_API_KEY, keep MODELFORGE_HARDWARE_REFRESH_ON_STARTUP=false, and start the base stack. Create an enrollment token from Compute nodes → Enroll node, or:

curl -sS -X POST https://modelforge.example/api/v1/admin/node-enrollments \
  -H "X-ModelForge-Admin-Token: $MODELFORGE_OPERATOR_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"expires_in_seconds":900,"display_name":"server01","role":"primary-inference","labels":{"site":"office"},"production_eligible":false,"lab_eligible":true,"benchmark_eligible":true}'

The response shows the enrollment secret once. Leave production_eligible false until the actual node has passed hardware, identity, recovery and security acceptance. Eligibility is scheduler metadata; setting it true does not launch a runtime or bypass later promotion evidence.

2. Start the agent on the GPU server

Copy/clone this repository to the server. Create .env without committing it:

MODELFORGE_AGENT_CONTROL_PLANE_URL=https://modelforge.example
MODELFORGE_AGENT_ENROLLMENT_TOKEN=<one-time-token>
MODELFORGE_AGENT_DISPLAY_NAME=Server 01
MODELFORGE_AGENT_HOSTNAME=server01
MODELFORGE_AGENT_TLS_VERIFY=true
MODELFORGE_AGENT_STATE_VOLUME=/mnt/user/appdata/modelforge-node-agent/state
MODELFORGE_AGENT_HF_CACHE_VOLUME=/mnt/cache/appdata/modelforge-node-agent/hf-cache
MODELFORGE_AGENT_ARTIFACT_VOLUME=/mnt/cache/appdata/modelforge-node-agent/artifacts
MODELFORGE_AGENT_QUARANTINE_VOLUME=/mnt/cache/appdata/modelforge-node-agent/quarantine
MODELFORGE_HF_TOKEN=<optional-secret-for-gated-repositories>

Then run:

docker compose -f docker-compose.node-agent.yml up -d --build
docker compose -f docker-compose.node-agent.yml logs --tail=100 node-agent

With the private-CA overlay, use both files for every lifecycle command:

docker compose -f docker-compose.node-agent.yml \
  -f docker-compose.node-agent.private-ca.yml up -d --build

After the node appears online, remove MODELFORGE_AGENT_ENROLLMENT_TOKEN from .env and recreate the service. The node-scoped credential persists in node-agent-state:

docker compose -f docker-compose.node-agent.yml up -d --force-recreate node-agent

Do not delete the state volume during ordinary upgrades; doing so discards identity and credential. Use the UI/API to revoke the old credential before deliberately re-enrolling.

For M3, register the host cache path as a storage root and map it to /data/artifacts/model-registry. The host path and container path are deliberately separate facts. Never substitute /mnt/user when the cache capacity guard fails. The agent advertises artifact.acquire.v1, polls one typed job at a time, and retains job-scoped partials below the approved root for bounded retry. Keep the optional Hub token only in secret environment/config; do not place it in Compose files or logs.

3. Verify and operate

Confirm the UI shows the expected hostname, agent/protocol version, heartbeat age, CPU/RAM/storage, GPU UUID/model/VRAM/driver and measured telemetry. Restart the agent and confirm node and accelerator UUIDs remain unchanged. Stop it long enough to observe stale then offline, restart it and confirm online plus a return audit event.

For temporary outages the agent retains its identity/credential and retries with bounded exponential backoff. Rotate with the authenticated admin rotation endpoint, securely replace /data/state/node-credential with its one-time response, and recreate the agent. Revoke without a replacement when a credential may be compromised. Disable a node to make it ineligible and immediately reject further publications. Upgrades are ordinary image rebuild/recreate operations; protocol-incompatible agents are visibly rejected instead of silently downgraded.

RTX 4080 acceptance checklist

On the actual target server, capture: nvidia-smi model/UUID/driver output, node-agent container status, /api/v1/hardware/nodes/{id} output with secrets removed, a restart identity comparison, stale/offline/recovery evidence, and UI screenshots. The release verdict may not claim RTX 4080 validation until these observations exist.

6.0 KiB Raw Permalink Blame History