Initial public ModelForge release

docs/architecture/adr/0001-capability-first-routing.md

@@ -0,0 +1,24 @@
+# ADR-0001: Capability-first routing
+
+**Status:** Accepted
+
+## Context
+
+Directly binding projects to model IDs creates pervasive coupling. Model upgrades then require application code/config changes and encourage accidental use of incompatible model revisions.
+
+## Decision
+
+Projects bind to versioned logical capabilities. The ModelForge gateway resolves the capability, project identity, channel and policy to a concrete deployment.
+
+## Consequences
+
+Positive:
+
+- model upgrades become infrastructure lifecycle operations;
+- project code remains stable;
+- benchmark and migration policy can attach to capabilities;
+- fallbacks can be expressed centrally.
+
+Tradeoff:
+
+- ModelForge must define precise capability contracts rather than a generic pass-through API only.

docs/architecture/adr/0002-separate-model-artifact-deployment.md

@@ -0,0 +1,11 @@
+# ADR-0002: Separate model, artifact and deployment concepts
+
+**Status:** Accepted
+
+## Decision
+
+Represent upstream logical models, immutable revisions, concrete/downloaded artifacts, derived artifacts, runtime profiles and deployments as distinct entities.
+
+## Rationale
+
+Performance and compatibility depend on much more than a model name. Quantization, runtime, context settings, launch parameters and exact revision can materially change memory use, speed and output.

docs/architecture/adr/0003-single-gpu-first.md

@@ -0,0 +1,15 @@
+# ADR-0003: Single-GPU first, multi-node ready
+
+**Status:** Accepted
+
+## Decision
+
+Build the initial scheduler for one local NVIDIA GPU using Docker Compose and NVML, while keeping `ComputeNode` and `Accelerator` abstractions in the domain model.
+
+## Rejected initially
+
+- Kubernetes
+- distributed inference scheduling
+- multi-tenant cloud architecture
+
+These add operational cost without solving the first product problem.

docs/architecture/adr/0004-embedding-upgrades-require-migration.md

@@ -0,0 +1,18 @@
+# ADR-0004: Embedding-space changes require explicit migration
+
+**Status:** Accepted
+
+## Decision
+
+A deployment change for a vector-producing capability is `requires_reindex` unless compatibility has been explicitly proven by the capability contract.
+
+## Required migration shape
+
+1. create a shadow index/version;
+2. backfill using the candidate embedding deployment;
+3. validate corpus and counts;
+4. execute project-specific retrieval benchmarks/shadow queries;
+5. promote the new index atomically;
+6. retain old index and deployment for rollback until retention expires.
+
+Matching vector dimension alone is never evidence of semantic compatibility.

docs/architecture/adr/0005-validated-manifests-and-relational-foundation.md

@@ -0,0 +1,26 @@
+# ADR-0005: Validated manifests and relational M0 foundation
+
+**Status:** Accepted
+
+## Context
+
+The starter duplicated candidate metadata in Python and YAML and had no persistent schema. Design
+invariants therefore could not be checked before M1/M2 work.
+
+## Decision
+
+Version-controlled YAML is the read-only M0 seed source and is parsed through strict Pydantic schemas.
+PostgreSQL is the future operational source of truth using distinct SQLAlchemy entities and an Alembic
+baseline. API routes consume a registry service rather than raw YAML or module constants. Audit events
+are append-only and hash-chain ready. Canonical endpoints use `/api/v1`.
+
+GPU allocation is an optional Compose overlay; the base development stack is CPU-compatible. Redis is
+profiled until a queue-owning milestone needs it.
+
+## Consequences
+
+- Invalid capability/project/security manifests fail readiness and tests.
+- The starter no longer presents seed metadata as a deployment.
+- M1 can persist hardware without reshaping model lifecycle entities.
+- Later schema changes require explicit Alembic operations; the metadata-driven creation is restricted
+  to the reviewed initial baseline revision.

docs/architecture/adr/0006-stable-hardware-identity-and-observation-semantics.md

@@ -0,0 +1,23 @@
+# ADR-0006: Stable hardware identity and typed observations
+
+**Status:** Accepted
+
+## Context
+
+Hostnames, GPU indexes and current telemetry are not stable identities. Containers can also expose an
+image-local machine ID that is identical on different hosts. Missing metrics cannot safely be stored
+as zero or ambiguous nulls.
+
+## Decision
+
+Direct nodes derive a privacy-preserving UUIDv5 from the OS machine ID; container deployments use a
+persisted UUID volume or explicit configured identity. GPU UUID is the accelerator identity within a
+node. Optional observations pair nullable values with `known`, `unknown`, `unsupported`, `unavailable`
+or `temporarily_failed`. Inventory and latest telemetry persist separately. Reconciliation preserves
+temporarily absent hardware.
+
+## Consequences
+
+Repeated scans are idempotent, container restarts preserve node identity, and absence is not confused
+with a measured zero. Moving the same GPU between nodes is represented honestly as a different node-
+accelerator relation. M1 stores only latest telemetry; history/export belongs to observability work.

docs/architecture/adr/0007-outbound-node-agent.md

@@ -0,0 +1,29 @@
+# ADR-0007: Outbound authenticated compute-node agent
+
+- Status: accepted
+- Date: 2026-08-25
+
+## Context
+
+The central control plane must manage hardware on separate GPU servers without conflating the API
+host with the compute node. SSH orchestration, Docker socket exposure and inbound node services widen
+the trust boundary and do not provide a stable reporting protocol.
+
+## Decision
+
+Run a lightweight unprivileged agent on each compute node. It initiates HTTPS requests to a
+versioned central API, reports typed inventory/telemetry and heartbeats, and persists its identity,
+credential and stream sequences. Enrollment secrets are random, short-lived, single-use and stored
+only as SHA-256 hashes. Issued credentials are random, node-scoped, hash-only at rest, revocable and
+never returned by list APIs. Administrative onboarding uses a separately configured operator secret.
+
+The control plane uses server-received heartbeat time for liveness and retains agent observation time
+for provenance. Node roles, labels and eligibility are operator-managed metadata; inventory never
+grants production eligibility automatically.
+
+## Consequences
+
+The compute server needs only outbound API reachability. Lost agent state requires explicit
+revocation/re-enrollment. HTTPS termination and CA trust are deployment responsibilities. Protocol
+evolution must remain explicit and test older/replayed/incompatible reports. Runtime execution is
+intentionally absent until M5/M6 define adapters and scheduling.

docs/architecture/adr/0008-content-addressed-artifacts-and-node-storage.md

@@ -0,0 +1,31 @@
+# ADR-0008: Separate content identity from node-scoped storage location
+
+## Status
+
+Accepted — 2026-08-25
+
+## Context
+
+The M0 schema stored one `storage_uri` directly on an artifact and one source directly on a derived
+artifact. Operational registry use requires replicas, missing locations, remote-only metadata and
+transformations with multiple inputs. Treating a path as identity would make moves look like new
+artifacts and would weaken provenance.
+
+## Decision
+
+Artifact content identity is the exact revision plus SHA-256. `ArtifactLocation` associates content
+with a node-scoped `StorageRoot` and relative path. A derived output has ordered N-source lineage with
+captured source hashes. Storage capacity is observed and placement fails closed when unknown or when
+reserve policy would be crossed.
+
+Legacy `storage_uri` and primary-source columns remain nullable migration compatibility fields in M2;
+new services never use them as identity.
+
+## Consequences
+
+- replicas and moves do not change content identity;
+- missing/corrupt state is location-aware and auditable;
+- delete checks can name exact provenance/location dependencies;
+- remote compute storage can be configured without exposing workers publicly;
+- later download/runtime milestones can consume this boundary without redesigning the registry;
+- the additional joins are deliberate and preferable to provenance ambiguity.

docs/architecture/adr/0009-node-local-artifact-acquisition.md

@@ -0,0 +1,17 @@
+# ADR-0009: Node-local artifact acquisition
+
+## Status
+
+Accepted — 2026-08-25
+
+## Decision
+
+The control plane owns Hub metadata, immutable plans and job state. The selected compute node's
+outbound agent downloads and verifies bytes against its locally mounted approved storage root.
+Artifact operations are a closed typed protocol, not shell/SSH/command execution.
+
+## Consequences
+
+Capacity and path safety are evaluated where the bytes live, GPU Node needs no inbound service, and
+credentials/jobs remain node-scoped. SSH remains deployment/debug tooling outside the ModelForge
+runtime protocol.

docs/architecture/adr/0010-immutable-download-plans.md

@@ -0,0 +1,17 @@
+# ADR-0010: Immutable, explicitly approved download plans
+
+## Status
+
+Accepted — 2026-08-25
+
+## Decision
+
+A download is derived from an immutable exact commit and selected file inventory. Its canonical
+payload, target, expected bytes/checksums, preflights, expiry and idempotency hash cannot change.
+Plan creation and approval are separate actions. Mutable refs and stale/unapproved plans cannot
+execute.
+
+## Consequences
+
+Operators review the precise transfer before bytes move; retries reuse the same intent; upstream
+movement requires a new revision and plan; duplicate execution resolves to one job.

docs/architecture/adr/0011-quarantine-atomic-promotion.md

@@ -0,0 +1,18 @@
+# ADR-0011: Set-level quarantine and atomic promotion
+
+## Status
+
+Accepted — 2026-08-25
+
+## Decision
+
+All required files for an artifact set are downloaded to a job-scoped directory inside the target
+filesystem. Static verification precedes one atomic directory rename to the exact-revision store.
+The database registers artifacts only after that promotion report and records static integrity as
+distinct from later security/runtime approval.
+
+## Consequences
+
+Partial sets cannot appear locally verified, cross-filesystem partial moves are avoided, retries
+can reuse durable partials, and a sidecar manifest supports reconciliation if the file move and DB
+transaction are interrupted.

docs/architecture/adr/0012-typed-static-runtime-worker.md

@@ -0,0 +1,15 @@
+# ADR-0012: Typed static runtime worker boundary
+
+Status: accepted (M4)
+
+## Decision
+
+Run heavy ML dependencies in a separately built, static Compose worker. It polls outbound with a
+node credential, accepts only the `runtime.probe.v1` schema, mounts exact artifacts read-only and
+uses a child process per probe. Do not expose Docker, shell, SSH, generic Python or public runtime
+control surfaces.
+
+## Consequences
+
+Runtime images are larger but independently pinned. Node Agent remains lightweight. Adding an
+adapter requires a typed implementation and image revision; remote payloads cannot invent launches.

docs/architecture/adr/0013-exact-artifact-lab-execution-approval.md

@@ -0,0 +1,14 @@
+# ADR-0013: Exact-ArtifactSet LAB execution approval
+
+Status: accepted (M4)
+
+## Decision
+
+Separate integrity verification from permission to execute. Store LAB approval against one exact
+ArtifactSet and fingerprint the evidence reviewed. Expiry, revocation or evidence changes invalidate
+it. Do not infer production approval.
+
+## Consequences
+
+Operators make the first code/data execution boundary explicit and auditable. New revisions and
+artifact variants require separate review even when the model name is unchanged.

docs/architecture/adr/0014-static-versus-proven-runtime-support.md

@@ -0,0 +1,14 @@
+# ADR-0014: Static versus locally proven runtime support
+
+Status: accepted (M4)
+
+## Decision
+
+Represent static compatibility and successful local execution as different evidence. Runtime
+declarations can establish format/architecture expectations; only a completed probe can establish
+local support for an exact environment/profile/node/artifact fingerprint.
+
+## Consequences
+
+Unknown or incomplete evidence remains `unknown`/`requires_probe`; no optimistic compatibility
+fallback exists. Driver, image, runtime, artifact or hardware changes make evidence stale.

docs/architecture/adr/0015-capability-first-serving-gateway.md

@@ -0,0 +1,8 @@
+# ADR-0015 — Capability-first serving gateway
+
+Status: accepted (M5)
+
+Projects call a central authenticated Gateway with a versioned capability. Concrete model/runtime
+selection is server-side. This preserves upgrade isolation, prevents public worker exposure and
+provides one QoS, audit and privacy boundary. No cloud or direct-worker fallback is allowed.
+

docs/architecture/adr/0016-database-backed-gpu-lease-coordination.md

@@ -0,0 +1,8 @@
+# ADR-0016 — Database-backed GPU lease coordination
+
+Status: accepted (M5)
+
+GPU admission uses persisted leases and a locked accelerator row. Correctness therefore does not
+depend on one API process or an in-memory mutex. Server timestamps, expiry and reconciliation make
+crashed-request reservations recoverable.
+

docs/architecture/adr/0017-residency-load-on-demand.md

@@ -0,0 +1,8 @@
+# ADR-0017 — Explicit load-on-demand residency
+
+Status: accepted (M5)
+
+Long-lived residency and short request leases are separate objects. A typed outbound worker owns one
+resident deployment slot, serializes transitions and reuses it within its measured envelope.
+Keep-warm limits load thrashing; controlled drain/unload measures allocator-owned reclaim.
+

docs/architecture/adr/0018-embedding-space-identity.md

@@ -0,0 +1,8 @@
+# ADR-0018 — Embedding-space identity is independent of dimension
+
+Status: accepted (M5)
+
+Embedding identity hashes exact artifacts, runtime profile and capability contract. Equal vector
+dimension is insufficient evidence of compatibility. Every identity change is `requires_reindex`
+and is exposed in native responses.
+

docs/architecture/adr/0019-shadow-embedding-migrations.md

@@ -0,0 +1,8 @@
+# ADR 0019 — Shadow embedding migrations never mutate production aliases
+
+Status: accepted
+
+Embedding changes always build a distinct collection with explicit `embedding_space_id`. Progress
+is checkpointed after complete Qdrant batches and validated from canonical chunk fingerprints.
+M6 exposes cancel/resume/evaluate but no cutover. This prevents dimension equality, partial writes
+or evaluation success from silently becoming production state.

docs/architecture/adr/0020-immutable-evaluation-suite-revisions.md

@@ -0,0 +1,7 @@
+# ADR 0020 — Evaluation suite revisions are immutable
+
+Status: accepted
+
+Queries, relevance labels/grades, critical flags, metrics, top-K and retrieval configuration form
+one content-digested revision. Runs reference that revision permanently. A change creates a new
+revision, preserving reproducibility and preventing retrospective baseline changes.

docs/architecture/adr/0021-project-specific-evidence-hierarchy.md

@@ -0,0 +1,8 @@
+# ADR 0021 — Project-specific evidence precedes general model claims
+
+Status: accepted
+
+Evidence is ordered project → suite revision → corpus → retrieval configuration → target embedding
+space/deployment → environment. ModelForge may report ExampleRAG eligibility from comparable local
+runs, but M6 makes no global “best model” recommendation. A later Advisor may consume this evidence
+without weakening its provenance hierarchy.

docs/architecture/adr/0022-bounded-serving-poll-and-experiment-routes.md

@@ -0,0 +1,30 @@
+# ADR 0022 — Bounded serving poll and opaque experiment routes
+
+Status: accepted
+
+## Context
+
+M6 showed a warm Gateway p50 of 1144.46 ms although the model inference itself was much shorter.
+The worker polled for jobs and completions with fixed one-second sleeps, and a project evaluator had
+no capability-first way to address a non-production profile without learning a concrete deployment.
+
+## Decision
+
+- Job claims use server-side bounded long polling. The wait is capped at five seconds and the worker
+  renews it at one-second intervals, so shutdown and liveness remain responsive.
+- Gateway timing is decomposed into validation, resolution, scheduling, payload persistence,
+  dispatch, completion transport, result retrieval and worker preprocess/inference/serialization.
+  Only bounded summaries are retained; request content is never trace data.
+- A LAB_READY candidate may receive an opaque `capability-experiment` route. Authentication remains
+  scoped to `rag.embedding@1`; the caller neither supplies nor receives a model path.
+- Experiment deployments are non-production, use an execution approval rather than a production
+  approval, have their own EmbeddingSpace and cannot change the stable route.
+- On a single GPU, the scheduler unloads an inactive resident model before loading another candidate.
+  It refuses to switch while requests are active and continues to include unmanaged external VRAM
+  in the budget.
+
+## Consequences
+
+Warm serving no longer inherits a polling floor. Candidate evaluation remains capability-first and
+auditable, while the one-GPU implementation avoids unsafe simultaneous loads. A candidate route is
+not a canary and provides no production-cutover operation.

docs/architecture/adr/0023-deterministic-advisor-and-raw-evidence-matrix.md

@@ -0,0 +1,31 @@
+# ADR 0023 — Deterministic Advisor and raw evidence matrix
+
+Status: accepted
+
+## Decision
+
+Model comparisons are first-class immutable evidence snapshots. Each evaluated row references an
+existing EvaluationRun; quality metrics and critical-case outcomes are materialized from that run
+and its pairwise comparison rather than accepted as caller claims. Operational, resource, migration,
+security and provenance evidence remain separate fields. Blocked and unknown candidates remain
+visible as blocked or unknown and never receive fabricated metrics.
+
+Advisor recommendations are deterministic. The persisted policy requires project-local evidence
+class A for a high-confidence recommendation, treats any critical regression as a hard block,
+requires verified supply chain and runtime fit, and requires non-negative quality deltas. A p95
+increase above 100% is an operational review ceiling, not a weighted quality score. Policy versions
+and rationale are persisted with each recommendation.
+
+The only verdicts are `KEEP_CURRENT`, `PROMOTION_ELIGIBLE`, `PROMOTION_NOT_RECOMMENDED` and
+`REQUIRES_MORE_EVIDENCE`. There is deliberately no automatic approval or promotion endpoint.
+
+## Evidence hierarchy
+
+1. A — local project evaluation on the pinned suite/corpus/configuration.
+2. B — local generic benchmark or measured preflight.
+3. C — independent external benchmark.
+4. D — upstream model card or vendor claim.
+5. E — downloads, likes or trending data.
+
+Lower evidence cannot overrule project-local critical regressions without a new explicit policy and
+new recommendation fingerprint.

docs/architecture/adr/0024-canonical-gitea-baseline.md

@@ -0,0 +1,18 @@
+# ADR-0024: Canonical Git and private Gitea baseline
+
+## Status
+
+Accepted — 2026-08-25.
+
+## Decision
+
+The first honest repository commit captures the already verified M7 disk state. No synthetic
+milestone history is created. `Jens/ITWorx-ModelForge` on private ITWorx Gitea is canonical;
+`master` is the default branch and `m7-baseline` identifies the immutable pre-M8 governance commit.
+Force-push and branch deletion are blocked while non-force direct pushes remain compatible with the
+existing autonomous workflow.
+
+## Consequences
+
+Baseline provenance and release gates are recorded separately from later logical M8 commits.
+Secrets, runtime state and model artifacts remain excluded from history.

docs/architecture/adr/0025-frozen-retrieval-candidate-pools.md

@@ -0,0 +1,16 @@
+# ADR-0025: Freeze first-stage candidate pools for reranking comparisons
+
+## Status
+
+Accepted — 2026-08-25.
+
+## Decision
+
+Control and reranked results must consume the same immutable ordered candidate pool, identified by
+suite case, embedding source, corpus and retrieval configuration. Persist IDs, scores and content
+hashes, never text. Top 40 is a ceiling, not permission to pad an in-scope pool.
+
+## Consequences
+
+Reranker gains cannot be caused by retrieval drift. A reranker cannot claim recovery of an item not
+present in the pool, and mismatched pools are non-comparable.

docs/architecture/adr/0026-first-class-retrieval-pipeline-identity.md

@@ -0,0 +1,16 @@
+# ADR-0026: First-class retrieval pipeline identity
+
+## Status
+
+Accepted — 2026-08-25.
+
+## Decision
+
+Identity covers embedding space, sparse/fusion digests, optional reranker deployment/configuration
+and candidate/output K. Embedding-only and reranked configurations are distinct immutable domain
+objects.
+
+## Consequences
+
+Advisor and UI can compare pipeline behavior without conflating it with an embedding deployment.
+Reranker addition is behavioral; embedding replacement requires reindexing.

docs/architecture/adr/0027-deterministic-pipeline-advisor.md

@@ -0,0 +1,16 @@
+# ADR-0027: Deterministic Advisor recommendations for retrieval pipelines
+
+## Status
+
+Accepted — 2026-08-25.
+
+## Decision
+
+Advisor derives pipeline metrics from completed fixed-pool runs and applies the same hard gates as
+embedding recommendations. Typed target kind, verdict, evidence level, deltas, migration impact and
+blockers are persisted. No weighted popularity score or automatic action is allowed.
+
+## Consequences
+
+`KEEP_CURRENT` is a successful evidence result. A future reranker recommendation can be expressed
+without implying an embedding-space change or reindex.

docs/architecture/adr/0028-typed-multimodal-payloads-and-evaluation.md

@@ -0,0 +1,16 @@
+# ADR 0028: Typed multimodal payloads and evaluation families
+
+- Status: accepted
+- Date: 2026-08-26
+
+## Context
+
+Embedding-only text contracts cannot safely represent images, documents or audio. Reusing one generic score would also make unlike capability quality incomparable and encourage accidental promotion.
+
+## Decision
+
+Each modality receives a versioned capability contract, bounded input/output models, a specialized offline runtime adapter where needed, and its own typed evaluation family. Binary content crosses the gateway only through a transient payload store and never becomes durable control-plane content. Visual embedding spaces include preprocessing identity. Resource-class labels are advisory; measured byte envelopes drive admission.
+
+## Consequences
+
+Consumers remain model-agnostic and workers remain inaccessible. A new capability requires explicit validation and evaluation work, but it can be added without redesigning the gateway, scheduler, provenance model or advisor. Cross-family scores and silent visual-vector reuse are rejected by construction.

docs/architecture/adr/0029-nvml-truth-and-multi-resident-placement.md

@@ -0,0 +1,25 @@
+# ADR 0029 — NVML truth and multi-resident placement
+
+Status: accepted
+
+## Context
+
+The M5–M9 scheduler safely serialized one resident deployment, but that prevented proven small
+capabilities from coexisting. Naively subtracting both NVML-used and managed residency would double
+count memory; trusting absent telemetry as zero would overcommit the GPU.
+
+## Decision
+
+NVML used is physical truth. Managed residency is attribution within that observation, and only
+unmaterialized lease bytes are additionally reserved. Unknown/stale telemetry fails closed. A
+single versioned planning function drives dry-run and actual admission under a database accelerator
+lock. The runtime worker may retain multiple exact deployment identities. Eviction is limited to
+managed idle residency and recovery adopts only an exact immutable identity report.
+
+## Consequences
+
+GPU Node can serve multiple capabilities when current observations and measured envelopes prove fit,
+without weakening the 1 GiB minimum reserve. External workloads remain visible but unmanaged.
+Conservative rejections may reduce utilization during telemetry uncertainty or pressure; that is an
+intentional safety result, not a capability-health failure.
+

docs/architecture/adr/0030-operational-project-bindings-and-fit-evidence.md

@@ -0,0 +1,20 @@
+# ADR-0030: Operational project bindings and immutable project-fit evidence
+
+## Status
+
+Accepted in M11.
+
+## Decision
+
+An external consumer is operational only through a scoped service identity bound to one real
+`ProjectBinding`. Project environment, purpose and server-owned QoS are attributes of that identity.
+Project-fit conclusions are separate immutable evidence records bound to the project contract and,
+when applicable, the exact deployment.
+
+## Consequences
+
+Planning manifests cannot masquerade as live usage. Consumers cannot select a model, runtime, worker,
+node or priority. Credential rotation preserves binding identity while revoking the old secret.
+Project-fit evidence can explain readiness but cannot promote. Usage and “Why Installed” can be
+derived from domain relations without storing content or adding project-specific gateway routes.
+

docs/architecture/adr/0031-deferred-owner-photo-validation-and-evidence-classes.md

@@ -0,0 +1,64 @@
+# ADR-0031: Deferred owner-photo validation and visual evidence classes
+
+## Status
+
+Accepted in M11 on 2026-08-26.
+
+## Context
+
+M11 proved ExampleVision's capability-first shadow integration, complete reference index, scoped
+credentials, privacy boundary, Gateway/Scheduler/GPU Node execution and bounded QoS. It did not receive
+an independently reviewed owner-photo dataset. Treating that external data dependency as an
+unbounded engineering blocker conflated integration readiness with project production fitness.
+
+Catalog artwork cannot prove camera robustness. Public physical captures can improve robustness
+evidence, but their licenses, provenance and labels vary and they cannot substitute for ExampleVision
+production captures when calibrating automatic decisions.
+
+## Decision
+
+Visual project evidence has one immutable class:
+
+- `OWNER_PHOTO`: physical-card captures from the target user/project context, independently labelled;
+- `PUBLIC_PHYSICAL_CAPTURE`: licensed, provenance-reviewed physical-card captures from a public
+  source;
+- `CATALOG_REFERENCE`: official/reference artwork or derivatives used only for index, embedding and
+  technical route validation.
+
+`ProjectFitEvidence` also records separate immutable statements for engineering integration,
+project-production recommendation, production validation and production action. Engineering may be
+`PASS` while project production fit remains `REQUIRES_MORE_EVIDENCE` and production action remains
+`NONE`.
+
+M11 engineering closure does not require owner photos. The separate
+`POKEVAULT_VISION_PRODUCTION_VALIDATION` gate does. It can be `SATISFIED` only by non-empty,
+independently reviewed `OWNER_PHOTO` evidence with zero critical errors and blockers and an eligible
+project-fit recommendation. Public physical captures and catalog references are rejected as
+production-gate substitutes.
+
+The M11 release taxonomy is:
+
+- `PASS`;
+- `PASS WITH DEFERRED EXTERNAL VALIDATION`;
+- `PASS WITH DOCUMENTED CANDIDATE LIMITATIONS`;
+- `BLOCKED`.
+
+M11 uses `PASS WITH DEFERRED EXTERNAL VALIDATION` while owner-photo validation is absent.
+
+## Consequences
+
+- M11 can close after its engineering gates pass without claiming real-photo accuracy.
+- `VISION_AUTO_ACCEPT=false` and ExampleVision human confirmation remain mandatory.
+- Existing Vision evidence remains immutable and `REQUIRES_MORE_EVIDENCE`.
+- No public dataset result can activate automatic recognition or satisfy production validation.
+- A later production proposal must bind owner-photo dataset revision, independent ground truth,
+  leakage checks, critical/false-confident results, frozen calibration and explicit approval.
+- Advisor and UI expose engineering, project-fit, production-gate and action state separately.
+
+## Rejected alternatives
+
+- Keeping M11 indefinitely blocked: this makes an external manual data-collection activity part of
+  engineering completion despite a proven shadow integration.
+- Treating catalog artwork as real-photo evidence: this hides the camera-domain gap.
+- Treating public captures as owner photos: this loses project provenance and calibration relevance.
+- Enabling auto-accept from public data alone: this bypasses the project-specific production gate.

docs/architecture/adr/0032-evidence-bound-production-approvals.md

@@ -0,0 +1,31 @@
+# ADR-0032: Evidence-bound production approvals
+
+## Status
+
+Accepted in M12 on 2026-08-26.
+
+## Context
+
+Technical probes, Advisor recommendations and engineering integration each prove useful but distinct
+facts. Reusing their labels as production permission would silently reinterpret stale evidence and
+collapse platform readiness into project fitness.
+
+## Decision
+
+Production permission is a first-class ApprovalRequest bound to an immutable ApprovalPolicy revision
+and a fingerprinted snapshot of exact declared IDs plus database-resolved facts. Policies are
+revisioned. Requests expire, can be revoked and become stale when referenced facts change. Default
+production policy hard-gates security, license, evaluation, resource readiness, rollback and project
+validation. Approval remains separate from execution.
+
+## Consequences
+
+M11's engineering PASS can coexist with a blocked ExampleVision production request. Old approvals retain
+their interpretation and cannot silently authorize changed artifacts, runtimes, fit or scheduler
+evidence. More records and explicit operator steps are required, but every permission is explainable.
+
+## Rejected alternatives
+
+- Treating `LAB_READY` or `PROMOTION_ELIGIBLE` as permission: conflates evidence with authority.
+- Storing only a model/capability name: loses exact provenance.
+- Re-evaluating old requests under the latest policy: silently changes historical meaning.

docs/architecture/adr/0033-promotion-plans-and-rollback-snapshots.md

@@ -0,0 +1,30 @@
+# ADR-0033: Promotion plans and rollback snapshots
+
+## Status
+
+Accepted in M12 on 2026-08-26.
+
+## Context
+
+Approval alone does not describe blast radius, canary semantics, resource placement, migration or
+restoration. A single-click approval-and-swap cannot be reconciled safely after races or restart.
+
+## Decision
+
+An approved request permits an immutable PromotionPlan. Execution journals a separate idempotent
+LifecycleOperation and captures an immutable rollback snapshot before activation. Subject versions
+provide compare-and-swap protection, and the database enforces single production-stable deployment.
+Successful canary requires a second explicit review; failures and incomplete restart stages roll back
+conservatively.
+
+## Consequences
+
+Requester, approver and executor are auditable even for one owner. Plan changes create new plans.
+Failed rollback becomes manual intervention, never false stable. Embedding reindex and schema-breaking
+plans are representable but not auto-executed by M12.
+
+## Rejected alternatives
+
+- Mutating deployment rows directly: loses intention and recovery state.
+- Automatic promotion after canary: removes the explicit production decision.
+- Reconstructing rollback from current rows: current truth may already be partially changed.

docs/architecture/adr/0034-retention-and-dependency-aware-cleanup.md

@@ -0,0 +1,28 @@
+# ADR-0034: Retention and dependency-aware cleanup
+
+## Status
+
+Accepted in M12 on 2026-08-26.
+
+## Context
+
+Unused-looking model bytes may still back a deployment, rollback, evaluation, project, migration or
+another node. Immediate deletion after promotion destroys recoverability and provenance.
+
+## Decision
+
+RetentionPolicy is revisioned; prior stable targets receive a minimum rollback window. Deprecation,
+archive and physical removal are separate. Cleanup always begins with a dependency-digest dry-run,
+requires explicit operator execution and rechecks dependencies transactionally. Location removal is
+node-specific and records prior facts; metadata/provenance remain.
+
+## Consequences
+
+Storage recovery may be delayed by valid dependencies or retention. Cleanup races return stale/409
+instead of deleting. Multiple node copies are reconciled independently and audit remains queryable.
+
+## Rejected alternatives
+
+- Age-only automatic cleanup: cannot see active or rollback dependencies.
+- Cascading deletes: erases evidence and makes blast radius implicit.
+- Treating one removed location as global deletion: is incorrect in a multi-node registry.

docs/architecture/adr/0035-typed-migration-adapters-and-immutable-plans.md

@@ -0,0 +1,16 @@
+# ADR-0035: Typed migration adapters and immutable plans
+
+## Status
+
+Accepted in M13 on 2026-08-26.
+
+## Decision
+
+ModelForge stores immutable, evidence-bound migration intent and accepts results only for a closed,
+fingerprinted adapter operation set. Project data planes implement concrete transformations. No plan
+contains executable SQL, shell or Python.
+
+## Consequences
+
+Adding a backend requires trusted adapter code and tests. Core stays storage-neutral, source/target
+semantics remain reproducible and capability clients cannot obtain migration authority.

docs/architecture/adr/0036-external-truth-cutover-and-reconciliation.md

@@ -0,0 +1,16 @@
+# ADR-0036: External-truth cutover and reconciliation
+
+## Status
+
+Accepted in M13 on 2026-08-26.
+
+## Decision
+
+Cutover uses a durable operation journal, immediate CAS/source check and adapter-owned atomic
+primitive. Database commit follows verified external identity and health. Startup reconciliation
+requires measured external truth and chooses commit, rollback or manual intervention conservatively.
+
+## Consequences
+
+ModelForge does not pretend to provide a distributed transaction. Crash windows are explicit,
+idempotent and auditable; ambiguous state cannot silently become production.

docs/architecture/adr/0037-migration-validation-and-rollback-retention.md

@@ -0,0 +1,16 @@
+# ADR-0037: Migration validation and rollback retention
+
+## Status
+
+Accepted in M13 on 2026-08-26.
+
+## Decision
+
+Cutover requires an immutable validation snapshot. Technical LAB eligibility and project-production
+eligibility are separate. The exact prior external identity is snapshotted before switch and remains
+cleanup-blocking through the configured rollback window.
+
+## Consequences
+
+Completeness alone cannot promote a model. Isolated mechanics can be rehearsed despite documented
+project evidence gaps, without implying production fitness or sacrificing rollback evidence.

docs/architecture/adr/0038-slo-and-error-budget-model.md

@@ -0,0 +1,17 @@
+# ADR-0038: Versioned SLO and error-budget model
+
+## Status
+
+Accepted in M14 on 2026-08-27.
+
+## Decision
+
+Separate immutable SLI definitions from versioned SLO policies and immutable evaluation snapshots.
+Only production policies receive error budgets. Minimum population and freshness are evaluated
+before health; short and rolling-window burn rates remain distinct.
+
+## Consequences
+
+Threshold changes cannot rewrite historical meaning. LAB evidence cannot masquerade as production
+reliability. A serving capability can consume budget without being misreported as unavailable, and
+too little evidence remains explicitly `INSUFFICIENT_DATA`.

docs/architecture/adr/0039-deduplicated-alert-lifecycle.md

@@ -0,0 +1,17 @@
+# ADR-0039: Deduplicated alert lifecycle
+
+## Status
+
+Accepted in M14 on 2026-08-27.
+
+## Decision
+
+Use immutable versioned rules and deterministic logical fingerprints with pending, firing,
+acknowledged, resolved and maintenance-suppressed states. Every transition is durable; M14 performs
+no automatic destructive remediation.
+
+## Consequences
+
+Polling does not produce alert spam, acknowledgement cannot erase evidence, recovery stays visible
+and maintenance cannot silently disable monitoring. Lightweight incidents claim causality only for
+evidenced node/downstream relationships.

docs/architecture/adr/0040-bounded-capacity-telemetry-retention.md

@@ -0,0 +1,17 @@
+# ADR-0040: Bounded capacity telemetry retention
+
+## Status
+
+Accepted in M14 on 2026-08-27.
+
+## Decision
+
+Sample the existing authoritative hardware/scheduler state at minute granularity, retain raw
+snapshots for seven days and hourly aggregates for 90 days. Forecast only with at least 12 samples
+spanning one hour. Monitoring persistence is not a serving dependency.
+
+## Consequences
+
+ModelForge gains explainable GPU/storage trends without a second NVML collector or an unbounded
+time-series table. Sparse/stale history is reported honestly, and external workloads remain observed
+but unmanaged.

docs/architecture/adr/0041-authoritative-versus-rebuildable-recovery-state.md

@@ -0,0 +1,31 @@
+# ADR-0041: Authoritative versus rebuildable recovery state
+
+## Status
+
+Accepted in M15 on 2026-08-27.
+
+## Decision
+
+Classify every persisted state source as `AUTHORITATIVE`, `REBUILDABLE`, `EPHEMERAL`, `EXTERNAL` or
+`SECRET` in a versioned `RecoveryPolicyRevision`, and enforce the classification in the contract:
+authoritative state may not declare a manifest-only or absent backup and must state an RPO,
+ephemeral state may not claim a payload backup, external state must be marked as a dependency,
+secret state must name a secret recovery class, and only rebuildable state may be rehydrated.
+
+Partition the database itself along the same line. Fingerprinted history is compared across a
+restore; `CURRENT_TRUTH_TABLES` — NVML telemetry, GPU leases, residency, inventory runs, storage
+volume states — is cleared during reconciliation and re-measured by a running control plane. A test
+asserts the two sets together cover the whole schema.
+
+## Consequences
+
+Recovery cost is proportional to what is genuinely irreplaceable: a 5 MB encrypted control-plane
+dump protects the provenance, lifecycle, migration, approval and audit truth, while 8.1 GB of model
+weights are protected by exact-revision manifests instead of being copied.
+
+A restored control plane cannot present a stale GPU lease or a dead NVML reading as present-day
+fact, and the `READY` gate refuses to pass while any current-truth row survives. External projects
+are recorded as dependencies rather than silently claimed as backed up.
+
+The cost is that a new milestone must classify every table it adds. The invariant test makes that a
+build failure rather than a silent recovery gap.

docs/architecture/adr/0042-immutable-verified-backup-sets.md

@@ -0,0 +1,35 @@
+# ADR-0042: Immutable verified backup sets
+
+## Status
+
+Accepted in M15 on 2026-08-27.
+
+## Decision
+
+Make `BackupSet` immutable recovery identity, sealed by `immutable_at` once written, carrying the
+ModelForge version and commit, the canonical repository and reference, the Alembic revision, the
+PostgreSQL server identity and tool version, the destination, the hashed manifest and payload, the
+included and excluded asset classes and the encryption identity.
+
+Treat verification, not creation, as the gate. Only `VERIFIED` is restore eligible, and verification
+checks the manifest hash, manifest-to-journal agreement, the presence of a database payload, every
+object's size and hash at an allowlisted path, decryption against the recorded plaintext hash, and
+archive readability. Anything else fails closed with a typed code.
+
+Protect the last known-good backups from retention unconditionally, and reconcile any backup
+interrupted mid-write to `FAILED`.
+
+## Consequences
+
+A backup cannot look usable because a file exists. A single flipped byte in a 5.4 MB encrypted
+payload, a one-integer edit to a manifest entry, a wrong encryption key and a mid-write interruption
+were each caught in live rehearsal with `HASH_MISMATCH`, `MANIFEST_HASH_MISMATCH`,
+`DECRYPTION_FAILED` and `MANIFEST_INCOMPLETE`, and none of them could become a restore plan.
+
+Immutability across several flushes required loosening the immutability guard to permit exactly the
+sealing flush — a backup is planned, written and only then stamped — while still rejecting every
+later change to identity fields.
+
+The platform can never retain zero recovery points because a timer expired, and a backup can never
+contain a record of itself as complete, so a restored plane consistently reports the backup it came
+from as not restore eligible.

docs/architecture/adr/0043-artifact-rehydration-and-dr-boundaries.md

@@ -0,0 +1,37 @@
+# ADR-0043: Artifact rehydration and disaster-recovery boundaries
+
+## Status
+
+Accepted in M15 on 2026-08-27.
+
+## Decision
+
+Recover rehydratable artifacts by redownloading their exact upstream revision through the existing
+acquisition plane — download plan, quarantine, hash verification against provenance, atomic
+promotion — rather than by copying weights into every backup. A floating revision is never used, and
+an artifact set without both an upstream repository and a resolved commit is classified
+`NON_REHYDRATABLE` and requires a payload backup.
+
+Treat a rehydrated artifact as new material: it re-enters quarantine, is re-checked against the
+security policy, keeps `trust_remote_code=false` and carries
+`static_checks_passed_unapproved` rather than inheriting its predecessor's approval.
+
+Draw the disaster-recovery boundary explicitly. ModelForge restores only into a database that is not
+its own, never writes to ExampleRAG, ExampleVision, Ollama, Plex, Tdarr or Gitea, and does not claim to
+protect its own Git host. Restore execution against a production target requires an explicit mode
+and an explicit deployment flag; the destructive half of recovery lives in the operator CLI and
+runbooks rather than in a remote API call.
+
+## Consequences
+
+Backups stay small and fast while a 247 MB artifact set was recovered in 24.2 seconds with all seven
+digests matching provenance, into a disposable location, leaving the 75 production artifact
+locations untouched.
+
+Recovery cannot be used to launder an unapproved artifact into an approved one, and an unreachable
+upstream produces `ARTIFACT_REHYDRATION_BLOCKED` instead of a false claim that the platform is
+recovered.
+
+The boundary is a real constraint: a genuine loss of a non-rehydratable or derived artifact needs a
+payload backup that must be configured deliberately, and full protection of the Git host remains
+platform operations rather than a ModelForge feature.

docs/architecture/adr/0044-bounded-fault-injection-and-executable-invariants.md

@@ -0,0 +1,39 @@
+# ADR-0044: Bounded fault injection and executable invariants
+
+## Status
+
+Accepted in M16 on 2026-08-27.
+
+## Decision
+
+Inject every fault from outside the product — the container runtime, existing rehearsal seams, real
+credentials, real load — and give ModelForge no chaos endpoint, no command route and no way to run
+a shell. Subprocess use stays allowlisted to the recovery plane, with argv built as a list and
+never a string, and static tests enforce all of it.
+
+Express the platform's safety properties as fifteen read-only invariant queries over authoritative
+state rather than as prose. Run them before a fault, after it, and after recovery. A scenario that
+survives its fault but breaks an invariant reports `INVARIANT_VIOLATED` rather than passing, and a
+scenario that cannot restore the platform reports `RECOVERY_FAILED` rather than leaving the
+environment dirty.
+
+Where a property is already enforced by a partial unique index, keep the index as the primary
+defence and treat the invariant as defence in depth for a bypassed guard.
+
+## Consequences
+
+Fault injection cannot become an attack surface: there is nothing in the product to call. The cost
+is that some faults can only be reached with runtime access, which is the correct trade for a
+control plane.
+
+Invariants are usable during an incident because they never write, never call an external system
+and never read the observability database — the case where monitoring itself has failed is exactly
+when the question "is state still sound?" matters most.
+
+Each invariant carries a negative test that constructs its violation, because a check that never
+fires reports safety it did not verify. Two of the M16 scenarios found real product defects and one
+scenario was itself wrong first, reporting that a heartbeat kept advancing during an outage it was
+not actually causing; recording observations rather than only pass/fail is what surfaced that.
+
+Every new milestone must classify the tables it adds, or the fingerprint and invariant coverage
+tests fail the build.

docs/architecture/adr/0045-chaos-safe-reconciliation-semantics.md

@@ -0,0 +1,41 @@
+# ADR-0045: Chaos-safe reconciliation semantics
+
+## Status
+
+Accepted in M16 on 2026-08-27.
+
+## Decision
+
+Every reconciliation path converges rather than accumulates, and each subsystem declares which of
+three answers it gives after an interruption:
+
+```text
+resolve      the safe outcome is knowable from local state; apply it exactly once
+report       the safe outcome depends on external truth; surface it and stop
+fail closed  the artefact cannot be trusted; mark it unusable
+```
+
+Lifecycle resolves: an incomplete operation rolls back from its immutable snapshot, once, and a
+second start changes nothing. Migration reports: an interrupted cutover stays in its intermediate
+stage because external alias truth cannot be inferred after a crash. Backups fail closed: a backup
+interrupted mid-write can never become restore eligible, and a restore interrupted mid-phase
+requires operator review rather than resuming.
+
+Single-use claims are atomic conditional updates whose row count decides the winner, not
+read-then-write checks. Idempotency keys are unique in the database, not in a queue, so a retry
+storm converges on one row regardless of which component retried.
+
+## Consequences
+
+A restart storm is uneventful: four consecutive control-plane restarts under load produced no
+authoritative drift, and a Redis outage produced no duplicate serving job.
+
+The cost is that some recoveries stop and wait for a human. That is deliberate — a migration
+cutover that guesses is how a platform ends up serving two embedding spaces, and an operator
+reconciling through the typed adapter is cheaper than a silent split.
+
+"Exactly once" has to be scoped to the operation, not the action class. M15 shipped a reconciliation
+marker keyed on the audit action alone, which meant a backup carrying an earlier restore's
+reconciliation event made every later restore believe it had already reconciled. The marker is now
+per-operation and the readiness gate additionally refuses to pass while stale current truth
+survives.

docs/architecture/adr/0046-v1-security-release-gate.md

@@ -0,0 +1,39 @@
+# ADR-0046: v1 security release gate
+
+## Status
+
+Accepted in M16 on 2026-08-27.
+
+## Decision
+
+Treat the security review as a source of build-enforced properties rather than a point-in-time
+report. Every conclusion the M16 review reached by hand is encoded as a test: no shell execution,
+no arbitrary execution primitive, subprocess use allowlisted, no command route, AES-256-GCM from a
+reviewed library with a fresh nonce per chunk and fail-closed decryption, no privileged mode or
+Docker socket, no floating dependency specifier, datastores bound to loopback, capabilities dropped,
+and a console image that serves a build rather than a development server.
+
+Publish nothing on a network interface that is not authenticated for it. PostgreSQL, Redis and the
+operator console default to loopback; the API is published deliberately because every admin route
+is operator-authenticated.
+
+Inventory what shipped, not what was intended: read the Python SBOM from the built images, bind each
+image to its source commit, and classify every advisory with evidence rather than counting them.
+
+## Consequences
+
+The most serious finding of M16 was configuration, not code: the control-plane database was
+published on every interface behind a development password, exposing provenance, credential hashes
+and the audit trail to the LAN. A test now fails if that binding returns.
+
+Encoding review conclusions as tests means M17 cannot quietly reintroduce them while packaging, and
+a future reviewer inherits the reasoning rather than having to rediscover it.
+
+The cost is friction: adding a subprocess call, a new route segment, or a dependency without a
+version now requires either a deliberate allowlist change or a failing build. That friction is the
+point.
+
+Some findings are correctly accepted rather than fixed. `pip` carried archive-extraction advisories
+and never runs at runtime, but leaving a known-vulnerable installer in a release image makes every
+future scan noisier, so the build upgrades it — a cheap fix is preferred to a standing exception
+even where the risk is not reachable.

docs/architecture/adr/0047-terminal-audited-node-decommission.md

@@ -0,0 +1,30 @@
+# ADR-0047: terminal audited node decommission
+
+## Status
+
+Accepted after v1.0 on 2026-08-28.
+
+## Decision
+
+Represent permanent compute-node removal as an immutable tombstone on `ComputeNode` plus one
+`NodeDecommissionOperation`, never as a hard delete. Require an authenticated preview/execute
+protocol bound to an optimistic node generation and a deterministic dependency digest. Recheck all
+blockers in the execution transaction, lock the dependency graph on PostgreSQL, and provide no
+force override.
+
+Revoke credentials and remove only current, rebuildable truth. Preserve every identity and record
+needed to explain acquisition, validation, runtime, deployment, serving, capacity and audit history.
+Block ordinary enrollment and every other known mutation path from resurrecting the tombstone.
+
+## Consequences
+
+Foreign-key `CASCADE`, `SET NULL` and `RESTRICT` semantics no longer determine whether node history
+survives an operator action: the row is retained deliberately. Operators get an explicit inventory
+of what is blocked, changed and preserved before execution, and retries return the original result.
+
+The transition takes broad, short-lived PostgreSQL table locks. Decommission is rare and safety is
+more important than maximizing concurrent scheduling throughput. New node-referencing tables must
+be added to the preview, lock set, cleanup classification and terminal invariant before release.
+
+Recovery from an outage and permanent decommission remain distinct. Reusing a decommissioned
+persisted identity is refused until a separately designed explicit recovery lifecycle exists.