Initial public ModelForge release

2026-09-01 21:30:16 +02:00
commit 7082ab955a
490 changed files with 104252 additions and 0 deletions
@@ -0,0 +1,31 @@
+# ADR 0023 — Deterministic Advisor and raw evidence matrix
+
+Status: accepted
+
+## Decision
+
+Model comparisons are first-class immutable evidence snapshots. Each evaluated row references an
+existing EvaluationRun; quality metrics and critical-case outcomes are materialized from that run
+and its pairwise comparison rather than accepted as caller claims. Operational, resource, migration,
+security and provenance evidence remain separate fields. Blocked and unknown candidates remain
+visible as blocked or unknown and never receive fabricated metrics.
+
+Advisor recommendations are deterministic. The persisted policy requires project-local evidence
+class A for a high-confidence recommendation, treats any critical regression as a hard block,
+requires verified supply chain and runtime fit, and requires non-negative quality deltas. A p95
+increase above 100% is an operational review ceiling, not a weighted quality score. Policy versions
+and rationale are persisted with each recommendation.
+
+The only verdicts are `KEEP_CURRENT`, `PROMOTION_ELIGIBLE`, `PROMOTION_NOT_RECOMMENDED` and
+`REQUIRES_MORE_EVIDENCE`. There is deliberately no automatic approval or promotion endpoint.
+
+## Evidence hierarchy
+
+1. A — local project evaluation on the pinned suite/corpus/configuration.
+2. B — local generic benchmark or measured preflight.
+3. C — independent external benchmark.
+4. D — upstream model card or vendor claim.
+5. E — downloads, likes or trending data.
+
+Lower evidence cannot overrule project-local critical regressions without a new explicit policy and
+new recommendation fingerprint.