Initial public ModelForge release

2026-09-01 21:30:16 +02:00
commit 7082ab955a
490 changed files with 104252 additions and 0 deletions
@@ -0,0 +1,13 @@
+# Evaluation framework
+
+ModelForge persists generic `EvaluationSuite`, immutable `EvaluationSuiteRevision`, bounded
+`EvaluationCase`, `EvaluationRun`, per-case result and `EvaluationComparison` records. A typed
+project adapter executes retrieval; ModelForge never executes user-supplied code.
+
+A revision digest covers cases, labels, critical flags, metrics, top-K, retrieval settings and
+thresholds. Editing any of these requires a new revision. Runs pin suite/dataset and corpus
+revisions, target index/space, retrieval-config digest and full local environment identity.
+
+Comparability is `comparable` only when project, suite revision, corpus revision and retrieval
+configuration match. Eligibility remains distinct from promotion and is blocked by incompatible
+runs, excess errors, a configured Recall@10 regression or critical regressions.