14 lines
845 B
Markdown
14 lines
845 B
Markdown
# Evaluation framework
|
|
|
|
ModelForge persists generic `EvaluationSuite`, immutable `EvaluationSuiteRevision`, bounded
|
|
`EvaluationCase`, `EvaluationRun`, per-case result and `EvaluationComparison` records. A typed
|
|
project adapter executes retrieval; ModelForge never executes user-supplied code.
|
|
|
|
A revision digest covers cases, labels, critical flags, metrics, top-K, retrieval settings and
|
|
thresholds. Editing any of these requires a new revision. Runs pin suite/dataset and corpus
|
|
revisions, target index/space, retrieval-config digest and full local environment identity.
|
|
|
|
Comparability is `comparable` only when project, suite revision, corpus revision and retrieval
|
|
configuration match. Eligibility remains distinct from promotion and is blocked by incompatible
|
|
runs, excess errors, a configured Recall@10 regression or critical regressions.
|