66 lines
3.4 KiB
Markdown
66 lines
3.4 KiB
Markdown
# Artifact Provenance
|
|
|
|
## Content identity
|
|
|
|
An artifact is identified by its parent exact revision and SHA-256 digest. Filename, storage root
|
|
and path are descriptive/location data and cannot change identity. One artifact can therefore have
|
|
zero, one or many physical `ArtifactLocation` records across compute nodes and storage roots.
|
|
|
|
Expected digest and byte size are registered before a local file is trusted. Artifact state is one
|
|
of `remote`, `local`, `verifying`, `verified`, `missing`, `corrupt`, `quarantined` or `archived`.
|
|
|
|
## Verification boundary
|
|
|
|
M2 verification performs only static file operations:
|
|
|
|
1. resolve a relative path beneath the configured storage root and reject path escape;
|
|
2. require a regular file;
|
|
3. stream bytes in 1 MiB chunks into SHA-256 while counting bytes;
|
|
4. compare both digest and size;
|
|
5. persist observed digest/size/time, status and an audit event.
|
|
|
|
It never deserializes weights, imports repository Python, executes model code or contacts a model
|
|
hub. Missing files become `missing`; digest or size mismatch becomes `corrupt`; only a complete match
|
|
becomes `verified`. Failure keeps the artifact quarantined.
|
|
|
|
## Derived lineage
|
|
|
|
`DerivedArtifactSource` stores an ordered N-source relation. Every source ID and hash is captured.
|
|
The output also records transformation type, tool, tool version, complete configuration,
|
|
environment snapshot, output digest and byte size. M2 registers this provenance but performs no
|
|
conversion or quantization.
|
|
|
|
Derived content fields are immutable after registration. Source edges make source-artifact deletion
|
|
return HTTP 409. This preserves the ability to reconstruct why an output exists even when it has no
|
|
current physical location.
|
|
|
|
## Audit actions
|
|
|
|
M2 emits append-only hash-chained events for model creation/update/deprecation/deletion, revision
|
|
registration, artifact registration and verification, derived registration, storage-root validation,
|
|
capacity checks and blocked deletion attempts.
|
|
|
|
## M3 acquisition chain
|
|
|
|
M3 persists the complete chain from model and exact revision through upstream snapshot/file,
|
|
artifact set, immutable approved plan, target-node job/attempt, quarantine path, static inspection,
|
|
local SHA-256, verified artifact and physical location. Artifact-set membership records required
|
|
files explicitly. `static_checks_passed_unapproved` means local integrity evidence only and cannot
|
|
be confused with a later supply-chain or runtime approval.
|
|
|
|
M4 extends the chain without mutating artifact identity: exact-set approval fingerprints reviewed
|
|
security facts, static assessments snapshot artifact/hardware/runtime facts, probes record the same
|
|
ArtifactSet/revision/profile/node plus measured runtime facts, and candidates refer to the successful
|
|
probe. Runtime image digests and environment fingerprints are evidence, not artifact digests.
|
|
|
|
The promoted `.modelforge-manifest.json` is reconciliation evidence beside the bytes. The database
|
|
remains operational truth; a retry can reconcile an exact promoted directory after an interrupted
|
|
completion report without changing content identity.
|
|
|
|
## M12 lifecycle evidence
|
|
|
|
Approval snapshots add exact policy, ArtifactSet, revision, runtime/probe/image, evaluation, project
|
|
fit, scheduler, migration and rollback references without copying inference content. Cleanup retains
|
|
this chain even when one binary location is later removed: the prior digest, size, location status,
|
|
actor, plan and event remain queryable.
|