146 lines
5.6 KiB
Markdown
146 lines
5.6 KiB
Markdown
# 19 — Acceptance criteria
|
|
|
|
## Product acceptance matrix
|
|
|
|
### Installation
|
|
|
|
- [x] Fresh Docker deployment starts from documented instructions.
|
|
- [x] Database migrations apply to an empty database.
|
|
- [x] All 28 P0 built-in packages import idempotently and match their seed-catalog definitions.
|
|
- [x] First-run ownership is protected by setup token/local policy, concurrent setup is safe and no default credential exists.
|
|
- [x] Restart preserves users, profiles, runs and artifacts.
|
|
|
|
### Library
|
|
|
|
- [x] Search returns expected title, tag and intent matches.
|
|
- [x] All filters work individually and in combination.
|
|
- [x] URL preserves search state.
|
|
- [x] Deprecated versions are not default recommendations.
|
|
- [x] Card and dense views are responsive and accessible.
|
|
|
|
### Playbook package
|
|
|
|
- [x] Every non-manifest package file is declared with role, digest and export behavior.
|
|
- [x] Condition AST is parsed without dynamic code evaluation.
|
|
- [x] Default mode belongs to supported modes and governed capabilities resolve consistently.
|
|
|
|
- [x] All bundled packages validate structurally and semantically.
|
|
- [x] Invalid input reports exact path and remediation.
|
|
- [x] Published versions are immutable.
|
|
- [x] Duplicate version with different digest is rejected.
|
|
- [x] Import/export round-trip preserves canonical digest.
|
|
|
|
### Repository profiles
|
|
|
|
- [x] Manual profile can be created without an integration.
|
|
- [x] Commands, protected paths and policies validate.
|
|
- [x] Editing creates a new revision.
|
|
- [x] Historical run retains its original profile snapshot.
|
|
- [x] YAML/JSON profile round-trip succeeds.
|
|
|
|
### Composer
|
|
|
|
- [x] Golden fixture conformance: production composition of every supplied P0 minimal example is byte-identical to `examples/rendered-prompts/` and matches the manifest digest.
|
|
|
|
- [x] Required inputs block export until resolved.
|
|
- [x] Compatibility warnings and errors are correct.
|
|
- [x] Autonomy outside playbook range is rejected.
|
|
- [x] Protected paths are rendered as explicit constraints.
|
|
- [x] Preview shows provenance and lint findings.
|
|
- [x] Same normalized inputs produce identical bytes/digest on Linux and Windows fixtures.
|
|
- [x] Repository evidence is fenced as untrusted.
|
|
|
|
### Prompt quality
|
|
|
|
- [x] Every representative implementation prompt contains mission, scope, constraints, workflow, validation, completion and reporting.
|
|
- [x] Observe prompts contain no change authorization.
|
|
- [x] Bugfix prompt requires reproduction and regression evidence.
|
|
- [x] Migration prompt requires backup/rollback behavior.
|
|
- [x] Secret fixtures are redacted or block export.
|
|
- [x] Blocking lint issues prevent final run generation.
|
|
|
|
### Runs and exports
|
|
|
|
- [x] Final generation creates immutable run snapshot.
|
|
- [x] Copy and Markdown export match stored prompt digest.
|
|
- [x] Run Pack manifest lists every file and correct digest.
|
|
- [x] Re-import verifies integrity.
|
|
- [x] Archive traversal and symlink attacks are rejected.
|
|
- [x] Artifact authorization prevents cross-workspace access.
|
|
|
|
### Gitea
|
|
|
|
- [x] Connection test records server capability state.
|
|
- [x] Token is never returned or logged.
|
|
- [x] Repository discovery paginates correctly.
|
|
- [x] Evidence collection respects allowlist and size limits.
|
|
- [x] No write endpoint is invoked.
|
|
- [x] Permission-limited capabilities degrade individually.
|
|
- [x] Last snapshot remains usable during outage.
|
|
- [x] SSRF and redirect security tests pass.
|
|
|
|
### Prompt Lab
|
|
|
|
- [x] Draft editor shows schema and semantic errors.
|
|
- [x] Version publication requires changelog.
|
|
- [x] Published content cannot be edited in place.
|
|
- [x] Example renders are reproducible.
|
|
- [x] Quality status cannot exceed evidence policy.
|
|
- [x] Version diff highlights scope, guardrail and validation changes.
|
|
|
|
### Accessibility and visual quality
|
|
|
|
- [x] Core flows operate by keyboard.
|
|
- [x] Focus states are visible.
|
|
- [x] Error summary links to invalid fields.
|
|
- [x] Reduced motion is respected.
|
|
- [x] Both themes meet contrast targets.
|
|
- [x] Desktop, laptop and narrow viewport checks pass.
|
|
- [x] No clipped or overlapping content in core views.
|
|
|
|
### Security
|
|
|
|
- [x] Cross-workspace authorization tests pass.
|
|
- [x] CSP and security headers verified.
|
|
- [x] Markdown and YAML XSS fixtures are neutralized.
|
|
- [x] Integration secrets encrypted at rest.
|
|
- [x] Secret redaction tests pass.
|
|
- [x] Import size/decompression limits work.
|
|
- [x] Dependency and secret scans have no unresolved critical/high product findings.
|
|
|
|
### Operations
|
|
|
|
- [x] Liveness and readiness behave as documented.
|
|
- [x] Worker jobs recover from process restart.
|
|
- [x] Failed jobs show actionable safe errors.
|
|
- [x] Backup and restore tested.
|
|
- [x] Migration preflight and rollback limits documented.
|
|
- [x] Clean-room smoke flow completed.
|
|
- [x] Release notes and final handoff are accurate.
|
|
|
|
### Identity and authorization
|
|
|
|
- [x] Cross-workspace authorization matrix passes for every private resource.
|
|
- [x] Viewer/editor/owner and instance-role boundaries behave as documented.
|
|
- [x] Session revocation, invitation and operator password-reset flows pass.
|
|
- [x] Setup endpoints are unavailable after first-run completion.
|
|
|
|
### Traceability
|
|
|
|
- [x] Every FR ID in document 01 has a final status and linked test evidence.
|
|
- [x] Accepted exceptions include owner, rationale and review date.
|
|
|
|
## Final release evidence
|
|
|
|
The release candidate must include a machine-readable and human-readable acceptance report containing:
|
|
|
|
- application commit and version;
|
|
- environment summary without secrets;
|
|
- commands executed;
|
|
- test counts and failures/skips;
|
|
- browser flows verified;
|
|
- migration and clean-room result;
|
|
- backup/restore result;
|
|
- known limitations;
|
|
- accepted exceptions with owner and rationale.
|