DevRunbook-Public/docs/19-acceptance-criteria.md

19 — Acceptance criteria

Product acceptance matrix

Installation

• Fresh Docker deployment starts from documented instructions.
• Database migrations apply to an empty database.
• All 28 P0 built-in packages import idempotently and match their seed-catalog definitions.
• First-run ownership is protected by setup token/local policy, concurrent setup is safe and no default credential exists.
• Restart preserves users, profiles, runs and artifacts.

Library

• Search returns expected title, tag and intent matches.
• All filters work individually and in combination.
• URL preserves search state.
• Deprecated versions are not default recommendations.
• Card and dense views are responsive and accessible.

Playbook package

• Every non-manifest package file is declared with role, digest and export behavior.

• Condition AST is parsed without dynamic code evaluation.

• Default mode belongs to supported modes and governed capabilities resolve consistently.

• All bundled packages validate structurally and semantically.

• Invalid input reports exact path and remediation.

• Published versions are immutable.

• Duplicate version with different digest is rejected.

• Import/export round-trip preserves canonical digest.

Repository profiles

• Manual profile can be created without an integration.
• Commands, protected paths and policies validate.
• Editing creates a new revision.
• Historical run retains its original profile snapshot.
• YAML/JSON profile round-trip succeeds.

Composer

• Golden fixture conformance: production composition of every supplied P0 minimal example is byte-identical to examples/rendered-prompts/ and matches the manifest digest.

• Required inputs block export until resolved.

• Compatibility warnings and errors are correct.

• Autonomy outside playbook range is rejected.

• Protected paths are rendered as explicit constraints.

• Preview shows provenance and lint findings.

• Same normalized inputs produce identical bytes/digest on Linux and Windows fixtures.

• Repository evidence is fenced as untrusted.

Prompt quality

• Every representative implementation prompt contains mission, scope, constraints, workflow, validation, completion and reporting.
• Observe prompts contain no change authorization.
• Bugfix prompt requires reproduction and regression evidence.
• Migration prompt requires backup/rollback behavior.
• Secret fixtures are redacted or block export.
• Blocking lint issues prevent final run generation.

Runs and exports

• Final generation creates immutable run snapshot.
• Copy and Markdown export match stored prompt digest.
• Run Pack manifest lists every file and correct digest.
• Re-import verifies integrity.
• Archive traversal and symlink attacks are rejected.
• Artifact authorization prevents cross-workspace access.

Gitea

• Connection test records server capability state.
• Token is never returned or logged.
• Repository discovery paginates correctly.
• Evidence collection respects allowlist and size limits.
• No write endpoint is invoked.
• Permission-limited capabilities degrade individually.
• Last snapshot remains usable during outage.
• SSRF and redirect security tests pass.

Prompt Lab

• Draft editor shows schema and semantic errors.
• Version publication requires changelog.
• Published content cannot be edited in place.
• Example renders are reproducible.
• Quality status cannot exceed evidence policy.
• Version diff highlights scope, guardrail and validation changes.

Accessibility and visual quality

• Core flows operate by keyboard.
• Focus states are visible.
• Error summary links to invalid fields.
• Reduced motion is respected.
• Both themes meet contrast targets.
• Desktop, laptop and narrow viewport checks pass.
• No clipped or overlapping content in core views.

Security

• Cross-workspace authorization tests pass.
• CSP and security headers verified.
• Markdown and YAML XSS fixtures are neutralized.
• Integration secrets encrypted at rest.
• Secret redaction tests pass.
• Import size/decompression limits work.
• Dependency and secret scans have no unresolved critical/high product findings.

Operations

• Liveness and readiness behave as documented.
• Worker jobs recover from process restart.
• Failed jobs show actionable safe errors.
• Backup and restore tested.
• Migration preflight and rollback limits documented.
• Clean-room smoke flow completed.
• Release notes and final handoff are accurate.

Identity and authorization

• Cross-workspace authorization matrix passes for every private resource.
• Viewer/editor/owner and instance-role boundaries behave as documented.
• Session revocation, invitation and operator password-reset flows pass.
• Setup endpoints are unavailable after first-run completion.

Traceability

• Every FR ID in document 01 has a final status and linked test evidence.
• Accepted exceptions include owner, rationale and review date.

Final release evidence

The release candidate must include a machine-readable and human-readable acceptance report containing:

• application commit and version;
• environment summary without secrets;
• commands executed;
• test counts and failures/skips;
• browser flows verified;
• migration and clean-room result;
• backup/restore result;
• known limitations;
• accepted exceptions with owner and rationale.