This commit is contained in:
ITWorx Pulse release export
@@ -0,0 +1,95 @@
|
||||
# Backend and worker standards
|
||||
|
||||
## Domain boundaries
|
||||
|
||||
Core packages define:
|
||||
- entities and reconciliation;
|
||||
- metrics query model;
|
||||
- dashboard validation/versioning;
|
||||
- events;
|
||||
- alert state machine;
|
||||
- incidents;
|
||||
- authorization policies.
|
||||
|
||||
Adapters implement:
|
||||
- PostgreSQL;
|
||||
- Prometheus;
|
||||
- Unraid;
|
||||
- Docker/constrained collector;
|
||||
- OIDC;
|
||||
- probes;
|
||||
- notifications.
|
||||
|
||||
Domain packages must not import HTTP handlers or concrete adapters.
|
||||
|
||||
## HTTP
|
||||
|
||||
- Route groups map to modules.
|
||||
- Middleware order is explicit.
|
||||
- Request body, query and path validation.
|
||||
- Body size limits.
|
||||
- Timeouts and cancellation.
|
||||
- Stable error mapping.
|
||||
- Structured request log with redaction.
|
||||
- Health/live and readiness separated.
|
||||
- No mutation route for host/Docker/storage.
|
||||
|
||||
## Worker
|
||||
|
||||
- Jobs have stable keys and schedules.
|
||||
- Database lock/lease prevents duplicate execution.
|
||||
- Work is idempotent.
|
||||
- Retries are bounded with jitter.
|
||||
- Poison/repeated failures become visible system status.
|
||||
- Shutdown waits for bounded graceful completion.
|
||||
- Job result counts and duration recorded.
|
||||
|
||||
## Prometheus adapter
|
||||
|
||||
- Server-side only.
|
||||
- Timeout and concurrency semaphore.
|
||||
- Semantic templates only by default.
|
||||
- Parse upstream warnings and staleness.
|
||||
- Protect metadata/label endpoints.
|
||||
- Normalize errors to stable codes.
|
||||
- Instrument query duration/series/points/cache.
|
||||
|
||||
## Inventory reconciliation
|
||||
|
||||
- Stable source alias mapping.
|
||||
- Snapshot can be repeated safely.
|
||||
- Missing item becomes tombstoned after source-specific policy, not immediately deleted.
|
||||
- User overrides remain separate.
|
||||
- Relation source/confidence retained.
|
||||
- Changes create deduplicated events.
|
||||
- Source failure does not tombstone all entities.
|
||||
|
||||
## Alerts
|
||||
|
||||
- Deterministic state machine.
|
||||
- Evaluation transaction/locking.
|
||||
- Clock abstracted in tests.
|
||||
- Unknown data explicit.
|
||||
- Rule versions immutable.
|
||||
- Notification side effects use outbox/idempotency pattern or equivalent.
|
||||
- Acknowledgement does not mutate underlying firing condition.
|
||||
|
||||
## Database
|
||||
|
||||
- Context-aware queries.
|
||||
- Parameterized SQL.
|
||||
- Transaction boundaries documented.
|
||||
- Pool limits.
|
||||
- Repository methods return domain types/errors.
|
||||
- Integration tests against real PostgreSQL.
|
||||
- No production reliance on SQLite semantics.
|
||||
|
||||
## Agent
|
||||
|
||||
- Read-only capability list.
|
||||
- No generic command execution endpoint.
|
||||
- Authenticated/mutually trusted channel if remote.
|
||||
- Bounded payloads.
|
||||
- Version/capability negotiation.
|
||||
- Local cache only where safe.
|
||||
- Every elevated mount/capability justified and tested.
|
||||
@@ -0,0 +1,36 @@
|
||||
# CI pipeline target
|
||||
|
||||
Codex must implement CI-equivalent commands locally even when no hosted CI is connected.
|
||||
|
||||
Recommended stages:
|
||||
|
||||
1. repository/planning validation;
|
||||
2. formatting;
|
||||
3. lint/static analysis;
|
||||
4. generated contract drift;
|
||||
5. wiring/reachability gate (`python tools/check_wiring.py`): every production package must have at least one non-test importer reachable from a binary (`cmd/api`, `cmd/worker`, `cmd/agent`, `cmd/migrate`), or be listed in `tools/wiring_allowlist.json` with a reason and a tracking task id; a task may not be marked done while its deliverable is unreachable;
|
||||
6. frontend type/unit (`pnpm test` runs Vitest + Testing Library, including ADR-0008 status invariants);
|
||||
7. Go unit/race where suitable;
|
||||
8. integration with PostgreSQL/fake sources;
|
||||
9. frontend build;
|
||||
10. API/OpenAPI/schema compatibility;
|
||||
11. Playwright smoke (`pnpm test:e2e`, desktop/mobile/wallboard Chromium projects);
|
||||
12. accessibility (axe-core in the Playwright smoke; serious or critical violations fail);
|
||||
13. dependency/license/secret scan;
|
||||
14. container build and image scan;
|
||||
15. compose/config validation;
|
||||
16. evidence summary.
|
||||
|
||||
The executable Gitea Actions definition is `.gitea/workflows/ci.yml`. The local
|
||||
equivalent is `scripts/verify.ps1`; install Chromium once with
|
||||
`pnpm exec playwright install chromium` before its browser stage. Browser tests
|
||||
mock only the versioned API boundary and exercise the real built React routes.
|
||||
|
||||
Release pipeline additionally:
|
||||
- clean checkout;
|
||||
- full E2E;
|
||||
- performance subset;
|
||||
- migration/backup/restore;
|
||||
- SBOM/provenance where feasible;
|
||||
- immutable image digest record;
|
||||
- deployment smoke and rollback proof.
|
||||
@@ -0,0 +1,35 @@
|
||||
# M1 dependency record
|
||||
|
||||
Research and install date: 2026-08-01
|
||||
|
||||
These exact versions are pinned in `package.json`/`apps/web/package.json` and `pnpm-lock.yaml`. Go is installed in the developer's user-local toolchain cache at Go 1.26.6 and is declared in `go.mod`/`go.work`; it is not vendored into the repository.
|
||||
|
||||
| Dependency | Version | Purpose | License | Alternative/decision |
|
||||
|---|---|---|---|---|
|
||||
| Go toolchain | 1.26.6 | API, worker, agent | BSD-style | Official current supported patch; required by the M13 release image gate to remove fixed Go standard-library High findings. |
|
||||
| React | 19.2.8 | Web UI | MIT | React is required by the architecture; framework/server rendering is unnecessary for the static dashboard shell. |
|
||||
| React DOM | 19.2.8 | Browser renderer | MIT | Pinned with React. |
|
||||
| Vite | 8.2.0 | TypeScript web build/dev server | MIT | Chosen over a heavier framework because the architecture calls for a static React app and Vite supports the available Node runtime. |
|
||||
| `@vitejs/plugin-react` | 6.0.5 | Vite React transform | MIT | Official Vite ecosystem plugin; pinned with Vite 8. |
|
||||
| TypeScript | 7.0.2 | Strict web type checking | Apache-2.0 | Required for the React + TypeScript architecture. |
|
||||
| React type declarations | 19.2.18 / 19.2.4 | Compile-time types | MIT | Pinned to the installed React major. |
|
||||
| `github.com/jackc/pgx/v5` | 5.10.0 | PostgreSQL connection pool and parameterized access | MIT | Selected for native context-aware pooling and PostgreSQL support; pinned after module/license review. |
|
||||
| `github.com/coreos/go-oidc/v3` | 3.20.0 | OIDC discovery, issuer/audience/JWK-backed ID-token verification | Apache-2.0 | Uses maintained standards-oriented verifier; server-side only, pinned after module/license review. |
|
||||
| `golang.org/x/oauth2` | 0.36.0 | Authorization-code exchange and PKCE request parameters | BSD-style | Official Go OAuth2 client primitives; pinned and kept behind the auth adapter. |
|
||||
|
||||
No charting, grid, or HTTP-router dependency is added yet. Those material choices require the relevant task's primary-source/security/license review and measurement. `pnpm-lock.yaml` records registry integrity data; Go module checksums are recorded in `go.sum`.
|
||||
|
||||
## Verification record
|
||||
|
||||
- `go version`: `go1.26.6 windows/amd64`.
|
||||
- `pnpm install --frozen-lockfile`: pass with pnpm 10.33.0.
|
||||
- Vite's official compatibility floor is Node 20.19+ or 22.12+; the local Node 24.18.1 satisfies it.
|
||||
- `go test ./...`, `go vet ./...`, TypeScript typecheck, Vite build, and repository bootstrap/test/lint scripts pass.
|
||||
- `go mod verify`: pass; pgx v5.10.0 and transitive modules are checksum-verified.
|
||||
- pgx v5.10.0 module metadata points to the upstream `github.com/jackc/pgx` repository; the cached module includes an MIT license.
|
||||
- go-oidc v3.20.0 and oauth2 v0.36.0 are checksum-verified; the cached go-oidc module includes an Apache-2.0 license and oauth2 is maintained under the Go project license.
|
||||
- Releasegate 2026-08-21 pins transitieve builddependency `nanoid` op 3.3.18 via een beperkte pnpm-override. Dit sluit GHSA-2v37-7h3g-55p8 in Vite -> PostCSS; `pnpm audit --audit-level high` en Trivy met developmentdependencies rapporteren daarna nul High/Critical-bevindingen. Nanoid blijft uitsluitend onderdeel van de MIT-gelicentieerde buildketen en wordt niet aan de browserruntime toegevoegd.
|
||||
|
||||
## Upgrade/removal path
|
||||
|
||||
Update package manifests and lockfile together, rerun the foundation scripts plus the affected milestone gate, review changelogs/security advisories, and record any compatibility or bundle/runtime impact. Removing Vite/React is an architectural change requiring an ADR; removing a foundation tool requires replacement commands and clean-room evidence.
|
||||
@@ -0,0 +1,59 @@
|
||||
# Dependency policy
|
||||
|
||||
A production dependency is accepted only when it:
|
||||
|
||||
- solves a real requirement better than a small maintained implementation;
|
||||
- is actively maintained;
|
||||
- has a compatible license;
|
||||
- has no unresolved unacceptable security issue;
|
||||
- supports the selected runtime/browser versions;
|
||||
- has clear upgrade and removal paths;
|
||||
- does not require excessive privilege or bundle size.
|
||||
|
||||
## Selection record
|
||||
|
||||
For material dependencies record:
|
||||
- package and version range;
|
||||
- purpose;
|
||||
- alternatives considered;
|
||||
- maintenance/release activity;
|
||||
- license;
|
||||
- security check;
|
||||
- bundle/image/runtime impact;
|
||||
- locking strategy.
|
||||
|
||||
This may be an ADR or a dependency manifest note.
|
||||
|
||||
## Default choices to validate during M0/M1
|
||||
|
||||
- React + TypeScript + Vite.
|
||||
- GridStack for layout.
|
||||
- uPlot for high-volume time series.
|
||||
- ECharts for complex visualizations.
|
||||
- TanStack Query for server state.
|
||||
- Accessible UI primitives/component library.
|
||||
- Go HTTP/router, OIDC/JWT and PostgreSQL libraries selected from maintained options.
|
||||
- PostgreSQL migration tool with explicit CLI and rollback strategy.
|
||||
- Playwright and automated accessibility tooling.
|
||||
- Testcontainers for integration tests.
|
||||
|
||||
These are defaults, not permission to install blindly. Verify current supported versions and compatibility.
|
||||
|
||||
## Rules
|
||||
|
||||
- Commit lockfiles.
|
||||
- Prefer exact image tags/digests in production records.
|
||||
- Avoid duplicate libraries for the same concern.
|
||||
- Do not use abandonware because an example already uses it.
|
||||
- Do not add Redis, Kafka, Elasticsearch or a second metrics database without measured need and ADR.
|
||||
- Remove unused dependencies immediately.
|
||||
- Run dependency/license/vulnerability checks at milestones and release.
|
||||
|
||||
|
||||
## M4-06 WebSocket selection
|
||||
|
||||
- Package/version: github.com/coder/websocket v1.8.15, pinned in go.mod and go.sum.
|
||||
- Purpose: RFC6455 server upgrade, context-aware reads/writes, ping/pong and bounded frame reads for the authenticated live endpoint.
|
||||
- Alternatives considered: hand-rolled RFC6455 handling was rejected because it increases protocol and security risk; gorilla/websocket was not needed for this narrow API; golang.org/x/net/websocket is deprecated.
|
||||
- Maintenance/security: current upstream release was resolved locally on 2026-08-01; the module has zero transitive dependencies and the source license is permissive MIT.
|
||||
- Runtime impact: server-only dependency, no browser bundle or privilege change; SetReadLimit and write deadlines enforce the endpoint budget.
|
||||
@@ -0,0 +1,125 @@
|
||||
# Engineering standards
|
||||
|
||||
## Repository shape
|
||||
|
||||
Target:
|
||||
|
||||
```text
|
||||
apps/
|
||||
web/
|
||||
api/
|
||||
worker/
|
||||
agent/
|
||||
packages/
|
||||
contracts/
|
||||
ui/
|
||||
test-fixtures/
|
||||
internal/
|
||||
domain modules or Go internal packages
|
||||
config/
|
||||
deploy/
|
||||
tests/
|
||||
docs/
|
||||
artifacts/evidence/
|
||||
```
|
||||
|
||||
Codex may refine the shape through an ADR, but privilege boundaries and clear ownership must remain.
|
||||
|
||||
## General
|
||||
|
||||
- Optimize for correctness, observability and maintainability.
|
||||
- Keep changes vertically complete.
|
||||
- Validate external inputs at boundaries.
|
||||
- Avoid global mutable state.
|
||||
- Use deterministic IDs/fingerprints where required.
|
||||
- Use UTC internally.
|
||||
- Add correlation IDs to request/job/event paths.
|
||||
- Preserve error causes and add context.
|
||||
- Never log secrets.
|
||||
- Use feature flags only when they have an owner, default and removal plan.
|
||||
|
||||
## Git and commits
|
||||
|
||||
- Focused commits aligned to task IDs.
|
||||
- Commit message format: `<TASK_ID>: <imperative summary>`.
|
||||
- Do not rewrite shared history.
|
||||
- No force push or destructive reset.
|
||||
- Keep generated evidence out of commits only when too large; summaries remain.
|
||||
- Tag releases only after final acceptance.
|
||||
|
||||
## Go
|
||||
|
||||
- Current supported stable Go version selected during M0 and recorded.
|
||||
- `go fmt`, `go vet`, static analysis and tests required.
|
||||
- Context propagated through I/O boundaries.
|
||||
- Errors wrapped with operation context.
|
||||
- Interfaces defined near consumers; avoid interface proliferation.
|
||||
- Goroutines have ownership, cancellation and bounded lifetime.
|
||||
- Worker jobs are idempotent.
|
||||
- SQL is parameterized; transactions explicit.
|
||||
- Migrations are forward/recovery tested.
|
||||
- HTTP handlers contain no core domain logic.
|
||||
|
||||
## TypeScript/React
|
||||
|
||||
- Strict TypeScript.
|
||||
- No `any` except narrow justified boundary adapters.
|
||||
- Runtime validation for external JSON.
|
||||
- Components separate data orchestration from presentation.
|
||||
- Server state uses a deliberate query/cache layer.
|
||||
- Live chart buffers do not live in broad global state.
|
||||
- Effects are cancellable and cleanup subscriptions.
|
||||
- Accessible semantic HTML first.
|
||||
- All user copy goes through localization.
|
||||
- Avoid giant components and prop drilling; centralize domain-specific hooks appropriately.
|
||||
|
||||
## API and contracts
|
||||
|
||||
- OpenAPI/JSON Schema is validated in CI.
|
||||
- Breaking changes are versioned.
|
||||
- Generated types are reproducible.
|
||||
- Error codes are stable.
|
||||
- Pagination, filtering and sorting are bounded.
|
||||
- Every endpoint has authz tests.
|
||||
- WebSocket messages are schema validated.
|
||||
|
||||
## Database
|
||||
|
||||
- Explicit migrations, no startup auto-mutation outside migration command.
|
||||
- Indexes justified by access path.
|
||||
- Constraints enforce invariants where practical.
|
||||
- JSONB payloads have size/schema limits.
|
||||
- Optimistic concurrency for user-edited versioned resources.
|
||||
- Timeouts and connection pool limits.
|
||||
- Test upgrade, restart, backup and restore.
|
||||
|
||||
## Configuration
|
||||
|
||||
- `.env.example` documents non-secret values.
|
||||
- Startup validates configuration and reports all invalid fields.
|
||||
- Secrets use secret files/runtime injection when possible.
|
||||
- No environment-specific values embedded in images.
|
||||
- Production and test compose overrides are separate.
|
||||
- Feature capability detection is visible in UI/system status.
|
||||
|
||||
## Observability
|
||||
|
||||
Pulse emits:
|
||||
- structured logs;
|
||||
- internal metrics;
|
||||
- health/readiness;
|
||||
- job status;
|
||||
- trace/correlation IDs;
|
||||
- redacted upstream error classes.
|
||||
|
||||
Avoid recursive monitoring dependence: an external dead-man check must detect total Pulse failure.
|
||||
|
||||
## Documentation
|
||||
|
||||
Behavioral changes update:
|
||||
- relevant specification;
|
||||
- API/schema;
|
||||
- runbook if operational;
|
||||
- evidence;
|
||||
- current state;
|
||||
- ADR when architectural.
|
||||
@@ -0,0 +1,92 @@
|
||||
# Frontend standards
|
||||
|
||||
## Architecture
|
||||
|
||||
Recommended modules:
|
||||
|
||||
```text
|
||||
app-shell
|
||||
auth
|
||||
routing
|
||||
i18n
|
||||
design-system
|
||||
dashboards
|
||||
widgets
|
||||
metrics
|
||||
inventory
|
||||
alerts
|
||||
incidents
|
||||
events
|
||||
settings
|
||||
operations
|
||||
```
|
||||
|
||||
Feature modules own routes, queries, views and tests. Shared UI remains domain-neutral.
|
||||
|
||||
## State
|
||||
|
||||
- URL state for shareable filters/time range when appropriate.
|
||||
- Server state via TanStack Query or measured equivalent.
|
||||
- Editor draft state isolated from saved dashboard state.
|
||||
- WebSocket subscriptions managed by one client/service.
|
||||
- Chart samples in bounded local stores/ring buffers.
|
||||
- Avoid duplicating server state across stores.
|
||||
|
||||
## Dashboard editor
|
||||
|
||||
- Save complete version atomically.
|
||||
- Use explicit edit session/draft.
|
||||
- Detect optimistic concurrency conflict.
|
||||
- Undo/redo operates on normalized editor commands or bounded snapshots.
|
||||
- Breakpoint layouts validated before save.
|
||||
- Keyboard move/resize and screen-reader labels.
|
||||
- Prevent accidental drag from chart interactions.
|
||||
|
||||
## Charts
|
||||
|
||||
- Lazy-load heavy chart implementations.
|
||||
- Initialize history once per query.
|
||||
- Append live data efficiently.
|
||||
- Cap points/series.
|
||||
- Dispose observers/listeners/instances.
|
||||
- Pause offscreen/background work.
|
||||
- Provide textual summary/table alternative.
|
||||
- Use consistent units, timestamps, legend and tooltip behavior.
|
||||
- Status thresholds do not overwrite data meaning.
|
||||
|
||||
## Tables/lists
|
||||
|
||||
- Cursor/server pagination for large datasets.
|
||||
- Virtualization for large rendered collections.
|
||||
- Stable row keys.
|
||||
- Accessible sorting/filtering labels.
|
||||
- Preserve filter state sensibly.
|
||||
- Loading and empty states are distinct.
|
||||
|
||||
## Error handling
|
||||
|
||||
- Route-level boundary.
|
||||
- Component/query errors show safe, actionable messages.
|
||||
- Correlation ID exposed for diagnostics.
|
||||
- Retry only when safe and bounded.
|
||||
- Authentication expiry has a clean flow.
|
||||
- Partial datasource failure does not blank the entire app.
|
||||
|
||||
## Styling
|
||||
|
||||
- Use design tokens; no scattered literal colors/spacing.
|
||||
- Consistent card padding and grid gaps.
|
||||
- Limited elevation.
|
||||
- Status colors only for status.
|
||||
- Respect reduced motion.
|
||||
- Avoid oversized decorative headers that reduce information space.
|
||||
- Desktop and mobile screenshots/visual regression for key routes.
|
||||
|
||||
## Testing
|
||||
|
||||
- Unit tests for formatting/transforms/editor reducers.
|
||||
- Component tests for states and accessibility.
|
||||
- Playwright for user journeys.
|
||||
- Axe or equivalent on all core routes and viewports.
|
||||
- Real browser verification for drag/resize, WebSocket reconnect and wallboard.
|
||||
- Leak/soak instrumentation for chart/subscription lifecycle.
|
||||
@@ -0,0 +1,73 @@
|
||||
# Performance budgets
|
||||
|
||||
Budgets are acceptance targets measured in the documented test environment.
|
||||
|
||||
## Target scale
|
||||
|
||||
- 1 Unraid host;
|
||||
- 150 containers;
|
||||
- 40 disks;
|
||||
- 300 service probes;
|
||||
- 2,500 active series across an intensive dashboard set;
|
||||
- 10 concurrent authenticated users;
|
||||
- 1 wallboard open for at least 24 hours.
|
||||
|
||||
Container ingestion retains bounded headroom up to 250 records so a host that
|
||||
briefly grows beyond the 150-container performance target remains observable.
|
||||
The 150-container fixture remains the required latency and UI acceptance scale;
|
||||
the additional headroom is a safety boundary, not a higher performance claim.
|
||||
|
||||
## Browser
|
||||
|
||||
| Metric | Target |
|
||||
|---|---:|
|
||||
| First meaningful overview on LAN, warm service | < 2.0 s |
|
||||
| Main route interaction ready | < 3.0 s |
|
||||
| Live sample visual delay | < 2.5 s at 2 s interval |
|
||||
| Drag/resize frame behavior | no sustained visible jank |
|
||||
| 24 h wallboard heap | bounded; no monotonic leak |
|
||||
| Active subscriptions after navigation | returns to expected baseline |
|
||||
| Large table scroll | responsive with virtualization |
|
||||
|
||||
Record browser, hardware and network.
|
||||
|
||||
## API
|
||||
|
||||
| Metric | Target |
|
||||
|---|---:|
|
||||
| P95 cached/config API | < 250 ms |
|
||||
| P95 24 h bounded range query | < 750 ms excluding unavailable upstream |
|
||||
| P95 inventory list | < 500 ms at target scale |
|
||||
| WebSocket reconnect | automatic within 10 s under normal recovery |
|
||||
| Error response | bounded and correlated |
|
||||
|
||||
## Resource envelope
|
||||
|
||||
Initial production goals, to validate:
|
||||
- API/worker/agent combined idle memory should remain reasonable for Unraid;
|
||||
- CPU near idle outside query/evaluation bursts;
|
||||
- database growth predictable under retention;
|
||||
- no unbounded goroutines, queues, caches or event payloads.
|
||||
|
||||
Do not invent a pass. Record actual values and refine budgets through an ADR if hardware/source constraints provide evidence.
|
||||
|
||||
## Query limits
|
||||
|
||||
- max series and points per request;
|
||||
- max concurrent upstream requests;
|
||||
- step adjusted to viewport/time range;
|
||||
- heavy query rejection with guidance;
|
||||
- metadata/label enumeration bounded.
|
||||
|
||||
## Tests
|
||||
|
||||
- frontend bundle analysis;
|
||||
- Lighthouse or equivalent where meaningful;
|
||||
- scripted dashboard load;
|
||||
- WebSocket fan-out/load;
|
||||
- Prometheus slow/error injection;
|
||||
- real wallboard soak of at least 17 hours under the explicit M10-14
|
||||
product-owner duration decision;
|
||||
- worker/probe concurrency;
|
||||
- database query plans for large lists;
|
||||
- restart/recovery under load.
|
||||
@@ -0,0 +1,121 @@
|
||||
# Quality gates
|
||||
|
||||
## Per task
|
||||
|
||||
- Deliverables exist.
|
||||
- Acceptance checks pass.
|
||||
- Formatting/lint/type checks for changed code pass.
|
||||
- Relevant unit/integration/browser tests pass.
|
||||
- Diff review complete.
|
||||
- Docs/contracts updated.
|
||||
- Evidence summary complete.
|
||||
- No introduced secret or critical security issue.
|
||||
- A production Go package has at least one non-test importer reachable from a binary (`cmd/api`, `cmd/worker`, `cmd/agent`, `cmd/migrate`), verified by `python tools/check_wiring.py`; a task may not be marked done while its deliverable is unreachable, unless it is allowlisted in `tools/wiring_allowlist.json` with a reason and a tracking task id.
|
||||
- State updated via `projectctl`.
|
||||
|
||||
## Per milestone
|
||||
|
||||
- Every milestone task done.
|
||||
- Full milestone test set passes.
|
||||
- Architecture drift review.
|
||||
- Dependency/security/license check.
|
||||
- TODO/FIXME/skipped-test/debug scan.
|
||||
- Wiring/reachability scan (`python tools/check_wiring.py`): no package this milestone claims to deliver is unreachable and unallowlisted.
|
||||
- Migration/restart behavior where relevant.
|
||||
- UX/accessibility check for user-visible milestones.
|
||||
- Performance check for hot paths.
|
||||
- Milestone evidence index.
|
||||
- `python tools/projectctl.py gate <MILESTONE>` passes.
|
||||
|
||||
## M0 gate
|
||||
|
||||
- Repository/tooling/server discovery recorded.
|
||||
- Existing services/ports/networks/volumes/monitoring sources inventoried.
|
||||
- Backups/rollback plan for touched configs.
|
||||
- Current versions/capabilities verified from primary sources.
|
||||
- Architecture/security baseline reviewed.
|
||||
- M1-M9 plan adjusted to facts.
|
||||
- No destructive production change.
|
||||
|
||||
## M1 gate
|
||||
|
||||
- Clean local build.
|
||||
- API/web/worker/database start and health.
|
||||
- Migrations empty/restart/repeat.
|
||||
- Auth/RBAC skeleton and audit.
|
||||
- CI-equivalent checks.
|
||||
- No secrets.
|
||||
- Compose isolation/hardening baseline.
|
||||
|
||||
## M2 gate
|
||||
|
||||
- Prometheus and Unraid/mock adapters.
|
||||
- Inventory entities/relations/source ownership.
|
||||
- Discovery idempotency and source failure safety.
|
||||
- Datasource health/freshness.
|
||||
- API and UI inventory.
|
||||
- Target-scale reconciliation test.
|
||||
|
||||
## M3 gate
|
||||
|
||||
- Full dashboard CRUD/versioning/import/export.
|
||||
- Grid edit and per-breakpoint layouts.
|
||||
- Widget catalog/config states.
|
||||
- Undo/redo/restore/concurrency.
|
||||
- Desktop/mobile accessibility.
|
||||
- Browser persistence/reload proof.
|
||||
|
||||
## M4 gate
|
||||
|
||||
- Semantic metrics and bounded query planner.
|
||||
- Historical/live charts.
|
||||
- WebSocket auth, dedup, backpressure, reconnect.
|
||||
- Stale/unknown.
|
||||
- Performance/load and leak baseline.
|
||||
- Query security tests.
|
||||
|
||||
## M5 gate
|
||||
|
||||
- Host/process/container/application coverage.
|
||||
- Restart loop and application aggregation.
|
||||
- Events/detail pages/top-N/status.
|
||||
- Failure scenarios and scale.
|
||||
|
||||
## M6 gate
|
||||
|
||||
- Array/disks/SMART/pools/shares/capacity.
|
||||
- Read-only safety.
|
||||
- Storage stale/unknown and alerts inputs.
|
||||
- Simulated degradation.
|
||||
- No real destructive test.
|
||||
|
||||
## M7 gate
|
||||
|
||||
- Probe engine and SSRF controls.
|
||||
- TLS/DNS/network/service history.
|
||||
- Dependencies/topology.
|
||||
- Container-running/service-down detection.
|
||||
- Suppression inputs.
|
||||
|
||||
## M8 gate
|
||||
|
||||
- Rule versions/state machine/hysteresis.
|
||||
- Unknown/silence/maintenance/suppression.
|
||||
- Notifications audit.
|
||||
- Incident grouping/timeline/notes.
|
||||
- Alert storm scenario produces expected grouping.
|
||||
- Concurrency/restart tests.
|
||||
|
||||
## M9/final gate
|
||||
|
||||
- Complete Dutch UX, mobile and wallboard.
|
||||
- Accessibility and performance budgets.
|
||||
- Security hardening/scans.
|
||||
- Real wallboard soak of at least 17 hours. This supersedes the original
|
||||
24-hour duration only through the explicit product-owner decision recorded
|
||||
for M10-14 on 2026-08-11; all other continuity and performance budgets remain.
|
||||
- Backup/restore and upgrade/rollback.
|
||||
- Clean-room install.
|
||||
- Production deployment, restart and smoke.
|
||||
- Final requirement/evidence matrix.
|
||||
- Runbook and current state accurate.
|
||||
@@ -0,0 +1,137 @@
|
||||
# Test strategy
|
||||
|
||||
## Test pyramid and evidence
|
||||
|
||||
Tests prove behavior at the cheapest reliable level, but critical workflows require end-to-end proof.
|
||||
|
||||
## 1. Unit tests
|
||||
|
||||
Required for:
|
||||
- status aggregation;
|
||||
- freshness/staleness;
|
||||
- metric transformations and units;
|
||||
- query limit calculations;
|
||||
- inventory identity/reconciliation;
|
||||
- dashboard validation/migration/editor reducer;
|
||||
- alert state machine, hysteresis and suppression;
|
||||
- incident correlation rules;
|
||||
- authorization policy;
|
||||
- probe target validation/SSRF rules;
|
||||
- formatting/localization.
|
||||
|
||||
Use deterministic clocks and fixtures.
|
||||
|
||||
## 2. Contract tests
|
||||
|
||||
For:
|
||||
- Prometheus responses/errors/warnings;
|
||||
- Unraid API capabilities and payload variants;
|
||||
- agent protocol;
|
||||
- OIDC claims;
|
||||
- notification connectors;
|
||||
- the isolated real-stack gate (`scripts/integration-smoke.ps1`) for collector → PostgreSQL → API → UI and alert → webhook delivery;
|
||||
- dashboard/live/event schemas.
|
||||
|
||||
Captured fixtures must be redacted and versioned.
|
||||
|
||||
## 3. Integration tests
|
||||
|
||||
Use real PostgreSQL through Testcontainers or equivalent.
|
||||
|
||||
Cover:
|
||||
- migrations from empty and prior versions;
|
||||
- transaction/concurrency;
|
||||
- optimistic locking;
|
||||
- discovery/reconciliation;
|
||||
- alert evaluation/outbox;
|
||||
- backup/restore;
|
||||
- API authorization;
|
||||
- WebSocket persistence/reconnect interactions where practical.
|
||||
|
||||
Prometheus and Unraid can use deterministic simulators/fake servers, plus optional non-destructive live contract checks.
|
||||
|
||||
## 4. End-to-end browser tests
|
||||
|
||||
Playwright core flows:
|
||||
- login/session;
|
||||
- overview healthy/degraded/unknown;
|
||||
- dashboard create/edit/drag/resize/config/save/reload;
|
||||
- version restore and conflict;
|
||||
- time range and cross-filter;
|
||||
- entity drill-down;
|
||||
- alert acknowledge/silence;
|
||||
- incident view/note;
|
||||
- mobile navigation;
|
||||
- wallboard reconnect;
|
||||
- permission differences;
|
||||
- source failure and recovery.
|
||||
|
||||
Run desktop and mobile viewports. Add a wallboard viewport.
|
||||
|
||||
## 5. Accessibility
|
||||
|
||||
Automated checks on every core route/state:
|
||||
- desktop and mobile;
|
||||
- keyboard flow;
|
||||
- focus after modal/drawer/drag alternative;
|
||||
- status without color;
|
||||
- chart summary/alternative;
|
||||
- reduced motion.
|
||||
|
||||
Manual spot checks for screen-reader naming and dashboard editor keyboard behavior.
|
||||
|
||||
## 6. Performance/load/soak
|
||||
|
||||
- API benchmarks and P95 load tests.
|
||||
- WebSocket clients/subscriptions/fan-out.
|
||||
- Query dedup/cache behavior.
|
||||
- 24-hour wallboard heap/subscription/resource soak.
|
||||
- 150 container/40 disk/300 probe fixture scale.
|
||||
- slow Prometheus/database/agent recovery.
|
||||
- frontend rendering with maximum supported widgets.
|
||||
|
||||
## 7. Security
|
||||
|
||||
- RBAC matrix per endpoint and WebSocket message.
|
||||
- OIDC state/nonce/issuer/audience.
|
||||
- CSRF/cookie/origin.
|
||||
- SSRF, redirect, DNS rebinding, metadata targets.
|
||||
- XSS through names/events/import/Markdown.
|
||||
- query template and raw query limits.
|
||||
- rate/body/message limits.
|
||||
- secret scan and diagnostic redaction.
|
||||
- dependency/image/static scan.
|
||||
- compose mounts/capabilities/network exposure.
|
||||
- backup contents.
|
||||
|
||||
## 8. Failure simulation
|
||||
|
||||
Use `fixtures/scenarios/`:
|
||||
- stale Prometheus;
|
||||
- source disconnect;
|
||||
- CPU saturation;
|
||||
- memory pressure/OOM;
|
||||
- container restart loop;
|
||||
- service down while container runs;
|
||||
- disk temperature;
|
||||
- SMART warning;
|
||||
- cache/pool pressure;
|
||||
- array degradation fixture;
|
||||
- DNS/gateway failure;
|
||||
- UPS on battery;
|
||||
- WebSocket slow client/reconnect;
|
||||
- database restart.
|
||||
|
||||
Never induce destructive real faults.
|
||||
|
||||
## 9. Gate behavior
|
||||
|
||||
A failing required test:
|
||||
- keeps task/milestone incomplete;
|
||||
- is diagnosed and repaired;
|
||||
- may be quarantined only for a proven external nondeterministic issue, with owner, expiry and alternate evidence;
|
||||
- is never simply deleted or skipped.
|
||||
|
||||
## 10. Clean-room
|
||||
|
||||
Final release is built/deployed from a clean checkout using documented inputs, no developer `.env`, caches or untracked files. Migrations, seed/default dashboards, auth config, health, smoke and restart are verified.
|
||||
Reference in New Issue
Block a user