# Test strategy

## Test pyramid and evidence

Tests prove behavior at the cheapest reliable level, but critical workflows require end-to-end proof.

## 1. Unit tests

Required for:
- status aggregation;
- freshness/staleness;
- metric transformations and units;
- query limit calculations;
- inventory identity/reconciliation;
- dashboard validation/migration/editor reducer;
- alert state machine, hysteresis and suppression;
- incident correlation rules;
- authorization policy;
- probe target validation/SSRF rules;
- formatting/localization.

Use deterministic clocks and fixtures.

## 2. Contract tests

For:
- Prometheus responses/errors/warnings;
- Unraid API capabilities and payload variants;
- agent protocol;
- OIDC claims;
- notification connectors;
- the isolated real-stack gate (`scripts/integration-smoke.ps1`) for collector → PostgreSQL → API → UI and alert → webhook delivery;
- dashboard/live/event schemas.

Captured fixtures must be redacted and versioned.

## 3. Integration tests

Use real PostgreSQL through Testcontainers or equivalent.

Cover:
- migrations from empty and prior versions;
- transaction/concurrency;
- optimistic locking;
- discovery/reconciliation;
- alert evaluation/outbox;
- backup/restore;
- API authorization;
- WebSocket persistence/reconnect interactions where practical.

Prometheus and Unraid can use deterministic simulators/fake servers, plus optional non-destructive live contract checks.

## 4. End-to-end browser tests

Playwright core flows:
- login/session;
- overview healthy/degraded/unknown;
- dashboard create/edit/drag/resize/config/save/reload;
- version restore and conflict;
- time range and cross-filter;
- entity drill-down;
- alert acknowledge/silence;
- incident view/note;
- mobile navigation;
- wallboard reconnect;
- permission differences;
- source failure and recovery.

Run desktop and mobile viewports. Add a wallboard viewport.

## 5. Accessibility

Automated checks on every core route/state:
- desktop and mobile;
- keyboard flow;
- focus after modal/drawer/drag alternative;
- status without color;
- chart summary/alternative;
- reduced motion.

Manual spot checks for screen-reader naming and dashboard editor keyboard behavior.

## 6. Performance/load/soak

- API benchmarks and P95 load tests.
- WebSocket clients/subscriptions/fan-out.
- Query dedup/cache behavior.
- 24-hour wallboard heap/subscription/resource soak.
- 150 container/40 disk/300 probe fixture scale.
- slow Prometheus/database/agent recovery.
- frontend rendering with maximum supported widgets.

## 7. Security

- RBAC matrix per endpoint and WebSocket message.
- OIDC state/nonce/issuer/audience.
- CSRF/cookie/origin.
- SSRF, redirect, DNS rebinding, metadata targets.
- XSS through names/events/import/Markdown.
- query template and raw query limits.
- rate/body/message limits.
- secret scan and diagnostic redaction.
- dependency/image/static scan.
- compose mounts/capabilities/network exposure.
- backup contents.

## 8. Failure simulation

Use `fixtures/scenarios/`:
- stale Prometheus;
- source disconnect;
- CPU saturation;
- memory pressure/OOM;
- container restart loop;
- service down while container runs;
- disk temperature;
- SMART warning;
- cache/pool pressure;
- array degradation fixture;
- DNS/gateway failure;
- UPS on battery;
- WebSocket slow client/reconnect;
- database restart.

Never induce destructive real faults.

## 9. Gate behavior

A failing required test:
- keeps task/milestone incomplete;
- is diagnosed and repaired;
- may be quarantined only for a proven external nondeterministic issue, with owner, expiry and alternate evidence;
- is never simply deleted or skipped.

## 10. Clean-room

Final release is built/deployed from a clean checkout using documented inputs, no developer `.env`, caches or untracked files. Migrations, seed/default dashboards, auth config, health, smoke and restart are verified.