M48: harden demo operations and offsite recovery
MobilityOps acceptance / backend (push) Failing after 20s
MobilityOps acceptance / frontend (push) Successful in 28s
MobilityOps acceptance / e2e (push) Skipped

This commit is contained in:
NuklearRabbit
2026-08-21 22:17:49 +02:00
parent a24098c583
commit 00191e9b54
28 changed files with 1136 additions and 332 deletions
+13
View File
@@ -56,10 +56,19 @@ unique `GRAFANA_ADMIN_PASSWORD` before first start. Provisioning includes the Mo
overview dashboard and alerts for API/database outage, 5xx rate, p95 latency, real outbox
backlog and real outbox failures. Synthetic retry scenarios never trigger outbox alerts.
The hourly `live-canary.yml` workflow independently checks HTTPS readiness, the TLS
certificate horizon and the non-destructive Chromium/Firefox production journey. Configure
the optional `LIVE_CANARY_HEARTBEAT_URL` repository secret to make missed scheduled jobs
visible in an external dead-man monitor. A successful app release does not restart the
database, backup or monitoring services; use `deploy/unraid/refresh-infrastructure.sh`
deliberately when those definitions change.
Never execute `pytest` inside the deployed API container: the acceptance fixtures reset
their database deliberately. `make test` uses `compose.test.yaml`, a fixed
`mobilityops-test` Compose project and its own disposable PostgreSQL volume, and removes
that project on success or failure. The Gitea workflow uses the same isolation boundary.
CI also runs `scripts/run-readonly-load-smoke.py` against persisted list/dashboard routes;
the gate requires zero HTTP errors and a p95 below 1.5 seconds at its bounded concurrency.
## Operational mode (non-demo login)
@@ -242,3 +251,7 @@ Publish the scheduled quality-scan workflow the same way:
independently-authenticated API surface;
- failed demo experiment: Operations Manager reset (`POST /api/v1/demo/reset`) restores
the deterministic seed, including all named S1S6 demo scenarios.
- loss of the Unraid host: recover the latest OneDrive dump only after its downloaded
checksum and `pg_restore --list` pass; use the guarded restore command above. OneDrive
synchronization health requires both a daily verified round trip and a weekly real
restore drill. OAuth setup is documented in `deploy/unraid/README.md`.
+4 -2
View File
@@ -15,8 +15,10 @@ retention values. Operations Managers are the only role permitted to export or a
- Audit events are configured for 2,555 days by default. They are immutable operational
evidence; changing this period requires legal approval and a separate, audited purge
implementation. MobilityOps reports the policy but never silently deletes audit data.
- Database backups default to 30 days with at least seven newest recovery points. A
configured secondary mount must follow the same policy.
- Database backups default to 30 days with at least seven newest recovery points. The demo's
optional OneDrive off-site copy contains the same synthetic dataset and follows the same
retention intent; remove that folder when retiring the demo. Real personal data remains
out of scope and must not be introduced merely because cloud backup is available.
- Logs are bounded by Docker rotation. They must not contain request bodies, passwords,
OIDC tokens or customer fields.