M7: portfolio polish and final acceptance

Automated migrations on container startup (backend/entrypoint.sh), scripted n8n workflow activation (make n8n-setup), Playwright E2E test covering the full 9-step demo script (verified passing against the live stack, including the previously-unverified 360px responsive layout), evidence screenshots of all main pages, architecture diagram, and artifacts/evidence/final-summary.md with commit/commands/test counts/RAGcore and n8n evidence/MCP sample calls/known limitations/portfolio wording. Verified the complete clean-checkout path from a genuinely wiped-volumes state: automatic migrations, seed, 66 backend tests passing, and a live S1 return round-tripped through a freshly-activated n8n instance. Added .gitattributes to force LF line endings on shell scripts, preventing a real cross-platform breakage of entrypoint.sh's shebang.
This commit is contained in:
NuklearRabbit
2026-08-01 23:28:28 +02:00
parent c5b7e21f81
commit 108b5d04fc
23 changed files with 612 additions and 25 deletions
+4
View File
@@ -0,0 +1,4 @@
* text=auto eol=lf
*.sh text eol=lf
*.png binary
*.jpg binary
+17 -1
View File
@@ -1,4 +1,4 @@
.PHONY: up down logs test seed reset lint
.PHONY: up down logs test lint seed reset n8n-setup demo e2e
up:
docker compose up --build -d
@@ -21,3 +21,19 @@ seed:
reset:
docker compose down -v
docker compose up --build -d
# One-time per environment: imports and activates the n8n return-processing workflow.
# The n8n owner account itself cannot be scripted safely and must be created once at
# http://localhost:5678/setup (any email/password, no verification required) before
# this target's activation takes effect. See docs/17-runbook.md.
n8n-setup:
docker compose exec n8n n8n import:workflow --input=//imports/mobilityops-return-processing.json
docker compose exec n8n n8n publish:workflow --id=mobilityops-return-processing
docker compose restart n8n
# Full deterministic demo bootstrap: build, migrate (automatic on api startup), seed.
demo: up
docker compose exec api python -m app.cli seed --reset
e2e:
cd frontend && npx playwright test
+18 -2
View File
@@ -2,7 +2,7 @@
## Current milestone
M6 — complete. Starting M7 next.
M7 — complete. All milestones (M0M7) done. See `artifacts/evidence/final-summary.md` for the acceptance evidence.
## Locked decisions
@@ -126,4 +126,20 @@ None. External service credentials may be absent; use the documented demo/degrad
## Exact next action
Start M7 (portfolio polish and final acceptance): read `docs/06-ui-ux.md`, `docs/14-testing-and-acceptance.md`, `docs/16-portfolio-case-study.md`. Work through `docs/14-testing-and-acceptance.md`'s clean-checkout acceptance list item by item from a genuinely fresh checkout (new clone, `cp .env.example .env`, documented bootstrap commands only) rather than this session's already-running/already-seeded stack — several manual one-time steps accumulated this session (n8n owner setup + workflow import/publish/restart, see M4 notes) that must either be scripted into the bootstrap or written up precisely in `docs/17-runbook.md`, since a true clean checkout won't have them done yet. Also needed: Playwright end-to-end test covering the full documented 5-minute demo script (login → dashboard → S1 return → S2 merge → S4 overlap already-visible → S6 knowledge question → audit → responsive nav at 360px — the ≤360px layout was flagged as visually unverified back in M1 and should finally be checked here), screenshots of the seven main pages, an architecture diagram, `artifacts/evidence/final-summary.md` per the doc's exact required contents (commit/tag, commands, test counts — currently 66 backend tests across M0M6 — screenshots, RAGcore success/unavailable evidence, n8n success/retry evidence, MCP sample calls, known limitations, truthful portfolio wording), and a final pass removing any rough edges/placeholders across the UI.
### M7 — Portfolio polish and final acceptance
- **Automated clean-checkout migrations**: `backend/entrypoint.sh` now runs `alembic upgrade head` before starting uvicorn (Dockerfile `CMD` changed from `uvicorn ...` to `./entrypoint.sh`). Verified with a true `docker compose down -v` (all volumes wiped) → `docker compose up --build -d` → all 11 tables present, `/health` and web both green, all 66 backend tests pass, with zero manual migration step.
- **n8n one-time setup scripted where it can be**: `make n8n-setup` runs the import/publish/restart sequence (previously three manual commands discovered ad hoc in M4). The owner-account creation itself cannot be scripted safely (it's an interactive one-time step in n8n 2.x's own onboarding, not a MobilityOps concern) — documented precisely in the rewritten `docs/17-runbook.md`, including the exact URL and that no email verification is required. Re-ran this full sequence from the wiped-volumes state this session and confirmed the S1 return → outbox → live n8n → callback → `succeeded` round trip works on a genuinely clean checkout, not just the already-provisioned stack from M0M6.
- **Playwright E2E** (`frontend/e2e/demo.spec.ts`, `frontend/playwright.config.ts`): one test automating the full 9-step documented demo script end-to-end against the live stack — login, dashboard metrics, open `BK-DEMO-RETURN`, register an odometer-regression return (S1), verify the quality issue + queued automation event, merge the duplicate-customer scenario (S2), ask the damage question and verify both expected source citations (S6), inspect audit entries, and verify responsive nav + no horizontal overflow at 360px width. **Passing.** This also resolves the "≤360px layout visually unverified" gap flagged back in M1 — verified both by this test's overflow assertion and by the `9-mobile-dashboard.png` screenshot (nav wraps into rows, metric tiles collapse to a 2-column grid, no horizontal scroll).
- Added `frontend/e2e/_capture-screenshots.spec.ts` as evidence-generation tooling (underscore-prefixed, excluded from the default `playwright test` / `make e2e` run via `testIgnore` in the config — it calls `demo/reset`, which a real regression test shouldn't do as a side effect). Captured all 9 screenshots into `artifacts/evidence/screenshots/`.
- Wrote `artifacts/evidence/architecture.md` (mermaid, as-built — distinguishes verified-live components from implemented-but-never-reached-a-real-instance ones, i.e. RAGcore and the MCP Hub) and `artifacts/evidence/final-summary.md` (commit, exact commands, test counts, screenshot index, RAGcore success/unavailable evidence — including a live-demonstrated unavailable case against an unreachable host, not just the unit test — n8n success/retry evidence, MCP sample calls, known limitations, truthful portfolio wording per `docs/16-portfolio-case-study.md`'s template).
- Updated `README.md` (dropped stale "minimal bootable scaffold, not the finished application" wording and the old two-line quickstart in favor of `make demo` + a pointer to the runbook) and `docs/17-runbook.md` (full rewrite: exact bootstrap, n8n one-time setup, verification commands, required operational checks, recovery expectations).
- Final placeholder/dead-UI sweep: `grep`'d the full `frontend/src` and `backend/app` trees for scaffold/TODO/FIXME/"must be replaced" markers — none found. `FILE_INDEX.md` was left as-is; it's the original build-pack's archive-completeness manifest (a historical snapshot), not a living index that needs to track every file added since — updating it would misrepresent what it's for.
- Commands run and verified from this checkout (this milestone, cumulative across the whole build):
- `docker compose run --rm api pytest -q`**66 passed**, ruff clean.
- `cd frontend && npm run build` — clean.
- `cd frontend && npx playwright test`**1 passed** (full demo script, live stack).
- Full clean-checkout drill: `docker compose down -v``docker compose up --build -d``docker compose exec api python -m app.cli seed --reset``docker compose run --rm api pytest -q` (66 passed) → n8n owner setup + `make n8n-setup` → live S1 return round-tripped through the real n8n instance to `succeeded`.
## Definition of done
All eight milestones (M0M7) are complete. `docs/14-testing-and-acceptance.md`'s clean-checkout acceptance list has been walked item by item against a genuinely wiped-volumes checkout this session (see M7 notes above), and `artifacts/evidence/final-summary.md` contains the final evidence summary this file's own instructions (`CLAUDE.md`) require. The two items not fully closed — a live RAGcore instance and a live ITWorx MCP Hub instance — were never reachable in this environment; both integrations are implemented, unit/contract-tested, and documented as such rather than claimed as verified.
+11 -5
View File
@@ -33,19 +33,25 @@ It is not an ERP, CRM, accounting package, public booking site, payment system o
- `knowledge/` — fictitious source documents for the MobilityOps RAGcore workspace.
- `seed/` — deterministic synthetic dataset and generator.
- `n8n/` — importable workflow definitions.
- `backend/` and `frontend/` — minimal bootable scaffold, not the finished application.
- `backend/` — FastAPI/SQLAlchemy/Alembic API.
- `frontend/` — React/TypeScript/Vite web app, including the Playwright end-to-end suite (`frontend/e2e/`).
- `artifacts/evidence/` — final acceptance evidence (screenshots, architecture, `final-summary.md`).
## Local scaffold
## Quickstart
```bash
cp .env.example .env
docker compose up --build
make demo
```
Expected scaffold endpoints:
This builds and starts the full stack (migrations run automatically) and loads the
deterministic demo dataset. See `docs/17-runbook.md` for the one-time n8n workflow setup
required for the automation demo, and the full operational runbook.
Endpoints:
- Web: `http://localhost:1228`
- API health: `http://localhost:8128/health`
- n8n: `http://localhost:5678`
The finished project must keep these defaults configurable.
All defaults are configurable via `.env` (see `.env.example`).
+63
View File
@@ -0,0 +1,63 @@
# MobilityOps — as-built architecture
```mermaid
flowchart TB
subgraph Browser
UI["MobilityOps Web<br/>React + TypeScript"]
end
subgraph MobilityOps["MobilityOps (this repo)"]
API["FastAPI backend<br/>/api/v1/*"]
DISPATCH["Outbox dispatcher<br/>background thread"]
DB[(PostgreSQL)]
API --> DB
DISPATCH --> DB
end
subgraph External["External central services"]
N8N["n8n<br/>return-processing workflow"]
RAGDEMO["Demo KnowledgeProvider<br/>TF-IDF extractive, local files"]
RAGCORE["RAGcore<br/>(adapter built, no live instance)"]
HUB["ITWorx MCP Hub<br/>(endpoints built, no live instance)"]
end
UI -->|session cookie| API
API -->|GroundedAnswer| RAGDEMO
API -.->|configurable, unavailable-safe| RAGCORE
DISPATCH -->|POST vehicle.returned.v1| N8N
N8N -->|callback, X-Service-Token| API
HUB -.->|X-Service-Token, read-only| API
classDef unverified stroke-dasharray: 5 5;
class RAGCORE,HUB unverified;
```
Dashed boxes/arrows are implemented and unit/contract-tested but were never exercised
against a live instance in this environment (no reachable RAGcore or ITWorx MCP Hub).
Solid boxes were verified end-to-end, including a real n8n instance.
## Component responsibility (unchanged from `docs/03-architecture.md`)
| Component | Owns |
|---|---|
| MobilityOps | vehicles, customers, bookings, inspections, data-quality issues, audit, outbox/delivery state |
| RAGcore | procedure retrieval and grounded answers (demo provider substitutes locally) |
| ITWorx MCP Hub | MCP transport, tool publication, central tool-call audit |
| n8n | post-commit secondary orchestration only — never the source of truth for vehicle state |
## Reliability boundaries verified in this build
1. **Return commits atomically with its outbox event**`app/services/returns.py`, one
transaction; verified by `test_concurrent_returns_only_one_succeeds` (real Postgres row
locking, not mocked).
2. **Outbox delivery is at-least-once, idempotent by event ID** — verified live: the n8n
callback checks for an existing `AuditEvent` by event ID before recording a second time.
3. **RAGcore failure disables knowledge answers only**`RAGcoreKnowledgeProvider` degrades
to `unavailable`; the rest of the app is unaffected because the knowledge router is the
only consumer.
4. **MCP Hub failure does not affect the web application** — the four MCP provider
endpoints are a separate authenticated surface (`X-Service-Token`), invisible to the
browser-facing API/UI.
5. **n8n failure leaves events pending with bounded retries** — verified live: a seeded
`failed` event, retried through the UI, was picked up by the background dispatcher and
delivered through the real n8n instance within one poll cycle.
+177
View File
@@ -0,0 +1,177 @@
# MobilityOps — final acceptance evidence
## Commit
Built on top of commit `c5b7e21f81694f0339ad31e3bf044db952d0fbe0` (M6, "implement ITWorx
MCP Hub publication"). This evidence file and the rest of M7's polish are committed as
`M7: portfolio polish and final acceptance` — run `git log --oneline` for the exact hash.
## Exact commands (clean checkout)
```bash
git clone <repo> && cd MobilityOps
cp .env.example .env
make demo # docker compose up --build -d ; migrations run automatically ; seed --reset
```
One-time n8n setup (see `docs/17-runbook.md` for full detail — this cannot be scripted
end-to-end because it requires a one-time owner account created through n8n's web UI):
```bash
# open http://localhost:5678/setup in a browser, create any owner account
make n8n-setup
```
Verification:
```bash
docker compose run --rm api pytest -q # 66 passed
docker compose run --rm api ruff check . # All checks passed
cd frontend && npm run build # clean tsc + vite build
cd frontend && npx playwright test # 1 passed (full 5-minute demo script)
```
## Test counts
- **Backend**: 66 tests passing (`pytest`), 0 skipped, 0 failed. Ruff clean. Coverage by
area: seed determinism (2), auth/roles (4), dashboard (3), vehicles (4), bookings (3),
return workflow incl. real concurrent-submission test (9), data quality incl. S2/S4
scenarios (10), audit (2), n8n dispatcher incl. malformed-payload regression (6),
n8n callback idempotency (3), workflows/retry (4), knowledge incl. S6 scenario (7),
MCP provider endpoints (8), health (1).
- **Frontend**: `npm run build` — clean TypeScript + Vite build, zero errors.
- **End-to-end**: 1 Playwright test (`frontend/e2e/demo.spec.ts`) automating the full
documented 5-minute demo script (login → dashboard → S1 return → S2 merge → S6 knowledge
question → audit → 360px responsive check) — **passing** against the live stack.
## Screenshots of the seven main pages
Captured live against the deterministic seed (`artifacts/evidence/screenshots/`,
via `frontend/e2e/_capture-screenshots.spec.ts`):
| # | Page | File |
|---|---|---|
| 1 | Login | `1-login.png` |
| 2 | Dashboard | `2-dashboard.png` |
| 3 | Vehicles | `3-vehicles.png` |
| 4 | Bookings | `4-bookings.png` |
| 5 | Data Quality | `5-data-quality.png` |
| 6 | Knowledge (grounded S6 answer) | `6-knowledge.png` |
| 7 | Automation | `7-automation.png` |
| — | Audit (bonus, 8th nav item) | `8-audit.png` |
| — | Dashboard at 360px (responsive proof) | `9-mobile-dashboard.png` |
## RAGcore evidence
**Success (demo provider, the one actually satisfying acceptance in this environment)**
S6 question against the real `/api/v1/knowledge/questions` endpoint:
```json
{
"answer": "Per \"Vehicle return procedure\" (v2.0), section \"1. Register the return\": Open the active booking and record the ending odometer, fuel level, cleanliness, visible damage, technical warnings and relevant notes.",
"evidence_state": "grounded",
"sources": [
{"document_id": "vehicle-return-procedure", "title": "Vehicle return procedure", "version": "2.0", "section": "1. Register the return", "excerpt": "..."},
{"document_id": "vehicle-return-procedure", "title": "Vehicle return procedure", "version": "2.0", "section": "3. Determine next state", "excerpt": "..."},
{"document_id": "damage-procedure", "title": "Damage handling procedure", "version": "1.3", "section": "1. Immediate actions", "excerpt": "..."}
],
"provider": "demo",
"correlation_id": "b50094b7-1c84-4e39-9055-1dc03e8fd1f8"
}
```
**Unavailable (RAGcore adapter, live-demonstrated against an unreachable host)**
`KNOWLEDGE_PROVIDER=ragcore`, `RAGCORE_BASE_URL=http://ragcore-not-reachable:9999`:
```
health: {'provider': 'ragcore', 'available': False, 'detail': 'RAGcore unavailable: ConnectError: ...', 'document_count': 0}
ask: {'answer': '', 'evidence_state': 'unavailable', 'sources': [], 'provider': 'ragcore', 'correlation_id': 'demo-correlation'}
```
No live RAGcore instance was reachable in this environment, so the adapter's actual
request/response contract against a real RAGcore is unverified beyond this
degrade-safely behavior — see `contracts/ragcore-contract-assumptions.md` and
`PROJECT_STATE.md`'s M5 notes.
## n8n evidence
**Success** — a real return registered on `BK-DEMO-RETURN`, delivered through the actual
n8n instance (not mocked), confirmed via `GET /api/v1/workflows`:
```json
{"event_id": "aa5dfeee-90ca-452a-bdd1-0a0b6d3dd63f", "event_type": "vehicle.returned.v1", "aggregate_ref": "BK-DEMO-RETURN", "status": "succeeded", "attempts": 2, "last_error": null}
```
(`attempts: 2` because the first delivery attempt landed while n8n was mid-restart from
the one-time workflow-activation step — the dispatcher's backoff-and-retry handled it
without any manual intervention, which is itself evidence of the retry behavior working.)
**Retry (S5 scenario)** — seeded `BK-H-0020` (event `00000000-...-0020`), initially
`failed` after 3 attempts with `"Synthetic connection timeout to n8n"`:
1. Before: `{"status": "failed", "attempts": 3, "last_error": "Synthetic connection timeout to n8n"}`
2. Operations Manager clicks Retry on `/automation`.
3. Within one ~3s dispatcher poll cycle, delivered through the live n8n instance.
4. After: `{"status": "succeeded", "attempts": 4, "last_error": null}`
## MCP tool sample calls
All four provider endpoints, authenticated with `X-Service-Token`:
```
$ curl -H "X-Service-Token: <token>" http://localhost:8128/api/v1/integrations/mcp/operations-summary
{"tenant":"northstar-mobility-demo","metrics":{"available":21,"rented":11,"cleaning":6,"maintenance":5,"blocked":7,"open_quality_issues":22,"pending_or_failed_workflows":1}}
$ curl -H "X-Service-Token: <token>" "http://localhost:8128/api/v1/integrations/mcp/attention-vehicles?minimum_severity=high&limit=3"
[{"vehicle_ref":"MO-016","severity":"high","rule_type":"booking_overlap",...},
{"vehicle_ref":"MO-016","severity":"high","rule_type":"vehicle_status_conflict",...},
{"vehicle_ref":"MO-031","severity":"high","rule_type":"missing_required_field",...}]
$ curl -H "X-Service-Token: <token>" http://localhost:8128/api/v1/integrations/mcp/vehicles/MO-016
{"public_ref":"MO-016","make":"Hymer","model":"Exsis","model_year":2021,"location":"Geel","operational_status":"available","odometer_km":30497,"next_service_km":40000,"open_quality_issue_count":2,"current_booking_ref":null}
$ curl -H "X-Service-Token: <token>" -X POST -d '{"question":"What must I do when a vehicle returns with damage?","max_sources":2}' http://localhost:8128/api/v1/integrations/mcp/search-knowledge
{"answer":"Per \"Vehicle return procedure\" ...","evidence_state":"grounded","sources":[...2 items...],"provider":"demo",...}
```
Auth verified: missing header → `422`; wrong token → `401`. All four calls confirmed
recorded in `GET /api/v1/audit?action=mcp_tool_request` with `actor_type: "service"`.
No live ITWorx MCP Hub instance was reachable in this environment — these are direct
calls to MobilityOps's own provider endpoints, not a Hub round trip.
## Known PoC limitations
- **RAGcore and ITWorx MCP Hub were never reachable in this build environment.** Both
integrations are implemented against best-effort/documented contracts and are
unit/contract-tested (including their failure-degradation paths), but neither was
verified against a real counterpart service. The demo `KnowledgeProvider` is what
actually satisfies the knowledge-assistant acceptance criteria here.
- **n8n requires a one-time manual owner-account setup** per fresh environment
(`docker compose down -v` wipes it) — this is a property of the n8n 2.x image itself
(`N8N_BASIC_AUTH_ACTIVE` no longer gates the UI), not something MobilityOps can bypass.
Documented precisely in `docs/17-runbook.md`; the workflow import/activation itself
*is* scripted (`make n8n-setup`).
- **Inspection public refs are a simple `count+1` sequence**, not gap-safe under true
concurrent writers — acceptable for this single-tenant demo, would need a DB sequence
for a multi-writer production system.
- **The five data-quality rules use simplified idempotency** — `(rule_type, entity_type,
entity_id)` while open, rather than the doc's literal evidence-fingerprint scheme — see
`PROJECT_STATE.md`'s M3 notes for the reasoning (the fingerprint scheme would have let
the scan double-report issues already present in the seeded CSV).
- **No production authentication** — demo login is an HMAC-signed session cookie tied to
two fixed seeded users, appropriate for a PoC, not a real identity provider.
## Portfolio wording (truthful)
MobilityOps is a working proof of concept, not a production system and not deployed for
any real company. All customers, vehicles, bookings, and documents are synthetic
(deterministically generated). The application logic it demonstrates is real: a
transactional vehicle-return workflow with idempotency and concurrency control tested
against real concurrent database transactions; five explainable, deterministic
data-quality rules with a working customer-merge UI; a background outbox dispatcher
verified end-to-end against a real n8n instance including failure/retry; a
TF-IDF-weighted extractive knowledge assistant that never fabricates answers; and four
read-only, audited, service-authenticated integration endpoints. RAGcore and the ITWorx
MCP Hub integrations are implemented and tested in isolation but were not verified
against live instances of those systems in this environment.
Binary file not shown.

After

Width:  |  Height:  |  Size: 34 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 116 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 244 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.2 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 141 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 112 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 36 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

+3 -2
View File
@@ -10,6 +10,7 @@ COPY backend/alembic.ini ./
COPY backend/tests ./tests
COPY seed ./seed
COPY knowledge ./knowledge
RUN pip install --no-cache-dir --no-deps -e .
COPY backend/entrypoint.sh ./entrypoint.sh
RUN pip install --no-cache-dir --no-deps -e . && chmod +x ./entrypoint.sh
EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
CMD ["./entrypoint.sh"]
+5
View File
@@ -0,0 +1,5 @@
#!/bin/sh
set -e
alembic upgrade head
exec uvicorn app.main:app --host 0.0.0.0 --port 8000
+73 -14
View File
@@ -1,28 +1,87 @@
# PoC runbook
## Bootstrap
## Bootstrap (clean checkout)
```bash
cp .env.example .env
make demo
```
`make demo` runs `docker compose up --build -d` (migrations run automatically on API
container startup, see `backend/entrypoint.sh`) and then seeds the deterministic dataset.
Equivalently, without `make`:
```bash
cp .env.example .env
docker compose up --build -d
docker compose exec api python -m app.cli seed --reset
```
Claude must replace this scaffold runbook with exact migration, seed, test and integration commands after implementation.
Verify:
```bash
curl http://localhost:8128/health # {"status":"ok",...}
curl -o /dev/null -w "%{http_code}\n" http://localhost:1228/ # 200
docker compose run --rm api pytest -q # all tests pass
docker compose run --rm api ruff check . # clean
```
## n8n automation (one-time per environment)
The n8n image used here (n8nio/n8n:latest, 2.x) requires an owner account before any
workflow — including webhook registration — works reliably; `N8N_BASIC_AUTH_ACTIVE` no
longer gates this. This is a one-time step per fresh `docker compose down -v`:
1. Open `http://localhost:5678/setup` and create an owner account (any email/password
meeting the 8+ characters / 1 number / 1 capital rule — no email verification is
required). Skip the optional survey/license-key dialogs that follow.
2. Import and activate the return-processing workflow:
```bash
make n8n-setup
```
which runs:
```bash
docker compose exec n8n n8n import:workflow --input=//imports/mobilityops-return-processing.json
docker compose exec n8n n8n publish:workflow --id=mobilityops-return-processing
docker compose restart n8n
```
(`n8n import:workflow` always leaves the workflow deactivated regardless of its
`"active"` field; `publish:workflow` + a restart is what actually activates it.)
Verify the full round trip:
```bash
# after logging in and registering any return via the UI or API
curl -b cookies.txt http://localhost:8128/api/v1/workflows | grep succeeded
```
A failed/offline n8n does not roll back the return — the outbox event simply stays
`pending`/`failed` and is safely retryable from the Automation page.
## Required operational checks
- API and web health;
- database migration level;
- pending/failed outbox count;
- RAGcore provider state;
- n8n connectivity;
- MCP provider endpoint authorization;
- deterministic demo reset.
- API and web health (`GET /health`, web root `200`);
- database migration level (`docker compose exec api alembic current`);
- pending/failed outbox count (Automation page, or `GET /api/v1/workflows?status=failed`);
- RAGcore provider state (`GET /api/v1/knowledge/status`; demo provider is always
`available`, RAGcore adapter reports `unavailable` when unreachable);
- n8n connectivity (`docker compose logs n8n`, or submit a return and watch `/automation`);
- MCP provider endpoint authorization (`curl` the four `/api/v1/integrations/mcp/*`
routes with and without a valid `X-Service-Token` — see
`artifacts/evidence/final-summary.md` for sample calls);
- deterministic demo reset (`POST /api/v1/demo/reset` as Operations Manager, or `make seed`).
## Recovery expectations
- database restart: application reconnects;
- n8n outage: events remain pending and retryable;
- RAGcore outage: knowledge shows unavailable, operations continue;
- MCP Hub outage: web application unaffected;
- failed demo experiment: Operations Manager reset restores the seed.
- database restart: application reconnects (SQLAlchemy connection pool, `pool_pre_ping=True`);
- n8n outage: events remain `pending` and are retried with exponential backoff, then
`failed` after 5 attempts and safely retryable from `/automation`;
- RAGcore outage: `/knowledge` shows `unavailable`, all operational pages continue working;
- MCP Hub outage: the web application is unaffected — MCP endpoints are a separate,
independently-authenticated API surface;
- failed demo experiment: Operations Manager reset (`POST /api/v1/demo/reset`) restores
the deterministic seed, including all named S1S6 demo scenarios.
+53
View File
@@ -0,0 +1,53 @@
import { expect, test } from "@playwright/test";
// One-off tooling to capture evidence screenshots for artifacts/evidence/final-summary.md.
// Not part of the regular test suite (prefixed with `_` and excluded from CI runs).
const OUT = "../artifacts/evidence/screenshots";
test("capture the seven main pages", async ({ page, request }) => {
await request.post("http://localhost:8128/api/v1/demo/login", {
data: { role: "operations_manager" },
});
await request.post("http://localhost:8128/api/v1/demo/reset");
await page.goto("/login");
await page.screenshot({ path: `${OUT}/1-login.png` });
await page.getByRole("button", { name: "Open as Operations Manager" }).click();
await expect(page.getByRole("heading", { name: "Operational metrics" })).toBeVisible();
await page.screenshot({ path: `${OUT}/2-dashboard.png`, fullPage: true });
await page.goto("/vehicles");
await expect(page.locator(".data-table")).toBeVisible();
await page.screenshot({ path: `${OUT}/3-vehicles.png`, fullPage: true });
await page.goto("/bookings");
await expect(page.locator(".data-table")).toBeVisible();
await page.screenshot({ path: `${OUT}/4-bookings.png`, fullPage: true });
await page.goto("/data-quality");
await expect(page.locator(".data-table")).toBeVisible();
await page.screenshot({ path: `${OUT}/5-data-quality.png`, fullPage: true });
await page.goto("/knowledge");
await page
.getByPlaceholder(/What must I do when a vehicle returns with damage/)
.fill("What must I do when a vehicle returns with damage?");
await page.getByRole("button", { name: "Ask" }).click();
await expect(page.getByText("Grounded in cited procedures")).toBeVisible();
await page.screenshot({ path: `${OUT}/6-knowledge.png`, fullPage: true });
await page.goto("/automation");
await expect(page.locator(".data-table")).toBeVisible();
await page.screenshot({ path: `${OUT}/7-automation.png`, fullPage: true });
await page.goto("/audit");
await expect(page.locator(".data-table")).toBeVisible();
await page.screenshot({ path: `${OUT}/8-audit.png`, fullPage: true });
await page.setViewportSize({ width: 360, height: 800 });
await page.goto("/dashboard");
await expect(page.getByRole("heading", { name: "Operational metrics" })).toBeVisible();
await page.screenshot({ path: `${OUT}/9-mobile-dashboard.png` });
});
+98
View File
@@ -0,0 +1,98 @@
import { expect, test, type APIRequestContext } from "@playwright/test";
const API_BASE = process.env.MOBILITYOPS_API_URL ?? "http://localhost:8128";
async function resetDemoData(request: APIRequestContext) {
const login = await request.post(`${API_BASE}/api/v1/demo/login`, {
data: { role: "operations_manager" },
});
expect(login.ok()).toBeTruthy();
const reset = await request.post(`${API_BASE}/api/v1/demo/reset`);
expect(reset.ok()).toBeTruthy();
}
test.describe.configure({ mode: "serial" });
test("five-minute demo script end to end", async ({ page, request }) => {
// Reset via a throwaway API session so the UI test starts from the deterministic seed
// regardless of what earlier test runs mutated (S1 return, S2 merge, etc.).
await resetDemoData(request);
await test.step("1. login as Operations Manager", async () => {
await page.goto("/login");
await expect(page.getByText("Synthetic demo environment")).toBeVisible();
await page.getByRole("button", { name: "Open as Operations Manager" }).click();
await expect(page).toHaveURL(/\/dashboard$/);
});
await test.step("2. verify dashboard metrics are loaded", async () => {
await expect(page.getByRole("heading", { name: "Operational metrics" })).toBeVisible();
const metricValues = page.locator(".metric-value");
await expect(metricValues.first()).toBeVisible();
const values = await metricValues.allTextContents();
expect(values.length).toBeGreaterThan(0);
expect(values.some((v) => Number(v) > 0)).toBeTruthy();
});
await test.step("3. open active demo booking", async () => {
await page.goto("/bookings/BK-DEMO-RETURN");
await expect(page.getByRole("heading", { name: "BK-DEMO-RETURN" })).toBeVisible();
await expect(page.getByText("active", { exact: true })).toBeVisible();
});
await test.step("4. register an odometer-regression return (S1)", async () => {
const vehicleOdometerText = await page
.locator(".detail-grid div", { hasText: "Start odometer" })
.locator("dd")
.textContent();
const startOdometer = parseInt((vehicleOdometerText ?? "0").replace(/\D/g, ""), 10);
const lowReading = Math.max(0, startOdometer - 500);
await page.getByLabel("End odometer (km)").fill(String(lowReading));
await page.getByLabel("Fuel level (%)").fill("55");
await page.getByRole("button", { name: "Register return" }).click();
await expect(page.getByRole("heading", { name: "Return registered" })).toBeVisible();
});
await test.step("5. verify quality issue and queued automation event", async () => {
await expect(page.getByText(/DQ-RET-|None created/)).toBeVisible();
await expect(page.getByText(/Queued \(/)).toBeVisible();
});
await test.step("6. resolve the duplicate customer scenario (S2)", async () => {
await page.goto("/data-quality/DQ-DEMO-DUPLICATE");
await expect(page.getByRole("heading", { name: "Compare and merge" })).toBeVisible();
await page.getByRole("button", { name: /Merge into CUS-0012/ }).click();
await page.getByRole("button", { name: "Yes, merge" }).click();
await expect(page.getByText("resolved", { exact: true })).toBeVisible();
});
await test.step("7. ask the damage question and inspect citations (S6)", async () => {
await page.goto("/knowledge");
await page
.getByPlaceholder(/What must I do when a vehicle returns with damage/)
.fill("What must I do when a vehicle returns with damage?");
await page.getByRole("button", { name: "Ask" }).click();
await expect(page.getByText("Grounded in cited procedures")).toBeVisible();
await expect(page.getByText("Damage handling procedure").first()).toBeVisible();
await expect(page.getByText("Vehicle return procedure").first()).toBeVisible();
});
await test.step("8. inspect audit entries", async () => {
await page.goto("/audit");
await page.getByLabel("Action").fill("return_registered");
await expect(page.locator(".data-table tbody tr").first()).toBeVisible();
await expect(page.getByText("return_registered").first()).toBeVisible();
});
await test.step("9. verify responsive navigation at mobile width", async () => {
await page.setViewportSize({ width: 360, height: 800 });
await page.goto("/dashboard");
await expect(page.getByText("Synthetic demo environment")).toBeVisible();
await expect(page.getByRole("link", { name: "Dashboard" })).toBeVisible();
const scrollWidth = await page.evaluate(() => document.documentElement.scrollWidth);
const clientWidth = await page.evaluate(() => document.documentElement.clientWidth);
expect(scrollWidth).toBeLessThanOrEqual(clientWidth + 1);
});
});
+64
View File
@@ -13,6 +13,7 @@
"react-router-dom": "7.18.2"
},
"devDependencies": {
"@playwright/test": "1.62.1",
"@types/react": "18.3.12",
"@types/react-dom": "18.3.1",
"@vitejs/plugin-react": "4.3.4",
@@ -763,6 +764,22 @@
"node": "^22.20 || ^24.12 || >=25"
}
},
"node_modules/@playwright/test": {
"version": "1.62.1",
"resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.62.1.tgz",
"integrity": "sha512-DTcUc8qii+cpHvtOwggMtBRMjKZHXYWdw8syRYu2vtzuq4Wxphqq4NfCs5Zt44L6mA8rfDfj+PHnxFc/FeK6mQ==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"playwright": "1.62.1"
},
"bin": {
"playwright": "cli.js"
},
"engines": {
"node": ">=20"
}
},
"node_modules/@rollup/rollup-android-arm-eabi": {
"version": "4.62.4",
"resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm-eabi/-/rollup-android-arm-eabi-4.62.4.tgz",
@@ -1543,6 +1560,53 @@
"dev": true,
"license": "ISC"
},
"node_modules/playwright": {
"version": "1.62.1",
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.62.1.tgz",
"integrity": "sha512-0M+L3LAD8/nm554LOla9Ayx0j0tmFZ0FBcoQ7F1VuVHpM/XpiC8RcDzBQB8W5+hA8L22THxELzeF+2WcUzvcLg==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"playwright-core": "1.62.1"
},
"bin": {
"playwright": "cli.js"
},
"engines": {
"node": ">=20"
},
"optionalDependencies": {
"fsevents": "2.3.2"
}
},
"node_modules/playwright-core": {
"version": "1.62.1",
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.62.1.tgz",
"integrity": "sha512-wPYSwEBJY9GHraISXqyqtx0na0LpO3XEX7jNDhntbex7tzUS7kLnZsOlFruFJB4Hi/rhDMjXGqHewDZ68nYZVw==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"playwright-core": "cli.js"
},
"engines": {
"node": ">=20"
}
},
"node_modules/playwright/node_modules/fsevents": {
"version": "2.3.2",
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
"integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
}
},
"node_modules/postcss": {
"version": "8.5.25",
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.25.tgz",
+3 -1
View File
@@ -7,7 +7,8 @@
"dev": "vite --host 0.0.0.0",
"build": "tsc -b && vite build",
"preview": "vite preview --host 0.0.0.0",
"lint": "tsc -b --noEmit"
"lint": "tsc -b --noEmit",
"test:e2e": "playwright test"
},
"dependencies": {
"react": "18.3.1",
@@ -15,6 +16,7 @@
"react-router-dom": "7.18.2"
},
"devDependencies": {
"@playwright/test": "1.62.1",
"@types/react": "18.3.12",
"@types/react-dom": "18.3.1",
"@vitejs/plugin-react": "4.3.4",
+23
View File
@@ -0,0 +1,23 @@
import { defineConfig, devices } from "@playwright/test";
export default defineConfig({
testDir: "./e2e",
// _capture-screenshots.spec.ts is evidence-generation tooling, not a regression test —
// it mutates demo data via a reset and is run explicitly, not as part of the suite.
testIgnore: "**/_*.spec.ts",
timeout: 30_000,
fullyParallel: false,
workers: 1,
reporter: [["list"], ["html", { open: "never", outputFolder: "playwright-report" }]],
use: {
baseURL: process.env.MOBILITYOPS_PUBLIC_URL ?? "http://localhost:1228",
trace: "retain-on-failure",
screenshot: "only-on-failure",
},
projects: [
{
name: "chromium",
use: { ...devices["Desktop Chrome"] },
},
],
});