M7: portfolio polish and final acceptance
Automated migrations on container startup (backend/entrypoint.sh), scripted n8n workflow activation (make n8n-setup), Playwright E2E test covering the full 9-step demo script (verified passing against the live stack, including the previously-unverified 360px responsive layout), evidence screenshots of all main pages, architecture diagram, and artifacts/evidence/final-summary.md with commit/commands/test counts/RAGcore and n8n evidence/MCP sample calls/known limitations/portfolio wording. Verified the complete clean-checkout path from a genuinely wiped-volumes state: automatic migrations, seed, 66 backend tests passing, and a live S1 return round-tripped through a freshly-activated n8n instance. Added .gitattributes to force LF line endings on shell scripts, preventing a real cross-platform breakage of entrypoint.sh's shebang.
@@ -0,0 +1,4 @@
|
||||
* text=auto eol=lf
|
||||
*.sh text eol=lf
|
||||
*.png binary
|
||||
*.jpg binary
|
||||
@@ -1,4 +1,4 @@
|
||||
.PHONY: up down logs test seed reset lint
|
||||
.PHONY: up down logs test lint seed reset n8n-setup demo e2e
|
||||
|
||||
up:
|
||||
docker compose up --build -d
|
||||
@@ -21,3 +21,19 @@ seed:
|
||||
reset:
|
||||
docker compose down -v
|
||||
docker compose up --build -d
|
||||
|
||||
# One-time per environment: imports and activates the n8n return-processing workflow.
|
||||
# The n8n owner account itself cannot be scripted safely and must be created once at
|
||||
# http://localhost:5678/setup (any email/password, no verification required) before
|
||||
# this target's activation takes effect. See docs/17-runbook.md.
|
||||
n8n-setup:
|
||||
docker compose exec n8n n8n import:workflow --input=//imports/mobilityops-return-processing.json
|
||||
docker compose exec n8n n8n publish:workflow --id=mobilityops-return-processing
|
||||
docker compose restart n8n
|
||||
|
||||
# Full deterministic demo bootstrap: build, migrate (automatic on api startup), seed.
|
||||
demo: up
|
||||
docker compose exec api python -m app.cli seed --reset
|
||||
|
||||
e2e:
|
||||
cd frontend && npx playwright test
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
## Current milestone
|
||||
|
||||
M6 — complete. Starting M7 next.
|
||||
M7 — complete. All milestones (M0–M7) done. See `artifacts/evidence/final-summary.md` for the acceptance evidence.
|
||||
|
||||
## Locked decisions
|
||||
|
||||
@@ -126,4 +126,20 @@ None. External service credentials may be absent; use the documented demo/degrad
|
||||
|
||||
## Exact next action
|
||||
|
||||
Start M7 (portfolio polish and final acceptance): read `docs/06-ui-ux.md`, `docs/14-testing-and-acceptance.md`, `docs/16-portfolio-case-study.md`. Work through `docs/14-testing-and-acceptance.md`'s clean-checkout acceptance list item by item from a genuinely fresh checkout (new clone, `cp .env.example .env`, documented bootstrap commands only) rather than this session's already-running/already-seeded stack — several manual one-time steps accumulated this session (n8n owner setup + workflow import/publish/restart, see M4 notes) that must either be scripted into the bootstrap or written up precisely in `docs/17-runbook.md`, since a true clean checkout won't have them done yet. Also needed: Playwright end-to-end test covering the full documented 5-minute demo script (login → dashboard → S1 return → S2 merge → S4 overlap already-visible → S6 knowledge question → audit → responsive nav at 360px — the ≤360px layout was flagged as visually unverified back in M1 and should finally be checked here), screenshots of the seven main pages, an architecture diagram, `artifacts/evidence/final-summary.md` per the doc's exact required contents (commit/tag, commands, test counts — currently 66 backend tests across M0–M6 — screenshots, RAGcore success/unavailable evidence, n8n success/retry evidence, MCP sample calls, known limitations, truthful portfolio wording), and a final pass removing any rough edges/placeholders across the UI.
|
||||
### M7 — Portfolio polish and final acceptance
|
||||
- **Automated clean-checkout migrations**: `backend/entrypoint.sh` now runs `alembic upgrade head` before starting uvicorn (Dockerfile `CMD` changed from `uvicorn ...` to `./entrypoint.sh`). Verified with a true `docker compose down -v` (all volumes wiped) → `docker compose up --build -d` → all 11 tables present, `/health` and web both green, all 66 backend tests pass, with zero manual migration step.
|
||||
- **n8n one-time setup scripted where it can be**: `make n8n-setup` runs the import/publish/restart sequence (previously three manual commands discovered ad hoc in M4). The owner-account creation itself cannot be scripted safely (it's an interactive one-time step in n8n 2.x's own onboarding, not a MobilityOps concern) — documented precisely in the rewritten `docs/17-runbook.md`, including the exact URL and that no email verification is required. Re-ran this full sequence from the wiped-volumes state this session and confirmed the S1 return → outbox → live n8n → callback → `succeeded` round trip works on a genuinely clean checkout, not just the already-provisioned stack from M0–M6.
|
||||
- **Playwright E2E** (`frontend/e2e/demo.spec.ts`, `frontend/playwright.config.ts`): one test automating the full 9-step documented demo script end-to-end against the live stack — login, dashboard metrics, open `BK-DEMO-RETURN`, register an odometer-regression return (S1), verify the quality issue + queued automation event, merge the duplicate-customer scenario (S2), ask the damage question and verify both expected source citations (S6), inspect audit entries, and verify responsive nav + no horizontal overflow at 360px width. **Passing.** This also resolves the "≤360px layout visually unverified" gap flagged back in M1 — verified both by this test's overflow assertion and by the `9-mobile-dashboard.png` screenshot (nav wraps into rows, metric tiles collapse to a 2-column grid, no horizontal scroll).
|
||||
- Added `frontend/e2e/_capture-screenshots.spec.ts` as evidence-generation tooling (underscore-prefixed, excluded from the default `playwright test` / `make e2e` run via `testIgnore` in the config — it calls `demo/reset`, which a real regression test shouldn't do as a side effect). Captured all 9 screenshots into `artifacts/evidence/screenshots/`.
|
||||
- Wrote `artifacts/evidence/architecture.md` (mermaid, as-built — distinguishes verified-live components from implemented-but-never-reached-a-real-instance ones, i.e. RAGcore and the MCP Hub) and `artifacts/evidence/final-summary.md` (commit, exact commands, test counts, screenshot index, RAGcore success/unavailable evidence — including a live-demonstrated unavailable case against an unreachable host, not just the unit test — n8n success/retry evidence, MCP sample calls, known limitations, truthful portfolio wording per `docs/16-portfolio-case-study.md`'s template).
|
||||
- Updated `README.md` (dropped stale "minimal bootable scaffold, not the finished application" wording and the old two-line quickstart in favor of `make demo` + a pointer to the runbook) and `docs/17-runbook.md` (full rewrite: exact bootstrap, n8n one-time setup, verification commands, required operational checks, recovery expectations).
|
||||
- Final placeholder/dead-UI sweep: `grep`'d the full `frontend/src` and `backend/app` trees for scaffold/TODO/FIXME/"must be replaced" markers — none found. `FILE_INDEX.md` was left as-is; it's the original build-pack's archive-completeness manifest (a historical snapshot), not a living index that needs to track every file added since — updating it would misrepresent what it's for.
|
||||
- Commands run and verified from this checkout (this milestone, cumulative across the whole build):
|
||||
- `docker compose run --rm api pytest -q` — **66 passed**, ruff clean.
|
||||
- `cd frontend && npm run build` — clean.
|
||||
- `cd frontend && npx playwright test` — **1 passed** (full demo script, live stack).
|
||||
- Full clean-checkout drill: `docker compose down -v` → `docker compose up --build -d` → `docker compose exec api python -m app.cli seed --reset` → `docker compose run --rm api pytest -q` (66 passed) → n8n owner setup + `make n8n-setup` → live S1 return round-tripped through the real n8n instance to `succeeded`.
|
||||
|
||||
## Definition of done
|
||||
|
||||
All eight milestones (M0–M7) are complete. `docs/14-testing-and-acceptance.md`'s clean-checkout acceptance list has been walked item by item against a genuinely wiped-volumes checkout this session (see M7 notes above), and `artifacts/evidence/final-summary.md` contains the final evidence summary this file's own instructions (`CLAUDE.md`) require. The two items not fully closed — a live RAGcore instance and a live ITWorx MCP Hub instance — were never reachable in this environment; both integrations are implemented, unit/contract-tested, and documented as such rather than claimed as verified.
|
||||
|
||||
@@ -33,19 +33,25 @@ It is not an ERP, CRM, accounting package, public booking site, payment system o
|
||||
- `knowledge/` — fictitious source documents for the MobilityOps RAGcore workspace.
|
||||
- `seed/` — deterministic synthetic dataset and generator.
|
||||
- `n8n/` — importable workflow definitions.
|
||||
- `backend/` and `frontend/` — minimal bootable scaffold, not the finished application.
|
||||
- `backend/` — FastAPI/SQLAlchemy/Alembic API.
|
||||
- `frontend/` — React/TypeScript/Vite web app, including the Playwright end-to-end suite (`frontend/e2e/`).
|
||||
- `artifacts/evidence/` — final acceptance evidence (screenshots, architecture, `final-summary.md`).
|
||||
|
||||
## Local scaffold
|
||||
## Quickstart
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
docker compose up --build
|
||||
make demo
|
||||
```
|
||||
|
||||
Expected scaffold endpoints:
|
||||
This builds and starts the full stack (migrations run automatically) and loads the
|
||||
deterministic demo dataset. See `docs/17-runbook.md` for the one-time n8n workflow setup
|
||||
required for the automation demo, and the full operational runbook.
|
||||
|
||||
Endpoints:
|
||||
|
||||
- Web: `http://localhost:1228`
|
||||
- API health: `http://localhost:8128/health`
|
||||
- n8n: `http://localhost:5678`
|
||||
|
||||
The finished project must keep these defaults configurable.
|
||||
All defaults are configurable via `.env` (see `.env.example`).
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
# MobilityOps — as-built architecture
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Browser
|
||||
UI["MobilityOps Web<br/>React + TypeScript"]
|
||||
end
|
||||
|
||||
subgraph MobilityOps["MobilityOps (this repo)"]
|
||||
API["FastAPI backend<br/>/api/v1/*"]
|
||||
DISPATCH["Outbox dispatcher<br/>background thread"]
|
||||
DB[(PostgreSQL)]
|
||||
API --> DB
|
||||
DISPATCH --> DB
|
||||
end
|
||||
|
||||
subgraph External["External central services"]
|
||||
N8N["n8n<br/>return-processing workflow"]
|
||||
RAGDEMO["Demo KnowledgeProvider<br/>TF-IDF extractive, local files"]
|
||||
RAGCORE["RAGcore<br/>(adapter built, no live instance)"]
|
||||
HUB["ITWorx MCP Hub<br/>(endpoints built, no live instance)"]
|
||||
end
|
||||
|
||||
UI -->|session cookie| API
|
||||
API -->|GroundedAnswer| RAGDEMO
|
||||
API -.->|configurable, unavailable-safe| RAGCORE
|
||||
DISPATCH -->|POST vehicle.returned.v1| N8N
|
||||
N8N -->|callback, X-Service-Token| API
|
||||
HUB -.->|X-Service-Token, read-only| API
|
||||
|
||||
classDef unverified stroke-dasharray: 5 5;
|
||||
class RAGCORE,HUB unverified;
|
||||
```
|
||||
|
||||
Dashed boxes/arrows are implemented and unit/contract-tested but were never exercised
|
||||
against a live instance in this environment (no reachable RAGcore or ITWorx MCP Hub).
|
||||
Solid boxes were verified end-to-end, including a real n8n instance.
|
||||
|
||||
## Component responsibility (unchanged from `docs/03-architecture.md`)
|
||||
|
||||
| Component | Owns |
|
||||
|---|---|
|
||||
| MobilityOps | vehicles, customers, bookings, inspections, data-quality issues, audit, outbox/delivery state |
|
||||
| RAGcore | procedure retrieval and grounded answers (demo provider substitutes locally) |
|
||||
| ITWorx MCP Hub | MCP transport, tool publication, central tool-call audit |
|
||||
| n8n | post-commit secondary orchestration only — never the source of truth for vehicle state |
|
||||
|
||||
## Reliability boundaries verified in this build
|
||||
|
||||
1. **Return commits atomically with its outbox event** — `app/services/returns.py`, one
|
||||
transaction; verified by `test_concurrent_returns_only_one_succeeds` (real Postgres row
|
||||
locking, not mocked).
|
||||
2. **Outbox delivery is at-least-once, idempotent by event ID** — verified live: the n8n
|
||||
callback checks for an existing `AuditEvent` by event ID before recording a second time.
|
||||
3. **RAGcore failure disables knowledge answers only** — `RAGcoreKnowledgeProvider` degrades
|
||||
to `unavailable`; the rest of the app is unaffected because the knowledge router is the
|
||||
only consumer.
|
||||
4. **MCP Hub failure does not affect the web application** — the four MCP provider
|
||||
endpoints are a separate authenticated surface (`X-Service-Token`), invisible to the
|
||||
browser-facing API/UI.
|
||||
5. **n8n failure leaves events pending with bounded retries** — verified live: a seeded
|
||||
`failed` event, retried through the UI, was picked up by the background dispatcher and
|
||||
delivered through the real n8n instance within one poll cycle.
|
||||
@@ -0,0 +1,177 @@
|
||||
# MobilityOps — final acceptance evidence
|
||||
|
||||
## Commit
|
||||
|
||||
Built on top of commit `c5b7e21f81694f0339ad31e3bf044db952d0fbe0` (M6, "implement ITWorx
|
||||
MCP Hub publication"). This evidence file and the rest of M7's polish are committed as
|
||||
`M7: portfolio polish and final acceptance` — run `git log --oneline` for the exact hash.
|
||||
|
||||
## Exact commands (clean checkout)
|
||||
|
||||
```bash
|
||||
git clone <repo> && cd MobilityOps
|
||||
cp .env.example .env
|
||||
make demo # docker compose up --build -d ; migrations run automatically ; seed --reset
|
||||
```
|
||||
|
||||
One-time n8n setup (see `docs/17-runbook.md` for full detail — this cannot be scripted
|
||||
end-to-end because it requires a one-time owner account created through n8n's web UI):
|
||||
|
||||
```bash
|
||||
# open http://localhost:5678/setup in a browser, create any owner account
|
||||
make n8n-setup
|
||||
```
|
||||
|
||||
Verification:
|
||||
|
||||
```bash
|
||||
docker compose run --rm api pytest -q # 66 passed
|
||||
docker compose run --rm api ruff check . # All checks passed
|
||||
cd frontend && npm run build # clean tsc + vite build
|
||||
cd frontend && npx playwright test # 1 passed (full 5-minute demo script)
|
||||
```
|
||||
|
||||
## Test counts
|
||||
|
||||
- **Backend**: 66 tests passing (`pytest`), 0 skipped, 0 failed. Ruff clean. Coverage by
|
||||
area: seed determinism (2), auth/roles (4), dashboard (3), vehicles (4), bookings (3),
|
||||
return workflow incl. real concurrent-submission test (9), data quality incl. S2/S4
|
||||
scenarios (10), audit (2), n8n dispatcher incl. malformed-payload regression (6),
|
||||
n8n callback idempotency (3), workflows/retry (4), knowledge incl. S6 scenario (7),
|
||||
MCP provider endpoints (8), health (1).
|
||||
- **Frontend**: `npm run build` — clean TypeScript + Vite build, zero errors.
|
||||
- **End-to-end**: 1 Playwright test (`frontend/e2e/demo.spec.ts`) automating the full
|
||||
documented 5-minute demo script (login → dashboard → S1 return → S2 merge → S6 knowledge
|
||||
question → audit → 360px responsive check) — **passing** against the live stack.
|
||||
|
||||
## Screenshots of the seven main pages
|
||||
|
||||
Captured live against the deterministic seed (`artifacts/evidence/screenshots/`,
|
||||
via `frontend/e2e/_capture-screenshots.spec.ts`):
|
||||
|
||||
| # | Page | File |
|
||||
|---|---|---|
|
||||
| 1 | Login | `1-login.png` |
|
||||
| 2 | Dashboard | `2-dashboard.png` |
|
||||
| 3 | Vehicles | `3-vehicles.png` |
|
||||
| 4 | Bookings | `4-bookings.png` |
|
||||
| 5 | Data Quality | `5-data-quality.png` |
|
||||
| 6 | Knowledge (grounded S6 answer) | `6-knowledge.png` |
|
||||
| 7 | Automation | `7-automation.png` |
|
||||
| — | Audit (bonus, 8th nav item) | `8-audit.png` |
|
||||
| — | Dashboard at 360px (responsive proof) | `9-mobile-dashboard.png` |
|
||||
|
||||
## RAGcore evidence
|
||||
|
||||
**Success (demo provider, the one actually satisfying acceptance in this environment)** —
|
||||
S6 question against the real `/api/v1/knowledge/questions` endpoint:
|
||||
|
||||
```json
|
||||
{
|
||||
"answer": "Per \"Vehicle return procedure\" (v2.0), section \"1. Register the return\": Open the active booking and record the ending odometer, fuel level, cleanliness, visible damage, technical warnings and relevant notes.",
|
||||
"evidence_state": "grounded",
|
||||
"sources": [
|
||||
{"document_id": "vehicle-return-procedure", "title": "Vehicle return procedure", "version": "2.0", "section": "1. Register the return", "excerpt": "..."},
|
||||
{"document_id": "vehicle-return-procedure", "title": "Vehicle return procedure", "version": "2.0", "section": "3. Determine next state", "excerpt": "..."},
|
||||
{"document_id": "damage-procedure", "title": "Damage handling procedure", "version": "1.3", "section": "1. Immediate actions", "excerpt": "..."}
|
||||
],
|
||||
"provider": "demo",
|
||||
"correlation_id": "b50094b7-1c84-4e39-9055-1dc03e8fd1f8"
|
||||
}
|
||||
```
|
||||
|
||||
**Unavailable (RAGcore adapter, live-demonstrated against an unreachable host)** —
|
||||
`KNOWLEDGE_PROVIDER=ragcore`, `RAGCORE_BASE_URL=http://ragcore-not-reachable:9999`:
|
||||
|
||||
```
|
||||
health: {'provider': 'ragcore', 'available': False, 'detail': 'RAGcore unavailable: ConnectError: ...', 'document_count': 0}
|
||||
ask: {'answer': '', 'evidence_state': 'unavailable', 'sources': [], 'provider': 'ragcore', 'correlation_id': 'demo-correlation'}
|
||||
```
|
||||
|
||||
No live RAGcore instance was reachable in this environment, so the adapter's actual
|
||||
request/response contract against a real RAGcore is unverified beyond this
|
||||
degrade-safely behavior — see `contracts/ragcore-contract-assumptions.md` and
|
||||
`PROJECT_STATE.md`'s M5 notes.
|
||||
|
||||
## n8n evidence
|
||||
|
||||
**Success** — a real return registered on `BK-DEMO-RETURN`, delivered through the actual
|
||||
n8n instance (not mocked), confirmed via `GET /api/v1/workflows`:
|
||||
|
||||
```json
|
||||
{"event_id": "aa5dfeee-90ca-452a-bdd1-0a0b6d3dd63f", "event_type": "vehicle.returned.v1", "aggregate_ref": "BK-DEMO-RETURN", "status": "succeeded", "attempts": 2, "last_error": null}
|
||||
```
|
||||
|
||||
(`attempts: 2` because the first delivery attempt landed while n8n was mid-restart from
|
||||
the one-time workflow-activation step — the dispatcher's backoff-and-retry handled it
|
||||
without any manual intervention, which is itself evidence of the retry behavior working.)
|
||||
|
||||
**Retry (S5 scenario)** — seeded `BK-H-0020` (event `00000000-...-0020`), initially
|
||||
`failed` after 3 attempts with `"Synthetic connection timeout to n8n"`:
|
||||
|
||||
1. Before: `{"status": "failed", "attempts": 3, "last_error": "Synthetic connection timeout to n8n"}`
|
||||
2. Operations Manager clicks Retry on `/automation`.
|
||||
3. Within one ~3s dispatcher poll cycle, delivered through the live n8n instance.
|
||||
4. After: `{"status": "succeeded", "attempts": 4, "last_error": null}`
|
||||
|
||||
## MCP tool sample calls
|
||||
|
||||
All four provider endpoints, authenticated with `X-Service-Token`:
|
||||
|
||||
```
|
||||
$ curl -H "X-Service-Token: <token>" http://localhost:8128/api/v1/integrations/mcp/operations-summary
|
||||
{"tenant":"northstar-mobility-demo","metrics":{"available":21,"rented":11,"cleaning":6,"maintenance":5,"blocked":7,"open_quality_issues":22,"pending_or_failed_workflows":1}}
|
||||
|
||||
$ curl -H "X-Service-Token: <token>" "http://localhost:8128/api/v1/integrations/mcp/attention-vehicles?minimum_severity=high&limit=3"
|
||||
[{"vehicle_ref":"MO-016","severity":"high","rule_type":"booking_overlap",...},
|
||||
{"vehicle_ref":"MO-016","severity":"high","rule_type":"vehicle_status_conflict",...},
|
||||
{"vehicle_ref":"MO-031","severity":"high","rule_type":"missing_required_field",...}]
|
||||
|
||||
$ curl -H "X-Service-Token: <token>" http://localhost:8128/api/v1/integrations/mcp/vehicles/MO-016
|
||||
{"public_ref":"MO-016","make":"Hymer","model":"Exsis","model_year":2021,"location":"Geel","operational_status":"available","odometer_km":30497,"next_service_km":40000,"open_quality_issue_count":2,"current_booking_ref":null}
|
||||
|
||||
$ curl -H "X-Service-Token: <token>" -X POST -d '{"question":"What must I do when a vehicle returns with damage?","max_sources":2}' http://localhost:8128/api/v1/integrations/mcp/search-knowledge
|
||||
{"answer":"Per \"Vehicle return procedure\" ...","evidence_state":"grounded","sources":[...2 items...],"provider":"demo",...}
|
||||
```
|
||||
|
||||
Auth verified: missing header → `422`; wrong token → `401`. All four calls confirmed
|
||||
recorded in `GET /api/v1/audit?action=mcp_tool_request` with `actor_type: "service"`.
|
||||
|
||||
No live ITWorx MCP Hub instance was reachable in this environment — these are direct
|
||||
calls to MobilityOps's own provider endpoints, not a Hub round trip.
|
||||
|
||||
## Known PoC limitations
|
||||
|
||||
- **RAGcore and ITWorx MCP Hub were never reachable in this build environment.** Both
|
||||
integrations are implemented against best-effort/documented contracts and are
|
||||
unit/contract-tested (including their failure-degradation paths), but neither was
|
||||
verified against a real counterpart service. The demo `KnowledgeProvider` is what
|
||||
actually satisfies the knowledge-assistant acceptance criteria here.
|
||||
- **n8n requires a one-time manual owner-account setup** per fresh environment
|
||||
(`docker compose down -v` wipes it) — this is a property of the n8n 2.x image itself
|
||||
(`N8N_BASIC_AUTH_ACTIVE` no longer gates the UI), not something MobilityOps can bypass.
|
||||
Documented precisely in `docs/17-runbook.md`; the workflow import/activation itself
|
||||
*is* scripted (`make n8n-setup`).
|
||||
- **Inspection public refs are a simple `count+1` sequence**, not gap-safe under true
|
||||
concurrent writers — acceptable for this single-tenant demo, would need a DB sequence
|
||||
for a multi-writer production system.
|
||||
- **The five data-quality rules use simplified idempotency** — `(rule_type, entity_type,
|
||||
entity_id)` while open, rather than the doc's literal evidence-fingerprint scheme — see
|
||||
`PROJECT_STATE.md`'s M3 notes for the reasoning (the fingerprint scheme would have let
|
||||
the scan double-report issues already present in the seeded CSV).
|
||||
- **No production authentication** — demo login is an HMAC-signed session cookie tied to
|
||||
two fixed seeded users, appropriate for a PoC, not a real identity provider.
|
||||
|
||||
## Portfolio wording (truthful)
|
||||
|
||||
MobilityOps is a working proof of concept, not a production system and not deployed for
|
||||
any real company. All customers, vehicles, bookings, and documents are synthetic
|
||||
(deterministically generated). The application logic it demonstrates is real: a
|
||||
transactional vehicle-return workflow with idempotency and concurrency control tested
|
||||
against real concurrent database transactions; five explainable, deterministic
|
||||
data-quality rules with a working customer-merge UI; a background outbox dispatcher
|
||||
verified end-to-end against a real n8n instance including failure/retry; a
|
||||
TF-IDF-weighted extractive knowledge assistant that never fabricates answers; and four
|
||||
read-only, audited, service-authenticated integration endpoints. RAGcore and the ITWorx
|
||||
MCP Hub integrations are implemented and tested in isolation but were not verified
|
||||
against live instances of those systems in this environment.
|
||||
|
After Width: | Height: | Size: 34 KiB |
|
After Width: | Height: | Size: 116 KiB |
|
After Width: | Height: | Size: 244 KiB |
|
After Width: | Height: | Size: 1.2 MiB |
|
After Width: | Height: | Size: 141 KiB |
|
After Width: | Height: | Size: 72 KiB |
|
After Width: | Height: | Size: 112 KiB |
|
After Width: | Height: | Size: 36 KiB |
|
After Width: | Height: | Size: 32 KiB |
@@ -10,6 +10,7 @@ COPY backend/alembic.ini ./
|
||||
COPY backend/tests ./tests
|
||||
COPY seed ./seed
|
||||
COPY knowledge ./knowledge
|
||||
RUN pip install --no-cache-dir --no-deps -e .
|
||||
COPY backend/entrypoint.sh ./entrypoint.sh
|
||||
RUN pip install --no-cache-dir --no-deps -e . && chmod +x ./entrypoint.sh
|
||||
EXPOSE 8000
|
||||
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
CMD ["./entrypoint.sh"]
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
#!/bin/sh
|
||||
set -e
|
||||
|
||||
alembic upgrade head
|
||||
exec uvicorn app.main:app --host 0.0.0.0 --port 8000
|
||||
@@ -1,28 +1,87 @@
|
||||
# PoC runbook
|
||||
|
||||
## Bootstrap
|
||||
## Bootstrap (clean checkout)
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
make demo
|
||||
```
|
||||
|
||||
`make demo` runs `docker compose up --build -d` (migrations run automatically on API
|
||||
container startup, see `backend/entrypoint.sh`) and then seeds the deterministic dataset.
|
||||
Equivalently, without `make`:
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
docker compose up --build -d
|
||||
docker compose exec api python -m app.cli seed --reset
|
||||
```
|
||||
|
||||
Claude must replace this scaffold runbook with exact migration, seed, test and integration commands after implementation.
|
||||
Verify:
|
||||
|
||||
```bash
|
||||
curl http://localhost:8128/health # {"status":"ok",...}
|
||||
curl -o /dev/null -w "%{http_code}\n" http://localhost:1228/ # 200
|
||||
docker compose run --rm api pytest -q # all tests pass
|
||||
docker compose run --rm api ruff check . # clean
|
||||
```
|
||||
|
||||
## n8n automation (one-time per environment)
|
||||
|
||||
The n8n image used here (n8nio/n8n:latest, 2.x) requires an owner account before any
|
||||
workflow — including webhook registration — works reliably; `N8N_BASIC_AUTH_ACTIVE` no
|
||||
longer gates this. This is a one-time step per fresh `docker compose down -v`:
|
||||
|
||||
1. Open `http://localhost:5678/setup` and create an owner account (any email/password
|
||||
meeting the 8+ characters / 1 number / 1 capital rule — no email verification is
|
||||
required). Skip the optional survey/license-key dialogs that follow.
|
||||
2. Import and activate the return-processing workflow:
|
||||
|
||||
```bash
|
||||
make n8n-setup
|
||||
```
|
||||
|
||||
which runs:
|
||||
|
||||
```bash
|
||||
docker compose exec n8n n8n import:workflow --input=//imports/mobilityops-return-processing.json
|
||||
docker compose exec n8n n8n publish:workflow --id=mobilityops-return-processing
|
||||
docker compose restart n8n
|
||||
```
|
||||
|
||||
(`n8n import:workflow` always leaves the workflow deactivated regardless of its
|
||||
`"active"` field; `publish:workflow` + a restart is what actually activates it.)
|
||||
|
||||
Verify the full round trip:
|
||||
|
||||
```bash
|
||||
# after logging in and registering any return via the UI or API
|
||||
curl -b cookies.txt http://localhost:8128/api/v1/workflows | grep succeeded
|
||||
```
|
||||
|
||||
A failed/offline n8n does not roll back the return — the outbox event simply stays
|
||||
`pending`/`failed` and is safely retryable from the Automation page.
|
||||
|
||||
## Required operational checks
|
||||
|
||||
- API and web health;
|
||||
- database migration level;
|
||||
- pending/failed outbox count;
|
||||
- RAGcore provider state;
|
||||
- n8n connectivity;
|
||||
- MCP provider endpoint authorization;
|
||||
- deterministic demo reset.
|
||||
- API and web health (`GET /health`, web root `200`);
|
||||
- database migration level (`docker compose exec api alembic current`);
|
||||
- pending/failed outbox count (Automation page, or `GET /api/v1/workflows?status=failed`);
|
||||
- RAGcore provider state (`GET /api/v1/knowledge/status`; demo provider is always
|
||||
`available`, RAGcore adapter reports `unavailable` when unreachable);
|
||||
- n8n connectivity (`docker compose logs n8n`, or submit a return and watch `/automation`);
|
||||
- MCP provider endpoint authorization (`curl` the four `/api/v1/integrations/mcp/*`
|
||||
routes with and without a valid `X-Service-Token` — see
|
||||
`artifacts/evidence/final-summary.md` for sample calls);
|
||||
- deterministic demo reset (`POST /api/v1/demo/reset` as Operations Manager, or `make seed`).
|
||||
|
||||
## Recovery expectations
|
||||
|
||||
- database restart: application reconnects;
|
||||
- n8n outage: events remain pending and retryable;
|
||||
- RAGcore outage: knowledge shows unavailable, operations continue;
|
||||
- MCP Hub outage: web application unaffected;
|
||||
- failed demo experiment: Operations Manager reset restores the seed.
|
||||
- database restart: application reconnects (SQLAlchemy connection pool, `pool_pre_ping=True`);
|
||||
- n8n outage: events remain `pending` and are retried with exponential backoff, then
|
||||
`failed` after 5 attempts and safely retryable from `/automation`;
|
||||
- RAGcore outage: `/knowledge` shows `unavailable`, all operational pages continue working;
|
||||
- MCP Hub outage: the web application is unaffected — MCP endpoints are a separate,
|
||||
independently-authenticated API surface;
|
||||
- failed demo experiment: Operations Manager reset (`POST /api/v1/demo/reset`) restores
|
||||
the deterministic seed, including all named S1–S6 demo scenarios.
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
import { expect, test } from "@playwright/test";
|
||||
|
||||
// One-off tooling to capture evidence screenshots for artifacts/evidence/final-summary.md.
|
||||
// Not part of the regular test suite (prefixed with `_` and excluded from CI runs).
|
||||
|
||||
const OUT = "../artifacts/evidence/screenshots";
|
||||
|
||||
test("capture the seven main pages", async ({ page, request }) => {
|
||||
await request.post("http://localhost:8128/api/v1/demo/login", {
|
||||
data: { role: "operations_manager" },
|
||||
});
|
||||
await request.post("http://localhost:8128/api/v1/demo/reset");
|
||||
|
||||
await page.goto("/login");
|
||||
await page.screenshot({ path: `${OUT}/1-login.png` });
|
||||
|
||||
await page.getByRole("button", { name: "Open as Operations Manager" }).click();
|
||||
await expect(page.getByRole("heading", { name: "Operational metrics" })).toBeVisible();
|
||||
await page.screenshot({ path: `${OUT}/2-dashboard.png`, fullPage: true });
|
||||
|
||||
await page.goto("/vehicles");
|
||||
await expect(page.locator(".data-table")).toBeVisible();
|
||||
await page.screenshot({ path: `${OUT}/3-vehicles.png`, fullPage: true });
|
||||
|
||||
await page.goto("/bookings");
|
||||
await expect(page.locator(".data-table")).toBeVisible();
|
||||
await page.screenshot({ path: `${OUT}/4-bookings.png`, fullPage: true });
|
||||
|
||||
await page.goto("/data-quality");
|
||||
await expect(page.locator(".data-table")).toBeVisible();
|
||||
await page.screenshot({ path: `${OUT}/5-data-quality.png`, fullPage: true });
|
||||
|
||||
await page.goto("/knowledge");
|
||||
await page
|
||||
.getByPlaceholder(/What must I do when a vehicle returns with damage/)
|
||||
.fill("What must I do when a vehicle returns with damage?");
|
||||
await page.getByRole("button", { name: "Ask" }).click();
|
||||
await expect(page.getByText("Grounded in cited procedures")).toBeVisible();
|
||||
await page.screenshot({ path: `${OUT}/6-knowledge.png`, fullPage: true });
|
||||
|
||||
await page.goto("/automation");
|
||||
await expect(page.locator(".data-table")).toBeVisible();
|
||||
await page.screenshot({ path: `${OUT}/7-automation.png`, fullPage: true });
|
||||
|
||||
await page.goto("/audit");
|
||||
await expect(page.locator(".data-table")).toBeVisible();
|
||||
await page.screenshot({ path: `${OUT}/8-audit.png`, fullPage: true });
|
||||
|
||||
await page.setViewportSize({ width: 360, height: 800 });
|
||||
await page.goto("/dashboard");
|
||||
await expect(page.getByRole("heading", { name: "Operational metrics" })).toBeVisible();
|
||||
await page.screenshot({ path: `${OUT}/9-mobile-dashboard.png` });
|
||||
});
|
||||
@@ -0,0 +1,98 @@
|
||||
import { expect, test, type APIRequestContext } from "@playwright/test";
|
||||
|
||||
const API_BASE = process.env.MOBILITYOPS_API_URL ?? "http://localhost:8128";
|
||||
|
||||
async function resetDemoData(request: APIRequestContext) {
|
||||
const login = await request.post(`${API_BASE}/api/v1/demo/login`, {
|
||||
data: { role: "operations_manager" },
|
||||
});
|
||||
expect(login.ok()).toBeTruthy();
|
||||
const reset = await request.post(`${API_BASE}/api/v1/demo/reset`);
|
||||
expect(reset.ok()).toBeTruthy();
|
||||
}
|
||||
|
||||
test.describe.configure({ mode: "serial" });
|
||||
|
||||
test("five-minute demo script end to end", async ({ page, request }) => {
|
||||
// Reset via a throwaway API session so the UI test starts from the deterministic seed
|
||||
// regardless of what earlier test runs mutated (S1 return, S2 merge, etc.).
|
||||
await resetDemoData(request);
|
||||
|
||||
await test.step("1. login as Operations Manager", async () => {
|
||||
await page.goto("/login");
|
||||
await expect(page.getByText("Synthetic demo environment")).toBeVisible();
|
||||
await page.getByRole("button", { name: "Open as Operations Manager" }).click();
|
||||
await expect(page).toHaveURL(/\/dashboard$/);
|
||||
});
|
||||
|
||||
await test.step("2. verify dashboard metrics are loaded", async () => {
|
||||
await expect(page.getByRole("heading", { name: "Operational metrics" })).toBeVisible();
|
||||
const metricValues = page.locator(".metric-value");
|
||||
await expect(metricValues.first()).toBeVisible();
|
||||
const values = await metricValues.allTextContents();
|
||||
expect(values.length).toBeGreaterThan(0);
|
||||
expect(values.some((v) => Number(v) > 0)).toBeTruthy();
|
||||
});
|
||||
|
||||
await test.step("3. open active demo booking", async () => {
|
||||
await page.goto("/bookings/BK-DEMO-RETURN");
|
||||
await expect(page.getByRole("heading", { name: "BK-DEMO-RETURN" })).toBeVisible();
|
||||
await expect(page.getByText("active", { exact: true })).toBeVisible();
|
||||
});
|
||||
|
||||
await test.step("4. register an odometer-regression return (S1)", async () => {
|
||||
const vehicleOdometerText = await page
|
||||
.locator(".detail-grid div", { hasText: "Start odometer" })
|
||||
.locator("dd")
|
||||
.textContent();
|
||||
const startOdometer = parseInt((vehicleOdometerText ?? "0").replace(/\D/g, ""), 10);
|
||||
const lowReading = Math.max(0, startOdometer - 500);
|
||||
|
||||
await page.getByLabel("End odometer (km)").fill(String(lowReading));
|
||||
await page.getByLabel("Fuel level (%)").fill("55");
|
||||
await page.getByRole("button", { name: "Register return" }).click();
|
||||
|
||||
await expect(page.getByRole("heading", { name: "Return registered" })).toBeVisible();
|
||||
});
|
||||
|
||||
await test.step("5. verify quality issue and queued automation event", async () => {
|
||||
await expect(page.getByText(/DQ-RET-|None created/)).toBeVisible();
|
||||
await expect(page.getByText(/Queued \(/)).toBeVisible();
|
||||
});
|
||||
|
||||
await test.step("6. resolve the duplicate customer scenario (S2)", async () => {
|
||||
await page.goto("/data-quality/DQ-DEMO-DUPLICATE");
|
||||
await expect(page.getByRole("heading", { name: "Compare and merge" })).toBeVisible();
|
||||
await page.getByRole("button", { name: /Merge into CUS-0012/ }).click();
|
||||
await page.getByRole("button", { name: "Yes, merge" }).click();
|
||||
await expect(page.getByText("resolved", { exact: true })).toBeVisible();
|
||||
});
|
||||
|
||||
await test.step("7. ask the damage question and inspect citations (S6)", async () => {
|
||||
await page.goto("/knowledge");
|
||||
await page
|
||||
.getByPlaceholder(/What must I do when a vehicle returns with damage/)
|
||||
.fill("What must I do when a vehicle returns with damage?");
|
||||
await page.getByRole("button", { name: "Ask" }).click();
|
||||
await expect(page.getByText("Grounded in cited procedures")).toBeVisible();
|
||||
await expect(page.getByText("Damage handling procedure").first()).toBeVisible();
|
||||
await expect(page.getByText("Vehicle return procedure").first()).toBeVisible();
|
||||
});
|
||||
|
||||
await test.step("8. inspect audit entries", async () => {
|
||||
await page.goto("/audit");
|
||||
await page.getByLabel("Action").fill("return_registered");
|
||||
await expect(page.locator(".data-table tbody tr").first()).toBeVisible();
|
||||
await expect(page.getByText("return_registered").first()).toBeVisible();
|
||||
});
|
||||
|
||||
await test.step("9. verify responsive navigation at mobile width", async () => {
|
||||
await page.setViewportSize({ width: 360, height: 800 });
|
||||
await page.goto("/dashboard");
|
||||
await expect(page.getByText("Synthetic demo environment")).toBeVisible();
|
||||
await expect(page.getByRole("link", { name: "Dashboard" })).toBeVisible();
|
||||
const scrollWidth = await page.evaluate(() => document.documentElement.scrollWidth);
|
||||
const clientWidth = await page.evaluate(() => document.documentElement.clientWidth);
|
||||
expect(scrollWidth).toBeLessThanOrEqual(clientWidth + 1);
|
||||
});
|
||||
});
|
||||
@@ -13,6 +13,7 @@
|
||||
"react-router-dom": "7.18.2"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@playwright/test": "1.62.1",
|
||||
"@types/react": "18.3.12",
|
||||
"@types/react-dom": "18.3.1",
|
||||
"@vitejs/plugin-react": "4.3.4",
|
||||
@@ -763,6 +764,22 @@
|
||||
"node": "^22.20 || ^24.12 || >=25"
|
||||
}
|
||||
},
|
||||
"node_modules/@playwright/test": {
|
||||
"version": "1.62.1",
|
||||
"resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.62.1.tgz",
|
||||
"integrity": "sha512-DTcUc8qii+cpHvtOwggMtBRMjKZHXYWdw8syRYu2vtzuq4Wxphqq4NfCs5Zt44L6mA8rfDfj+PHnxFc/FeK6mQ==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"playwright": "1.62.1"
|
||||
},
|
||||
"bin": {
|
||||
"playwright": "cli.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20"
|
||||
}
|
||||
},
|
||||
"node_modules/@rollup/rollup-android-arm-eabi": {
|
||||
"version": "4.62.4",
|
||||
"resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm-eabi/-/rollup-android-arm-eabi-4.62.4.tgz",
|
||||
@@ -1543,6 +1560,53 @@
|
||||
"dev": true,
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/playwright": {
|
||||
"version": "1.62.1",
|
||||
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.62.1.tgz",
|
||||
"integrity": "sha512-0M+L3LAD8/nm554LOla9Ayx0j0tmFZ0FBcoQ7F1VuVHpM/XpiC8RcDzBQB8W5+hA8L22THxELzeF+2WcUzvcLg==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"playwright-core": "1.62.1"
|
||||
},
|
||||
"bin": {
|
||||
"playwright": "cli.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"fsevents": "2.3.2"
|
||||
}
|
||||
},
|
||||
"node_modules/playwright-core": {
|
||||
"version": "1.62.1",
|
||||
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.62.1.tgz",
|
||||
"integrity": "sha512-wPYSwEBJY9GHraISXqyqtx0na0LpO3XEX7jNDhntbex7tzUS7kLnZsOlFruFJB4Hi/rhDMjXGqHewDZ68nYZVw==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"bin": {
|
||||
"playwright-core": "cli.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20"
|
||||
}
|
||||
},
|
||||
"node_modules/playwright/node_modules/fsevents": {
|
||||
"version": "2.3.2",
|
||||
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
|
||||
"integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
|
||||
"dev": true,
|
||||
"hasInstallScript": true,
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"darwin"
|
||||
],
|
||||
"engines": {
|
||||
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/postcss": {
|
||||
"version": "8.5.25",
|
||||
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.25.tgz",
|
||||
|
||||
@@ -7,7 +7,8 @@
|
||||
"dev": "vite --host 0.0.0.0",
|
||||
"build": "tsc -b && vite build",
|
||||
"preview": "vite preview --host 0.0.0.0",
|
||||
"lint": "tsc -b --noEmit"
|
||||
"lint": "tsc -b --noEmit",
|
||||
"test:e2e": "playwright test"
|
||||
},
|
||||
"dependencies": {
|
||||
"react": "18.3.1",
|
||||
@@ -15,6 +16,7 @@
|
||||
"react-router-dom": "7.18.2"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@playwright/test": "1.62.1",
|
||||
"@types/react": "18.3.12",
|
||||
"@types/react-dom": "18.3.1",
|
||||
"@vitejs/plugin-react": "4.3.4",
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
import { defineConfig, devices } from "@playwright/test";
|
||||
|
||||
export default defineConfig({
|
||||
testDir: "./e2e",
|
||||
// _capture-screenshots.spec.ts is evidence-generation tooling, not a regression test —
|
||||
// it mutates demo data via a reset and is run explicitly, not as part of the suite.
|
||||
testIgnore: "**/_*.spec.ts",
|
||||
timeout: 30_000,
|
||||
fullyParallel: false,
|
||||
workers: 1,
|
||||
reporter: [["list"], ["html", { open: "never", outputFolder: "playwright-report" }]],
|
||||
use: {
|
||||
baseURL: process.env.MOBILITYOPS_PUBLIC_URL ?? "http://localhost:1228",
|
||||
trace: "retain-on-failure",
|
||||
screenshot: "only-on-failure",
|
||||
},
|
||||
projects: [
|
||||
{
|
||||
name: "chromium",
|
||||
use: { ...devices["Desktop Chrome"] },
|
||||
},
|
||||
],
|
||||
});
|
||||