NuklearRabbit
108b5d04fc
Automated migrations on container startup (backend/entrypoint.sh), scripted n8n workflow activation (make n8n-setup), Playwright E2E test covering the full 9-step demo script (verified passing against the live stack, including the previously-unverified 360px responsive layout), evidence screenshots of all main pages, architecture diagram, and artifacts/evidence/final-summary.md with commit/commands/test counts/RAGcore and n8n evidence/MCP sample calls/known limitations/portfolio wording. Verified the complete clean-checkout path from a genuinely wiped-volumes state: automatic migrations, seed, 66 backend tests passing, and a live S1 return round-tripped through a freshly-activated n8n instance. Added .gitattributes to force LF line endings on shell scripts, preventing a real cross-platform breakage of entrypoint.sh's shebang.
178 lines
9.4 KiB
Markdown
178 lines
9.4 KiB
Markdown
# MobilityOps — final acceptance evidence
|
|
|
|
## Commit
|
|
|
|
Built on top of commit `c5b7e21f81694f0339ad31e3bf044db952d0fbe0` (M6, "implement ITWorx
|
|
MCP Hub publication"). This evidence file and the rest of M7's polish are committed as
|
|
`M7: portfolio polish and final acceptance` — run `git log --oneline` for the exact hash.
|
|
|
|
## Exact commands (clean checkout)
|
|
|
|
```bash
|
|
git clone <repo> && cd MobilityOps
|
|
cp .env.example .env
|
|
make demo # docker compose up --build -d ; migrations run automatically ; seed --reset
|
|
```
|
|
|
|
One-time n8n setup (see `docs/17-runbook.md` for full detail — this cannot be scripted
|
|
end-to-end because it requires a one-time owner account created through n8n's web UI):
|
|
|
|
```bash
|
|
# open http://localhost:5678/setup in a browser, create any owner account
|
|
make n8n-setup
|
|
```
|
|
|
|
Verification:
|
|
|
|
```bash
|
|
docker compose run --rm api pytest -q # 66 passed
|
|
docker compose run --rm api ruff check . # All checks passed
|
|
cd frontend && npm run build # clean tsc + vite build
|
|
cd frontend && npx playwright test # 1 passed (full 5-minute demo script)
|
|
```
|
|
|
|
## Test counts
|
|
|
|
- **Backend**: 66 tests passing (`pytest`), 0 skipped, 0 failed. Ruff clean. Coverage by
|
|
area: seed determinism (2), auth/roles (4), dashboard (3), vehicles (4), bookings (3),
|
|
return workflow incl. real concurrent-submission test (9), data quality incl. S2/S4
|
|
scenarios (10), audit (2), n8n dispatcher incl. malformed-payload regression (6),
|
|
n8n callback idempotency (3), workflows/retry (4), knowledge incl. S6 scenario (7),
|
|
MCP provider endpoints (8), health (1).
|
|
- **Frontend**: `npm run build` — clean TypeScript + Vite build, zero errors.
|
|
- **End-to-end**: 1 Playwright test (`frontend/e2e/demo.spec.ts`) automating the full
|
|
documented 5-minute demo script (login → dashboard → S1 return → S2 merge → S6 knowledge
|
|
question → audit → 360px responsive check) — **passing** against the live stack.
|
|
|
|
## Screenshots of the seven main pages
|
|
|
|
Captured live against the deterministic seed (`artifacts/evidence/screenshots/`,
|
|
via `frontend/e2e/_capture-screenshots.spec.ts`):
|
|
|
|
| # | Page | File |
|
|
|---|---|---|
|
|
| 1 | Login | `1-login.png` |
|
|
| 2 | Dashboard | `2-dashboard.png` |
|
|
| 3 | Vehicles | `3-vehicles.png` |
|
|
| 4 | Bookings | `4-bookings.png` |
|
|
| 5 | Data Quality | `5-data-quality.png` |
|
|
| 6 | Knowledge (grounded S6 answer) | `6-knowledge.png` |
|
|
| 7 | Automation | `7-automation.png` |
|
|
| — | Audit (bonus, 8th nav item) | `8-audit.png` |
|
|
| — | Dashboard at 360px (responsive proof) | `9-mobile-dashboard.png` |
|
|
|
|
## RAGcore evidence
|
|
|
|
**Success (demo provider, the one actually satisfying acceptance in this environment)** —
|
|
S6 question against the real `/api/v1/knowledge/questions` endpoint:
|
|
|
|
```json
|
|
{
|
|
"answer": "Per \"Vehicle return procedure\" (v2.0), section \"1. Register the return\": Open the active booking and record the ending odometer, fuel level, cleanliness, visible damage, technical warnings and relevant notes.",
|
|
"evidence_state": "grounded",
|
|
"sources": [
|
|
{"document_id": "vehicle-return-procedure", "title": "Vehicle return procedure", "version": "2.0", "section": "1. Register the return", "excerpt": "..."},
|
|
{"document_id": "vehicle-return-procedure", "title": "Vehicle return procedure", "version": "2.0", "section": "3. Determine next state", "excerpt": "..."},
|
|
{"document_id": "damage-procedure", "title": "Damage handling procedure", "version": "1.3", "section": "1. Immediate actions", "excerpt": "..."}
|
|
],
|
|
"provider": "demo",
|
|
"correlation_id": "b50094b7-1c84-4e39-9055-1dc03e8fd1f8"
|
|
}
|
|
```
|
|
|
|
**Unavailable (RAGcore adapter, live-demonstrated against an unreachable host)** —
|
|
`KNOWLEDGE_PROVIDER=ragcore`, `RAGCORE_BASE_URL=http://ragcore-not-reachable:9999`:
|
|
|
|
```
|
|
health: {'provider': 'ragcore', 'available': False, 'detail': 'RAGcore unavailable: ConnectError: ...', 'document_count': 0}
|
|
ask: {'answer': '', 'evidence_state': 'unavailable', 'sources': [], 'provider': 'ragcore', 'correlation_id': 'demo-correlation'}
|
|
```
|
|
|
|
No live RAGcore instance was reachable in this environment, so the adapter's actual
|
|
request/response contract against a real RAGcore is unverified beyond this
|
|
degrade-safely behavior — see `contracts/ragcore-contract-assumptions.md` and
|
|
`PROJECT_STATE.md`'s M5 notes.
|
|
|
|
## n8n evidence
|
|
|
|
**Success** — a real return registered on `BK-DEMO-RETURN`, delivered through the actual
|
|
n8n instance (not mocked), confirmed via `GET /api/v1/workflows`:
|
|
|
|
```json
|
|
{"event_id": "aa5dfeee-90ca-452a-bdd1-0a0b6d3dd63f", "event_type": "vehicle.returned.v1", "aggregate_ref": "BK-DEMO-RETURN", "status": "succeeded", "attempts": 2, "last_error": null}
|
|
```
|
|
|
|
(`attempts: 2` because the first delivery attempt landed while n8n was mid-restart from
|
|
the one-time workflow-activation step — the dispatcher's backoff-and-retry handled it
|
|
without any manual intervention, which is itself evidence of the retry behavior working.)
|
|
|
|
**Retry (S5 scenario)** — seeded `BK-H-0020` (event `00000000-...-0020`), initially
|
|
`failed` after 3 attempts with `"Synthetic connection timeout to n8n"`:
|
|
|
|
1. Before: `{"status": "failed", "attempts": 3, "last_error": "Synthetic connection timeout to n8n"}`
|
|
2. Operations Manager clicks Retry on `/automation`.
|
|
3. Within one ~3s dispatcher poll cycle, delivered through the live n8n instance.
|
|
4. After: `{"status": "succeeded", "attempts": 4, "last_error": null}`
|
|
|
|
## MCP tool sample calls
|
|
|
|
All four provider endpoints, authenticated with `X-Service-Token`:
|
|
|
|
```
|
|
$ curl -H "X-Service-Token: <token>" http://localhost:8128/api/v1/integrations/mcp/operations-summary
|
|
{"tenant":"northstar-mobility-demo","metrics":{"available":21,"rented":11,"cleaning":6,"maintenance":5,"blocked":7,"open_quality_issues":22,"pending_or_failed_workflows":1}}
|
|
|
|
$ curl -H "X-Service-Token: <token>" "http://localhost:8128/api/v1/integrations/mcp/attention-vehicles?minimum_severity=high&limit=3"
|
|
[{"vehicle_ref":"MO-016","severity":"high","rule_type":"booking_overlap",...},
|
|
{"vehicle_ref":"MO-016","severity":"high","rule_type":"vehicle_status_conflict",...},
|
|
{"vehicle_ref":"MO-031","severity":"high","rule_type":"missing_required_field",...}]
|
|
|
|
$ curl -H "X-Service-Token: <token>" http://localhost:8128/api/v1/integrations/mcp/vehicles/MO-016
|
|
{"public_ref":"MO-016","make":"Hymer","model":"Exsis","model_year":2021,"location":"Geel","operational_status":"available","odometer_km":30497,"next_service_km":40000,"open_quality_issue_count":2,"current_booking_ref":null}
|
|
|
|
$ curl -H "X-Service-Token: <token>" -X POST -d '{"question":"What must I do when a vehicle returns with damage?","max_sources":2}' http://localhost:8128/api/v1/integrations/mcp/search-knowledge
|
|
{"answer":"Per \"Vehicle return procedure\" ...","evidence_state":"grounded","sources":[...2 items...],"provider":"demo",...}
|
|
```
|
|
|
|
Auth verified: missing header → `422`; wrong token → `401`. All four calls confirmed
|
|
recorded in `GET /api/v1/audit?action=mcp_tool_request` with `actor_type: "service"`.
|
|
|
|
No live ITWorx MCP Hub instance was reachable in this environment — these are direct
|
|
calls to MobilityOps's own provider endpoints, not a Hub round trip.
|
|
|
|
## Known PoC limitations
|
|
|
|
- **RAGcore and ITWorx MCP Hub were never reachable in this build environment.** Both
|
|
integrations are implemented against best-effort/documented contracts and are
|
|
unit/contract-tested (including their failure-degradation paths), but neither was
|
|
verified against a real counterpart service. The demo `KnowledgeProvider` is what
|
|
actually satisfies the knowledge-assistant acceptance criteria here.
|
|
- **n8n requires a one-time manual owner-account setup** per fresh environment
|
|
(`docker compose down -v` wipes it) — this is a property of the n8n 2.x image itself
|
|
(`N8N_BASIC_AUTH_ACTIVE` no longer gates the UI), not something MobilityOps can bypass.
|
|
Documented precisely in `docs/17-runbook.md`; the workflow import/activation itself
|
|
*is* scripted (`make n8n-setup`).
|
|
- **Inspection public refs are a simple `count+1` sequence**, not gap-safe under true
|
|
concurrent writers — acceptable for this single-tenant demo, would need a DB sequence
|
|
for a multi-writer production system.
|
|
- **The five data-quality rules use simplified idempotency** — `(rule_type, entity_type,
|
|
entity_id)` while open, rather than the doc's literal evidence-fingerprint scheme — see
|
|
`PROJECT_STATE.md`'s M3 notes for the reasoning (the fingerprint scheme would have let
|
|
the scan double-report issues already present in the seeded CSV).
|
|
- **No production authentication** — demo login is an HMAC-signed session cookie tied to
|
|
two fixed seeded users, appropriate for a PoC, not a real identity provider.
|
|
|
|
## Portfolio wording (truthful)
|
|
|
|
MobilityOps is a working proof of concept, not a production system and not deployed for
|
|
any real company. All customers, vehicles, bookings, and documents are synthetic
|
|
(deterministically generated). The application logic it demonstrates is real: a
|
|
transactional vehicle-return workflow with idempotency and concurrency control tested
|
|
against real concurrent database transactions; five explainable, deterministic
|
|
data-quality rules with a working customer-merge UI; a background outbox dispatcher
|
|
verified end-to-end against a real n8n instance including failure/retry; a
|
|
TF-IDF-weighted extractive knowledge assistant that never fabricates answers; and four
|
|
read-only, audited, service-authenticated integration endpoints. RAGcore and the ITWorx
|
|
MCP Hub integrations are implemented and tested in isolation but were not verified
|
|
against live instances of those systems in this environment.
|