M20: complete production acceptance
This commit is contained in:
NuklearRabbit
@@ -1,177 +1,83 @@
|
||||
# MobilityOps — final acceptance evidence
|
||||
|
||||
## Commit
|
||||
## Accepted implementation
|
||||
|
||||
Built on top of commit `c5b7e21f81694f0339ad31e3bf044db952d0fbe0` (M6, "implement ITWorx
|
||||
MCP Hub publication"). This evidence file and the rest of M7's polish are committed as
|
||||
`M7: portfolio polish and final acceptance` — run `git log --oneline` for the exact hash.
|
||||
- Implementation under test: `5dda574` (`master`). The final evidence commit contains
|
||||
documentation only and does not change the accepted runtime.
|
||||
- Live URL: `http://192.168.10.150:1236`.
|
||||
- Deployment: Unraid, `/mnt/user/appdata/mobilityops`, Compose project `mobilityops`.
|
||||
- Data: deterministic, explicitly labelled synthetic demo data.
|
||||
- Database migration: `f43d829ab610 (head)`.
|
||||
|
||||
## Exact commands (clean checkout)
|
||||
## Clean acceptance results — 2026-08-10
|
||||
|
||||
```bash
|
||||
git clone <repo> && cd MobilityOps
|
||||
cp .env.example .env
|
||||
make demo # docker compose up --build -d ; migrations run automatically ; seed --reset
|
||||
```text
|
||||
Backend, isolated PostgreSQL: 225 passed, 1 dependency deprecation warning (96.71 s)
|
||||
Playwright, live production: 145 passed (4.4 min)
|
||||
Ruff: clean
|
||||
Mypy: clean
|
||||
Frontend TypeScript/Vite: production build passed
|
||||
npm audit (prod and full): 0 vulnerabilities
|
||||
Alembic: f43d829ab610 (head)
|
||||
```
|
||||
|
||||
One-time n8n setup (see `docs/17-runbook.md` for full detail — this cannot be scripted
|
||||
end-to-end because it requires a one-time owner account created through n8n's web UI):
|
||||
Backend tests ran through `compose.yaml` plus `compose.test.yaml` in the disposable
|
||||
`mobilityops-test` project. Its database, network and volume were removed afterward; no
|
||||
test process ran inside the live API container.
|
||||
|
||||
```bash
|
||||
# open http://localhost:5678/setup in a browser, create any owner account
|
||||
make n8n-setup
|
||||
The single serial Playwright run covers 145 browser scenarios: both roles, protected
|
||||
routes, booking lifecycle and return, maintenance/release, user administration,
|
||||
data-quality scanning/assignment/resolution, audit, retryable automation, Knowledge Hub,
|
||||
the complete guided demo, keyboard accessibility and responsive layouts from 360 to
|
||||
1440 px.
|
||||
|
||||
## Live operational evidence
|
||||
|
||||
After acceptance, demo reset restored an internally checked hand-off state:
|
||||
|
||||
```text
|
||||
users 2 · customers 180 · vehicles 50 · bookings 254 · inspections 75
|
||||
maintenance 40 · data-quality issues 33 · workflow runs 20
|
||||
scenario_integrity.all_ready = true
|
||||
```
|
||||
|
||||
Verification:
|
||||
Readiness returned `ready` with PostgreSQL `up`; API, database and web containers were
|
||||
healthy. A verified custom-format database backup was written to:
|
||||
|
||||
```bash
|
||||
docker compose run --rm api pytest -q # 66 passed
|
||||
docker compose run --rm api ruff check . # All checks passed
|
||||
cd frontend && npm run build # clean tsc + vite build
|
||||
cd frontend && npx playwright test # 1 passed (full 5-minute demo script)
|
||||
```
|
||||
`/mnt/user/appdata/mobilityops/backups/postgres/mobilityops-20260810T111913Z.dump`
|
||||
|
||||
## Test counts
|
||||
## External integrations
|
||||
|
||||
- **Backend**: 66 tests passing (`pytest`), 0 skipped, 0 failed. Ruff clean. Coverage by
|
||||
area: seed determinism (2), auth/roles (4), dashboard (3), vehicles (4), bookings (3),
|
||||
return workflow incl. real concurrent-submission test (9), data quality incl. S2/S4
|
||||
scenarios (10), audit (2), n8n dispatcher incl. malformed-payload regression (6),
|
||||
n8n callback idempotency (3), workflows/retry (4), knowledge incl. S6 scenario (7),
|
||||
MCP provider endpoints (8), health (1).
|
||||
- **Frontend**: `npm run build` — clean TypeScript + Vite build, zero errors.
|
||||
- **End-to-end**: 1 Playwright test (`frontend/e2e/demo.spec.ts`) automating the full
|
||||
documented 5-minute demo script (login → dashboard → S1 return → S2 merge → S6 knowledge
|
||||
question → audit → 360px responsive check) — **passing** against the live stack.
|
||||
- **n8n:** MobilityOps uses the existing central server instance, not a second n8n.
|
||||
Dispatch is enabled and operational with 19 successful persisted runs, zero pending,
|
||||
zero unexpected failures and healthy heartbeat evidence for the vehicle-return and
|
||||
scheduled-quality workflows. One failed item is an explicitly labelled synthetic
|
||||
retry scenario and is excluded from incident counts.
|
||||
- **RAGcore:** live health returned available/ready for tenant
|
||||
`northstar-mobility-demo`, workspace `mobilityops`, collection
|
||||
`internal-procedures`. A live question returned a grounded answer in 5.1 seconds with
|
||||
three unique, collapsible procedure citations. The UI never substitutes a fabricated
|
||||
answer when evidence is insufficient or the provider is unavailable.
|
||||
- **ITWorx MCP Hub:** registration is enabled, the Hub is reachable and MobilityOps
|
||||
reports an audited real `fleet_ops_get_operations_summary` tool call. Tool endpoints
|
||||
are read-only, tenant-bound, client-identity validated, service-authenticated and emit
|
||||
correlation IDs.
|
||||
|
||||
## Screenshots of the seven main pages
|
||||
## Visual acceptance
|
||||
|
||||
Captured live against the deterministic seed (`artifacts/evidence/screenshots/`,
|
||||
via `frontend/e2e/_capture-screenshots.spec.ts`):
|
||||
The production UI was interactively inspected after the automated run. Desktop
|
||||
dashboard, mobile data-quality workbench, Integration Management and Knowledge Hub had
|
||||
no overlap or horizontal page overflow. Filters, SLA/assignment controls, integration
|
||||
states, citations, feedback controls, keyboard landmarks and mobile navigation remained
|
||||
visible and operable. Route-level lazy loading keeps the initial JavaScript chunk near
|
||||
212 kB instead of the former 572 kB monolith.
|
||||
|
||||
| # | Page | File |
|
||||
|---|---|---|
|
||||
| 1 | Login | `1-login.png` |
|
||||
| 2 | Dashboard | `2-dashboard.png` |
|
||||
| 3 | Vehicles | `3-vehicles.png` |
|
||||
| 4 | Bookings | `4-bookings.png` |
|
||||
| 5 | Data Quality | `5-data-quality.png` |
|
||||
| 6 | Knowledge (grounded S6 answer) | `6-knowledge.png` |
|
||||
| 7 | Automation | `7-automation.png` |
|
||||
| — | Audit (bonus, 8th nav item) | `8-audit.png` |
|
||||
| — | Dashboard at 360px (responsive proof) | `9-mobile-dashboard.png` |
|
||||
## Deliberate scope boundary
|
||||
|
||||
## RAGcore evidence
|
||||
|
||||
**Success (demo provider, the one actually satisfying acceptance in this environment)** —
|
||||
S6 question against the real `/api/v1/knowledge/questions` endpoint:
|
||||
|
||||
```json
|
||||
{
|
||||
"answer": "Per \"Vehicle return procedure\" (v2.0), section \"1. Register the return\": Open the active booking and record the ending odometer, fuel level, cleanliness, visible damage, technical warnings and relevant notes.",
|
||||
"evidence_state": "grounded",
|
||||
"sources": [
|
||||
{"document_id": "vehicle-return-procedure", "title": "Vehicle return procedure", "version": "2.0", "section": "1. Register the return", "excerpt": "..."},
|
||||
{"document_id": "vehicle-return-procedure", "title": "Vehicle return procedure", "version": "2.0", "section": "3. Determine next state", "excerpt": "..."},
|
||||
{"document_id": "damage-procedure", "title": "Damage handling procedure", "version": "1.3", "section": "1. Immediate actions", "excerpt": "..."}
|
||||
],
|
||||
"provider": "demo",
|
||||
"correlation_id": "b50094b7-1c84-4e39-9055-1dc03e8fd1f8"
|
||||
}
|
||||
```
|
||||
|
||||
**Unavailable (RAGcore adapter, live-demonstrated against an unreachable host)** —
|
||||
`KNOWLEDGE_PROVIDER=ragcore`, `RAGCORE_BASE_URL=http://ragcore-not-reachable:9999`:
|
||||
|
||||
```
|
||||
health: {'provider': 'ragcore', 'available': False, 'detail': 'RAGcore unavailable: ConnectError: ...', 'document_count': 0}
|
||||
ask: {'answer': '', 'evidence_state': 'unavailable', 'sources': [], 'provider': 'ragcore', 'correlation_id': 'demo-correlation'}
|
||||
```
|
||||
|
||||
No live RAGcore instance was reachable in this environment, so the adapter's actual
|
||||
request/response contract against a real RAGcore is unverified beyond this
|
||||
degrade-safely behavior — see `contracts/ragcore-contract-assumptions.md` and
|
||||
`PROJECT_STATE.md`'s M5 notes.
|
||||
|
||||
## n8n evidence
|
||||
|
||||
**Success** — a real return registered on `BK-DEMO-RETURN`, delivered through the actual
|
||||
n8n instance (not mocked), confirmed via `GET /api/v1/workflows`:
|
||||
|
||||
```json
|
||||
{"event_id": "aa5dfeee-90ca-452a-bdd1-0a0b6d3dd63f", "event_type": "vehicle.returned.v1", "aggregate_ref": "BK-DEMO-RETURN", "status": "succeeded", "attempts": 2, "last_error": null}
|
||||
```
|
||||
|
||||
(`attempts: 2` because the first delivery attempt landed while n8n was mid-restart from
|
||||
the one-time workflow-activation step — the dispatcher's backoff-and-retry handled it
|
||||
without any manual intervention, which is itself evidence of the retry behavior working.)
|
||||
|
||||
**Retry (S5 scenario)** — seeded `BK-H-0020` (event `00000000-...-0020`), initially
|
||||
`failed` after 3 attempts with `"Synthetic connection timeout to n8n"`:
|
||||
|
||||
1. Before: `{"status": "failed", "attempts": 3, "last_error": "Synthetic connection timeout to n8n"}`
|
||||
2. Operations Manager clicks Retry on `/automation`.
|
||||
3. Within one ~3s dispatcher poll cycle, delivered through the live n8n instance.
|
||||
4. After: `{"status": "succeeded", "attempts": 4, "last_error": null}`
|
||||
|
||||
## MCP tool sample calls
|
||||
|
||||
All four provider endpoints, authenticated with `X-Service-Token`:
|
||||
|
||||
```
|
||||
$ curl -H "X-Service-Token: <token>" http://localhost:8128/api/v1/integrations/mcp/operations-summary
|
||||
{"tenant":"northstar-mobility-demo","metrics":{"available":21,"rented":11,"cleaning":6,"maintenance":5,"blocked":7,"open_quality_issues":22,"pending_or_failed_workflows":1}}
|
||||
|
||||
$ curl -H "X-Service-Token: <token>" "http://localhost:8128/api/v1/integrations/mcp/attention-vehicles?minimum_severity=high&limit=3"
|
||||
[{"vehicle_ref":"MO-016","severity":"high","rule_type":"booking_overlap",...},
|
||||
{"vehicle_ref":"MO-016","severity":"high","rule_type":"vehicle_status_conflict",...},
|
||||
{"vehicle_ref":"MO-031","severity":"high","rule_type":"missing_required_field",...}]
|
||||
|
||||
$ curl -H "X-Service-Token: <token>" http://localhost:8128/api/v1/integrations/mcp/vehicles/MO-016
|
||||
{"public_ref":"MO-016","make":"Hymer","model":"Exsis","model_year":2021,"location":"Geel","operational_status":"available","odometer_km":30497,"next_service_km":40000,"open_quality_issue_count":2,"current_booking_ref":null}
|
||||
|
||||
$ curl -H "X-Service-Token: <token>" -X POST -d '{"question":"What must I do when a vehicle returns with damage?","max_sources":2}' http://localhost:8128/api/v1/integrations/mcp/search-knowledge
|
||||
{"answer":"Per \"Vehicle return procedure\" ...","evidence_state":"grounded","sources":[...2 items...],"provider":"demo",...}
|
||||
```
|
||||
|
||||
Auth verified: missing header → `422`; wrong token → `401`. All four calls confirmed
|
||||
recorded in `GET /api/v1/audit?action=mcp_tool_request` with `actor_type: "service"`.
|
||||
|
||||
No live ITWorx MCP Hub instance was reachable in this environment — these are direct
|
||||
calls to MobilityOps's own provider endpoints, not a Hub round trip.
|
||||
|
||||
## Known PoC limitations
|
||||
|
||||
- **RAGcore and ITWorx MCP Hub were never reachable in this build environment.** Both
|
||||
integrations are implemented against best-effort/documented contracts and are
|
||||
unit/contract-tested (including their failure-degradation paths), but neither was
|
||||
verified against a real counterpart service. The demo `KnowledgeProvider` is what
|
||||
actually satisfies the knowledge-assistant acceptance criteria here.
|
||||
- **n8n requires a one-time manual owner-account setup** per fresh environment
|
||||
(`docker compose down -v` wipes it) — this is a property of the n8n 2.x image itself
|
||||
(`N8N_BASIC_AUTH_ACTIVE` no longer gates the UI), not something MobilityOps can bypass.
|
||||
Documented precisely in `docs/17-runbook.md`; the workflow import/activation itself
|
||||
*is* scripted (`make n8n-setup`).
|
||||
- **Inspection public refs are a simple `count+1` sequence**, not gap-safe under true
|
||||
concurrent writers — acceptable for this single-tenant demo, would need a DB sequence
|
||||
for a multi-writer production system.
|
||||
- **The five data-quality rules use simplified idempotency** — `(rule_type, entity_type,
|
||||
entity_id)` while open, rather than the doc's literal evidence-fingerprint scheme — see
|
||||
`PROJECT_STATE.md`'s M3 notes for the reasoning (the fingerprint scheme would have let
|
||||
the scan double-report issues already present in the seeded CSV).
|
||||
- **No production authentication** — demo login is an HMAC-signed session cookie tied to
|
||||
two fixed seeded users, appropriate for a PoC, not a real identity provider.
|
||||
|
||||
## Portfolio wording (truthful)
|
||||
|
||||
MobilityOps is a working proof of concept, not a production system and not deployed for
|
||||
any real company. All customers, vehicles, bookings, and documents are synthetic
|
||||
(deterministically generated). The application logic it demonstrates is real: a
|
||||
transactional vehicle-return workflow with idempotency and concurrency control tested
|
||||
against real concurrent database transactions; five explainable, deterministic
|
||||
data-quality rules with a working customer-merge UI; a background outbox dispatcher
|
||||
verified end-to-end against a real n8n instance including failure/retry; a
|
||||
TF-IDF-weighted extractive knowledge assistant that never fabricates answers; and four
|
||||
read-only, audited, service-authenticated integration endpoints. RAGcore and the ITWorx
|
||||
MCP Hub integrations are implemented and tested in isolation but were not verified
|
||||
against live instances of those systems in this environment.
|
||||
This is the completed locked MobilityOps PoC, not an unrestricted ERP. It intentionally
|
||||
does not add accounting, payments, public reservations, CRM, inventory, HR, a second RAG
|
||||
stack, a separate MCP server or autonomous write tools. Production deployment for real
|
||||
personal data would still require an organisation identity provider, secrets rotation,
|
||||
central observability/alerting, retention policy and an operated disaster-recovery
|
||||
schedule. Those are deployment-governance concerns, not dead or simulated product
|
||||
functions in the accepted scope.
|
||||
|
||||
Reference in New Issue
Block a user