diff --git a/PROJECT_STATE.md b/PROJECT_STATE.md index bffebb7..1bc835e 100644 --- a/PROJECT_STATE.md +++ b/PROJECT_STATE.md @@ -2480,3 +2480,26 @@ evidence yet." and mypy are clean. - Exact next action: commit and deploy this milestone, run complete Playwright and visual acceptance against Unraid, refresh final evidence, push and verify the live hash. + +## M20 — final production acceptance and hand-off (2026-08-10) + +- Tightened RAGcore citation identity to title plus named section. Multiple chunks from + the same unsectioned document now render as one source card, while distinct named + sections remain independently citable. +- Stabilized acceptance selectors around persistent business identity instead of table + position: the guided demo filters for the generated MO-024 odometer issue, vehicle + attention checks use semantic reason markers, and reassigned work verifies the exact + captured issue reference after the queue reorders. +- Validation evidence: targeted knowledge **31 passed**; complete isolated PostgreSQL + backend **225 passed, 1 dependency warning**; complete live production Playwright + **145 passed in 4.4 minutes**. The live visual audit covered desktop dashboard, mobile + data-quality, Integration Management and a grounded RAGcore exchange without overlap + or horizontal page overflow. +- Live hand-off: readiness is `ready`; migration is `f43d829ab610 (head)`; API, database + and web containers are healthy. Reset restored all scenarios with the canonical 2 / 180 + / 50 / 254 / 75 / 40 / 33 / 20 entity counts. Central n8n, MCP Hub and RAGcore all + report operational/available; no separate n8n was started. +- Recovery evidence: verified live backup + `/mnt/user/appdata/mobilityops/backups/postgres/mobilityops-20260810T111913Z.dump`. +- Exact next action: none for the locked PoC. Routine operation, monitoring and any scope + expansion require a separate approved milestone. diff --git a/artifacts/evidence/final-summary.md b/artifacts/evidence/final-summary.md index e303d6e..17969c8 100644 --- a/artifacts/evidence/final-summary.md +++ b/artifacts/evidence/final-summary.md @@ -1,177 +1,83 @@ # MobilityOps — final acceptance evidence -## Commit +## Accepted implementation -Built on top of commit `c5b7e21f81694f0339ad31e3bf044db952d0fbe0` (M6, "implement ITWorx -MCP Hub publication"). This evidence file and the rest of M7's polish are committed as -`M7: portfolio polish and final acceptance` — run `git log --oneline` for the exact hash. +- Implementation under test: `5dda574` (`master`). The final evidence commit contains + documentation only and does not change the accepted runtime. +- Live URL: `http://192.168.10.150:1236`. +- Deployment: Unraid, `/mnt/user/appdata/mobilityops`, Compose project `mobilityops`. +- Data: deterministic, explicitly labelled synthetic demo data. +- Database migration: `f43d829ab610 (head)`. -## Exact commands (clean checkout) +## Clean acceptance results — 2026-08-10 -```bash -git clone && cd MobilityOps -cp .env.example .env -make demo # docker compose up --build -d ; migrations run automatically ; seed --reset +```text +Backend, isolated PostgreSQL: 225 passed, 1 dependency deprecation warning (96.71 s) +Playwright, live production: 145 passed (4.4 min) +Ruff: clean +Mypy: clean +Frontend TypeScript/Vite: production build passed +npm audit (prod and full): 0 vulnerabilities +Alembic: f43d829ab610 (head) ``` -One-time n8n setup (see `docs/17-runbook.md` for full detail — this cannot be scripted -end-to-end because it requires a one-time owner account created through n8n's web UI): +Backend tests ran through `compose.yaml` plus `compose.test.yaml` in the disposable +`mobilityops-test` project. Its database, network and volume were removed afterward; no +test process ran inside the live API container. -```bash -# open http://localhost:5678/setup in a browser, create any owner account -make n8n-setup +The single serial Playwright run covers 145 browser scenarios: both roles, protected +routes, booking lifecycle and return, maintenance/release, user administration, +data-quality scanning/assignment/resolution, audit, retryable automation, Knowledge Hub, +the complete guided demo, keyboard accessibility and responsive layouts from 360 to +1440 px. + +## Live operational evidence + +After acceptance, demo reset restored an internally checked hand-off state: + +```text +users 2 · customers 180 · vehicles 50 · bookings 254 · inspections 75 +maintenance 40 · data-quality issues 33 · workflow runs 20 +scenario_integrity.all_ready = true ``` -Verification: +Readiness returned `ready` with PostgreSQL `up`; API, database and web containers were +healthy. A verified custom-format database backup was written to: -```bash -docker compose run --rm api pytest -q # 66 passed -docker compose run --rm api ruff check . # All checks passed -cd frontend && npm run build # clean tsc + vite build -cd frontend && npx playwright test # 1 passed (full 5-minute demo script) -``` +`/mnt/user/appdata/mobilityops/backups/postgres/mobilityops-20260810T111913Z.dump` -## Test counts +## External integrations -- **Backend**: 66 tests passing (`pytest`), 0 skipped, 0 failed. Ruff clean. Coverage by - area: seed determinism (2), auth/roles (4), dashboard (3), vehicles (4), bookings (3), - return workflow incl. real concurrent-submission test (9), data quality incl. S2/S4 - scenarios (10), audit (2), n8n dispatcher incl. malformed-payload regression (6), - n8n callback idempotency (3), workflows/retry (4), knowledge incl. S6 scenario (7), - MCP provider endpoints (8), health (1). -- **Frontend**: `npm run build` — clean TypeScript + Vite build, zero errors. -- **End-to-end**: 1 Playwright test (`frontend/e2e/demo.spec.ts`) automating the full - documented 5-minute demo script (login → dashboard → S1 return → S2 merge → S6 knowledge - question → audit → 360px responsive check) — **passing** against the live stack. +- **n8n:** MobilityOps uses the existing central server instance, not a second n8n. + Dispatch is enabled and operational with 19 successful persisted runs, zero pending, + zero unexpected failures and healthy heartbeat evidence for the vehicle-return and + scheduled-quality workflows. One failed item is an explicitly labelled synthetic + retry scenario and is excluded from incident counts. +- **RAGcore:** live health returned available/ready for tenant + `northstar-mobility-demo`, workspace `mobilityops`, collection + `internal-procedures`. A live question returned a grounded answer in 5.1 seconds with + three unique, collapsible procedure citations. The UI never substitutes a fabricated + answer when evidence is insufficient or the provider is unavailable. +- **ITWorx MCP Hub:** registration is enabled, the Hub is reachable and MobilityOps + reports an audited real `fleet_ops_get_operations_summary` tool call. Tool endpoints + are read-only, tenant-bound, client-identity validated, service-authenticated and emit + correlation IDs. -## Screenshots of the seven main pages +## Visual acceptance -Captured live against the deterministic seed (`artifacts/evidence/screenshots/`, -via `frontend/e2e/_capture-screenshots.spec.ts`): +The production UI was interactively inspected after the automated run. Desktop +dashboard, mobile data-quality workbench, Integration Management and Knowledge Hub had +no overlap or horizontal page overflow. Filters, SLA/assignment controls, integration +states, citations, feedback controls, keyboard landmarks and mobile navigation remained +visible and operable. Route-level lazy loading keeps the initial JavaScript chunk near +212 kB instead of the former 572 kB monolith. -| # | Page | File | -|---|---|---| -| 1 | Login | `1-login.png` | -| 2 | Dashboard | `2-dashboard.png` | -| 3 | Vehicles | `3-vehicles.png` | -| 4 | Bookings | `4-bookings.png` | -| 5 | Data Quality | `5-data-quality.png` | -| 6 | Knowledge (grounded S6 answer) | `6-knowledge.png` | -| 7 | Automation | `7-automation.png` | -| — | Audit (bonus, 8th nav item) | `8-audit.png` | -| — | Dashboard at 360px (responsive proof) | `9-mobile-dashboard.png` | +## Deliberate scope boundary -## RAGcore evidence - -**Success (demo provider, the one actually satisfying acceptance in this environment)** — -S6 question against the real `/api/v1/knowledge/questions` endpoint: - -```json -{ - "answer": "Per \"Vehicle return procedure\" (v2.0), section \"1. Register the return\": Open the active booking and record the ending odometer, fuel level, cleanliness, visible damage, technical warnings and relevant notes.", - "evidence_state": "grounded", - "sources": [ - {"document_id": "vehicle-return-procedure", "title": "Vehicle return procedure", "version": "2.0", "section": "1. Register the return", "excerpt": "..."}, - {"document_id": "vehicle-return-procedure", "title": "Vehicle return procedure", "version": "2.0", "section": "3. Determine next state", "excerpt": "..."}, - {"document_id": "damage-procedure", "title": "Damage handling procedure", "version": "1.3", "section": "1. Immediate actions", "excerpt": "..."} - ], - "provider": "demo", - "correlation_id": "b50094b7-1c84-4e39-9055-1dc03e8fd1f8" -} -``` - -**Unavailable (RAGcore adapter, live-demonstrated against an unreachable host)** — -`KNOWLEDGE_PROVIDER=ragcore`, `RAGCORE_BASE_URL=http://ragcore-not-reachable:9999`: - -``` -health: {'provider': 'ragcore', 'available': False, 'detail': 'RAGcore unavailable: ConnectError: ...', 'document_count': 0} -ask: {'answer': '', 'evidence_state': 'unavailable', 'sources': [], 'provider': 'ragcore', 'correlation_id': 'demo-correlation'} -``` - -No live RAGcore instance was reachable in this environment, so the adapter's actual -request/response contract against a real RAGcore is unverified beyond this -degrade-safely behavior — see `contracts/ragcore-contract-assumptions.md` and -`PROJECT_STATE.md`'s M5 notes. - -## n8n evidence - -**Success** — a real return registered on `BK-DEMO-RETURN`, delivered through the actual -n8n instance (not mocked), confirmed via `GET /api/v1/workflows`: - -```json -{"event_id": "aa5dfeee-90ca-452a-bdd1-0a0b6d3dd63f", "event_type": "vehicle.returned.v1", "aggregate_ref": "BK-DEMO-RETURN", "status": "succeeded", "attempts": 2, "last_error": null} -``` - -(`attempts: 2` because the first delivery attempt landed while n8n was mid-restart from -the one-time workflow-activation step — the dispatcher's backoff-and-retry handled it -without any manual intervention, which is itself evidence of the retry behavior working.) - -**Retry (S5 scenario)** — seeded `BK-H-0020` (event `00000000-...-0020`), initially -`failed` after 3 attempts with `"Synthetic connection timeout to n8n"`: - -1. Before: `{"status": "failed", "attempts": 3, "last_error": "Synthetic connection timeout to n8n"}` -2. Operations Manager clicks Retry on `/automation`. -3. Within one ~3s dispatcher poll cycle, delivered through the live n8n instance. -4. After: `{"status": "succeeded", "attempts": 4, "last_error": null}` - -## MCP tool sample calls - -All four provider endpoints, authenticated with `X-Service-Token`: - -``` -$ curl -H "X-Service-Token: " http://localhost:8128/api/v1/integrations/mcp/operations-summary -{"tenant":"northstar-mobility-demo","metrics":{"available":21,"rented":11,"cleaning":6,"maintenance":5,"blocked":7,"open_quality_issues":22,"pending_or_failed_workflows":1}} - -$ curl -H "X-Service-Token: " "http://localhost:8128/api/v1/integrations/mcp/attention-vehicles?minimum_severity=high&limit=3" -[{"vehicle_ref":"MO-016","severity":"high","rule_type":"booking_overlap",...}, - {"vehicle_ref":"MO-016","severity":"high","rule_type":"vehicle_status_conflict",...}, - {"vehicle_ref":"MO-031","severity":"high","rule_type":"missing_required_field",...}] - -$ curl -H "X-Service-Token: " http://localhost:8128/api/v1/integrations/mcp/vehicles/MO-016 -{"public_ref":"MO-016","make":"Hymer","model":"Exsis","model_year":2021,"location":"Geel","operational_status":"available","odometer_km":30497,"next_service_km":40000,"open_quality_issue_count":2,"current_booking_ref":null} - -$ curl -H "X-Service-Token: " -X POST -d '{"question":"What must I do when a vehicle returns with damage?","max_sources":2}' http://localhost:8128/api/v1/integrations/mcp/search-knowledge -{"answer":"Per \"Vehicle return procedure\" ...","evidence_state":"grounded","sources":[...2 items...],"provider":"demo",...} -``` - -Auth verified: missing header → `422`; wrong token → `401`. All four calls confirmed -recorded in `GET /api/v1/audit?action=mcp_tool_request` with `actor_type: "service"`. - -No live ITWorx MCP Hub instance was reachable in this environment — these are direct -calls to MobilityOps's own provider endpoints, not a Hub round trip. - -## Known PoC limitations - -- **RAGcore and ITWorx MCP Hub were never reachable in this build environment.** Both - integrations are implemented against best-effort/documented contracts and are - unit/contract-tested (including their failure-degradation paths), but neither was - verified against a real counterpart service. The demo `KnowledgeProvider` is what - actually satisfies the knowledge-assistant acceptance criteria here. -- **n8n requires a one-time manual owner-account setup** per fresh environment - (`docker compose down -v` wipes it) — this is a property of the n8n 2.x image itself - (`N8N_BASIC_AUTH_ACTIVE` no longer gates the UI), not something MobilityOps can bypass. - Documented precisely in `docs/17-runbook.md`; the workflow import/activation itself - *is* scripted (`make n8n-setup`). -- **Inspection public refs are a simple `count+1` sequence**, not gap-safe under true - concurrent writers — acceptable for this single-tenant demo, would need a DB sequence - for a multi-writer production system. -- **The five data-quality rules use simplified idempotency** — `(rule_type, entity_type, - entity_id)` while open, rather than the doc's literal evidence-fingerprint scheme — see - `PROJECT_STATE.md`'s M3 notes for the reasoning (the fingerprint scheme would have let - the scan double-report issues already present in the seeded CSV). -- **No production authentication** — demo login is an HMAC-signed session cookie tied to - two fixed seeded users, appropriate for a PoC, not a real identity provider. - -## Portfolio wording (truthful) - -MobilityOps is a working proof of concept, not a production system and not deployed for -any real company. All customers, vehicles, bookings, and documents are synthetic -(deterministically generated). The application logic it demonstrates is real: a -transactional vehicle-return workflow with idempotency and concurrency control tested -against real concurrent database transactions; five explainable, deterministic -data-quality rules with a working customer-merge UI; a background outbox dispatcher -verified end-to-end against a real n8n instance including failure/retry; a -TF-IDF-weighted extractive knowledge assistant that never fabricates answers; and four -read-only, audited, service-authenticated integration endpoints. RAGcore and the ITWorx -MCP Hub integrations are implemented and tested in isolation but were not verified -against live instances of those systems in this environment. +This is the completed locked MobilityOps PoC, not an unrestricted ERP. It intentionally +does not add accounting, payments, public reservations, CRM, inventory, HR, a second RAG +stack, a separate MCP server or autonomous write tools. Production deployment for real +personal data would still require an organisation identity provider, secrets rotation, +central observability/alerting, retention policy and an operated disaster-recovery +schedule. Those are deployment-governance concerns, not dead or simulated product +functions in the accepted scope.