Files
MobilityOps/docs/demo-release/current-demo-gap-audit.md
T
NuklearRabbit 7c94eb9e87 docs(demo): audit current demo readiness gaps
Confirms the underlying data/business-logic is already demo-grade (Dutch/
Flemish names, .test emails, believable Belgian towns and RV brands; the
5 requested scenarios already exist as S1/S2/S4/S5/S6 in
docs/13-seed-and-demo-scenarios.md) -- the real gaps are structural: no
guided path, no visible fictional org identity (Northstar Mobility is
already the locked tenant name internally, just never shown), a
reproducible date-anchoring bug (seed dates are absolute and don't move
with reset -- BK-DEMO-RETURN's end date is already in the past as of
today), the knowledge page naming "RAGcore" directly instead of "demo
mode", technical-register integration-status labels, and no About page.
2026-08-03 12:54:27 +02:00

10 KiB
Raw Blame History

Current demo-readiness gap audit

Performed 2026-08-03 against feat/mobilityops-functional-completion @ e0c7ed60112510687627d20a957af91c8b9db7f8 (verified: correct branch, working tree clean, matches the deployed Unraid revision). Method: read CLAUDE.md/AGENTS.md (identical, no demo-specific guidance yet), README.md, PROJECT_STATE.md, the docs list in the brief, artifacts/final-acceptance/summary.md, artifacts/functional-completion/final-summary.md, seed/generate_seed.py, seed/*.csv, backend/app/seed_loader.py, backend/app/core/config.py, backend/app/api/routers/dashboard.py; live-browsed the deployed app at http://192.168.10.150:1236 (login, dashboard, a data-quality issue, a booking detail, knowledge, automation).

What already works well (do not rebuild)

  • The premium Control Rail UI, role-based nav, server-backed sessions, return preview/ commit, all five data-quality resolution flows, audit before/after, role-aware search, truthful integration status, demo reset, stale-lease recovery and the two n8n workflows are all implemented, tested and live-verified per artifacts/functional-completion/final-summary.md — this pass does not touch that business logic.
  • docs/13-seed-and-demo-scenarios.md already names exactly the scenarios this brief wants surfaced: S1 odometer-regression return (BK-DEMO-RETURN/MO-024), S2 duplicate customer (CUS-0012/CUS-0178, issue DQ-DEMO-DUPLICATE), S4 booking overlap (MO-016, issue DQ-DEMO-OVERLAP, bookings BK-DEMO-OVERLAP-A/B), S5 failed workflow (seeded failed outbox event, retryable from Automation), S6 grounded knowledge question. These map directly to the brief's 5 requested scenarios — no new scenarios need to be invented, only surfaced with guidance. (S3, "missing inspection before next booking" on MO-031, exists too but isn't one of the 5 requested; leave it as-is, it still shows up in the attention queue.)
  • seed/generate_seed.py already produces believable Flemish/Dutch data: first/last name pools (Sofie, Lotte, Bram, Peeters, Janssens, Vermeulen, ...), real Kempen-region towns (Geel, Turnhout, Herentals, Mol, Westerlo, Tessenderlo-Ham), .test email addresses, and believable RV/camper makes (Adria, Dethleffs, Hymer, Bürstner, ...). Confirmed live: DQ-DEMO-DUPLICATE shows customer "Bram Peeters" (Geel) with email bram.peeters.12@example.test. The underlying data quality is already demo-grade — the gap is structural/experiential, not data flavour.

Confirmed gaps

1. No fictional organisation identity anywhere

Login/demo-entry screen says "MobilityOps · Control Centre" and generic marketing copy ("Every hand-off. One clear view.") but never names a demonstration organisation. A stranger cannot tell whose fleet this is. Confirmed via full-repo search: "Northstar Mobility" is never rendered in the UI, but it is already the project's locked fictitious tenant namePROJECT_STATE.md's "Locked decisions" section states verbatim "Fictitious tenant: Northstar Mobility Demo", and ragcore_tenant defaults to "northstar-mobility-demo" in backend/app/core/config.py, compose.yaml, .env.example, knowledge/manifest.json and contracts/mcp-tools.json. So using "Northstar Mobility" as the visible demo brand (as this brief suggests) is not a new invention — it surfaces a decision the project already made and only ever used internally as a RAGcore-tenant slug. No renaming/config changes needed, just make it visible.

2. No guided path — a visitor has no idea where to start

The dashboard shows real attention-queue items (duplicate customer, booking overlap, vehicle status conflict, missing field) and today's movements (BK-DEMO-RETURN 11:00), but nothing tells a first-time visitor which of these is worth clicking, in what order, or why. There is no "start guided demo" affordance, no scenario overview, no step-by-step walkthrough. Everything technically works; nothing narrates.

3. Live date anchoring is broken — confirmed, reproducible

seed/bookings.csv stores absolute fixed ISO timestamps authored around anchor date 2026-08-01 (generate_seed.py --anchor 2026-08-01). backend/app/core/config.py's demo_today setting ("2026-08-01") is used only by dashboard.py::_today() to filter "today's movements" against that frozen date — nothing re-anchors the underlying booking/inspection/maintenance/outbox dates themselves. Live-confirmed right now (today is 2026-08-03): BK-DEMO-RETURN shows status active with ends_at = 01/08/2026 09:00 — already two days in the past, for a booking that's supposed to look "in progress, due for return soon." BK-DEMO-NEXT (meant to read as "near-future") starts 2026-08-02, also already past. Every day this environment isn't reset, this gets worse, and after any seed --reset it snaps right back to the same frozen 2026-08-01-relative positions regardless of the real reset moment. This is exactly the failure mode brief section 7 describes and must be fixed at the source (seed_loader.py), not papered over in the UI.

4. Knowledge assistant names "RAGcore" directly, not honestly "demo mode"

/knowledge page copy: "Answers are shown only when RAGcore returns sufficient cited evidence" and a "RAGcore" label on the sources block — while the actual active provider is KNOWLEDGE_PROVIDER=demo (confirmed: badge below correctly shows demo · Available · 10 procedures indexed, which contradicts the body copy one line below it). This is the exact misleading-integration-status problem brief section 11 warns against — the small badge is honest, the prose isn't. No suggested questions are offered.

5. Integration status labels are technical, not demo-plain-language

Automation page shows raw states like not_configured/degraded/operational (English, developer register) rather than the plain-language labels section 12 asks for (Operationeel / Demomodus / Niet gekoppeld / Voorbereid / Verwerking mislukt / Opnieuw proberen mogelijk).

6. Data-quality resolution panels are functionally complete but not narrated

Confirmed live on DQ-DEMO-DUPLICATE: the compare/merge UI works well, but nothing explains why this matters operationally (duplicate billing/contact risk) or what happens next in plain terms before the user acts. Same pattern across the other 4 rule-type panels (already reviewed in this session's earlier batch). No "demo scenario" filter exists to surface the curated issues first.

7. No "About this demo" page

No page anywhere explains what's real, what's synthetic, what's live vs. demo-mode vs. not-connected, or the architecture/testing approach in plain terms. A visitor exploring alone has no way to self-answer "is this real?"

8. No permanent, explained "synthetic demo" indicator

The existing demo-banner ("Synthetic demo data · no real customer or vehicle information") is a full-width static bar repeated on every page — present, but static text only, no link to more explanation, no reset-timestamp/anchor info, and (per section 4) shouldn't be a dominant warning bar; it should be a subtle, explained badge.

9. Demo reset exists but is not discoverable as a demo feature

POST /api/v1/demo/reset and its sidebar UI trigger (this session's earlier batch) work correctly and are Operations-Manager-gated, but reset isn't connected to any demo narrative (no "prepare demo again" framing, not reachable from an About page or guide, no post-reset scenario-integrity confirmation beyond the raw counts).

10. No demo manifest / no single source of "what's on right now"

Nothing exposes demo mode, scenario list, integration status and guidance availability as one small structured payload the frontend can render consistently (badge tooltip, guide, scenario overview, About page all currently would have to duplicate this knowledge if built independently).

11. Everything is in English; the brief asks for consistent Dutch demo text

All existing UI copy is English. The brief explicitly asks for consistent Dutch product text for the demo-facing additions. Recorded decision (no existing doc answers this, and translating the entire already-shipped, already-tested Control Rail UI is out of proportion to "add guidance," would touch dozens of files and contradicts "this is not a redesign"): new demo-productization surfaces are written in Dutch (demo entry copy, Demo Guide, scenario overview, About-this-demo page, the demo badge/tooltip, plain-language integration-status labels). Existing operational screens (Dashboard, Vehicles, Bookings, Data Quality workbench, Audit, Automation internals) stay in English — they are already shipped, tested, and section 16 explicitly forbids rebuilding working screens without functional reason. This mirrors how a real bilingual guided demo works: narration in one language, software in whatever it already ships in.

Scope reminder (not gaps — explicitly out of scope per the brief)

  • No RAGcore or MCP Hub live implementation.
  • No new business logic in the 5 resolution flows — only explanatory framing around the existing ones.
  • No full redesign, new component library, new color palette, or full retranslation.

Plan (implementation batches, matches task list)

  1. Fix date anchoring in seed_loader.py (shift all seeded dates by today anchor at every reset); switch dashboard.py::_today() to real wall-clock date; add a demo_data_seeded audit marker recording the anchor/reset moment.
  2. GET /api/v1/demo/manifest (org name, demo mode, scenarios, integration status, reset timestamp) + Dutch demo entry screen + permanent demo badge with popover.
  3. Demo Guide (collapsible panel/bottom sheet, 8 steps) + scenario overview (5 cards) on the dashboard.
  4. Layer plain-language Dutch explanation onto the return flow, the 5 DQ panels and the knowledge assistant (fix the RAGcore-naming bug), without touching their logic.
  5. Plain-language integration status labels, richer audit narration for guide-linked events, "Over deze demo" page, reset wired into guide/About/OM menu with a post-reset scenario-integrity check.
  6. Full guided-demo Playwright test + targeted tests per section 19.
  7. Clean-checkout demo drill, docs, Unraid deploy, final evidence with screenshots.