docs(demo): audit current demo readiness gaps
Confirms the underlying data/business-logic is already demo-grade (Dutch/ Flemish names, .test emails, believable Belgian towns and RV brands; the 5 requested scenarios already exist as S1/S2/S4/S5/S6 in docs/13-seed-and-demo-scenarios.md) -- the real gaps are structural: no guided path, no visible fictional org identity (Northstar Mobility is already the locked tenant name internally, just never shown), a reproducible date-anchoring bug (seed dates are absolute and don't move with reset -- BK-DEMO-RETURN's end date is already in the past as of today), the knowledge page naming "RAGcore" directly instead of "demo mode", technical-register integration-status labels, and no About page.
This commit is contained in:
NuklearRabbit
@@ -0,0 +1,157 @@
|
||||
# Current demo-readiness gap audit
|
||||
|
||||
Performed 2026-08-03 against `feat/mobilityops-functional-completion` @
|
||||
`e0c7ed60112510687627d20a957af91c8b9db7f8` (verified: correct branch, working tree
|
||||
clean, matches the deployed Unraid revision). Method: read `CLAUDE.md`/`AGENTS.md`
|
||||
(identical, no demo-specific guidance yet), `README.md`, `PROJECT_STATE.md`, the docs
|
||||
list in the brief, `artifacts/final-acceptance/summary.md`,
|
||||
`artifacts/functional-completion/final-summary.md`, `seed/generate_seed.py`,
|
||||
`seed/*.csv`, `backend/app/seed_loader.py`, `backend/app/core/config.py`,
|
||||
`backend/app/api/routers/dashboard.py`; live-browsed the deployed app at
|
||||
`http://192.168.10.150:1236` (login, dashboard, a data-quality issue, a booking detail,
|
||||
knowledge, automation).
|
||||
|
||||
## What already works well (do not rebuild)
|
||||
|
||||
- The premium Control Rail UI, role-based nav, server-backed sessions, return preview/
|
||||
commit, all five data-quality resolution flows, audit before/after, role-aware search,
|
||||
truthful integration status, demo reset, stale-lease recovery and the two n8n
|
||||
workflows are all implemented, tested and live-verified per
|
||||
`artifacts/functional-completion/final-summary.md` — this pass does not touch that
|
||||
business logic.
|
||||
- `docs/13-seed-and-demo-scenarios.md` already names exactly the scenarios this brief
|
||||
wants surfaced: **S1** odometer-regression return (`BK-DEMO-RETURN`/`MO-024`), **S2**
|
||||
duplicate customer (`CUS-0012`/`CUS-0178`, issue `DQ-DEMO-DUPLICATE`), **S4** booking
|
||||
overlap (`MO-016`, issue `DQ-DEMO-OVERLAP`, bookings `BK-DEMO-OVERLAP-A/B`), **S5**
|
||||
failed workflow (seeded failed outbox event, retryable from Automation), **S6**
|
||||
grounded knowledge question. These map directly to the brief's 5 requested scenarios
|
||||
— no new scenarios need to be invented, only surfaced with guidance. (S3, "missing
|
||||
inspection before next booking" on `MO-031`, exists too but isn't one of the 5
|
||||
requested; leave it as-is, it still shows up in the attention queue.)
|
||||
- `seed/generate_seed.py` already produces believable Flemish/Dutch data: first/last
|
||||
name pools (Sofie, Lotte, Bram, Peeters, Janssens, Vermeulen, ...), real Kempen-region
|
||||
towns (Geel, Turnhout, Herentals, Mol, Westerlo, Tessenderlo-Ham), `.test` email
|
||||
addresses, and believable RV/camper makes (Adria, Dethleffs, Hymer, Bürstner, ...).
|
||||
Confirmed live: `DQ-DEMO-DUPLICATE` shows customer "Bram Peeters" (Geel) with email
|
||||
`bram.peeters.12@example.test`. **The underlying data quality is already
|
||||
demo-grade — the gap is structural/experiential, not data flavour.**
|
||||
|
||||
## Confirmed gaps
|
||||
|
||||
### 1. No fictional organisation identity anywhere
|
||||
Login/demo-entry screen says "MobilityOps · Control Centre" and generic marketing copy
|
||||
("Every hand-off. One clear view.") but never names a demonstration organisation. A
|
||||
stranger cannot tell *whose* fleet this is. **Confirmed via full-repo search:**
|
||||
"Northstar Mobility" is never rendered in the UI, but it is **already the project's
|
||||
locked fictitious tenant name** — `PROJECT_STATE.md`'s "Locked decisions" section
|
||||
states verbatim *"Fictitious tenant: Northstar Mobility Demo"*, and
|
||||
`ragcore_tenant` defaults to `"northstar-mobility-demo"` in `backend/app/core/config.py`,
|
||||
`compose.yaml`, `.env.example`, `knowledge/manifest.json` and `contracts/mcp-tools.json`.
|
||||
So using "Northstar Mobility" as the visible demo brand (as this brief suggests) is not
|
||||
a new invention — it surfaces a decision the project already made and only ever used
|
||||
internally as a RAGcore-tenant slug. No renaming/config changes needed, just make it
|
||||
visible.
|
||||
|
||||
### 2. No guided path — a visitor has no idea where to start
|
||||
The dashboard shows real attention-queue items (duplicate customer, booking overlap,
|
||||
vehicle status conflict, missing field) and today's movements (`BK-DEMO-RETURN`
|
||||
11:00), but nothing tells a first-time visitor *which* of these is worth clicking, in
|
||||
what order, or why. There is no "start guided demo" affordance, no scenario overview,
|
||||
no step-by-step walkthrough. Everything technically works; nothing narrates.
|
||||
|
||||
### 3. Live date anchoring is broken — confirmed, reproducible
|
||||
`seed/bookings.csv` stores **absolute fixed ISO timestamps** authored around anchor
|
||||
date 2026-08-01 (`generate_seed.py --anchor 2026-08-01`). `backend/app/core/config.py`'s
|
||||
`demo_today` setting (`"2026-08-01"`) is used *only* by
|
||||
`dashboard.py::_today()` to filter "today's movements" against that frozen date — nothing
|
||||
re-anchors the underlying booking/inspection/maintenance/outbox dates themselves.
|
||||
**Live-confirmed right now** (today is 2026-08-03): `BK-DEMO-RETURN` shows status
|
||||
`active` with `ends_at = 01/08/2026 09:00` — already two days in the past, for a
|
||||
booking that's supposed to look "in progress, due for return soon." `BK-DEMO-NEXT`
|
||||
(meant to read as "near-future") starts 2026-08-02, also already past. Every day this
|
||||
environment isn't reset, this gets worse, and after any `seed --reset` it snaps right
|
||||
back to the same frozen 2026-08-01-relative positions regardless of the real reset
|
||||
moment. This is exactly the failure mode brief section 7 describes and must be fixed at
|
||||
the source (`seed_loader.py`), not papered over in the UI.
|
||||
|
||||
### 4. Knowledge assistant names "RAGcore" directly, not honestly "demo mode"
|
||||
`/knowledge` page copy: "Answers are shown only when **RAGcore** returns sufficient
|
||||
cited evidence" and a "RAGcore" label on the sources block — while the actual active
|
||||
provider is `KNOWLEDGE_PROVIDER=demo` (confirmed: badge below correctly shows
|
||||
`demo · Available · 10 procedures indexed`, which contradicts the body copy one line
|
||||
below it). This is the exact misleading-integration-status problem brief section 11
|
||||
warns against — the small badge is honest, the prose isn't. No suggested questions are
|
||||
offered.
|
||||
|
||||
### 5. Integration status labels are technical, not demo-plain-language
|
||||
Automation page shows raw states like `not_configured`/`degraded`/`operational` (English,
|
||||
developer register) rather than the plain-language labels section 12 asks for
|
||||
(Operationeel / Demomodus / Niet gekoppeld / Voorbereid / Verwerking mislukt / Opnieuw
|
||||
proberen mogelijk).
|
||||
|
||||
### 6. Data-quality resolution panels are functionally complete but not narrated
|
||||
Confirmed live on `DQ-DEMO-DUPLICATE`: the compare/merge UI works well, but nothing
|
||||
explains *why this matters operationally* (duplicate billing/contact risk) or what
|
||||
happens next in plain terms before the user acts. Same pattern across the other 4
|
||||
rule-type panels (already reviewed in this session's earlier batch). No "demo scenario"
|
||||
filter exists to surface the curated issues first.
|
||||
|
||||
### 7. No "About this demo" page
|
||||
No page anywhere explains what's real, what's synthetic, what's live vs. demo-mode vs.
|
||||
not-connected, or the architecture/testing approach in plain terms. A visitor exploring
|
||||
alone has no way to self-answer "is this real?"
|
||||
|
||||
### 8. No permanent, explained "synthetic demo" indicator
|
||||
The existing `demo-banner` ("Synthetic demo data · no real customer or vehicle
|
||||
information") is a full-width static bar repeated on every page — present, but static
|
||||
text only, no link to more explanation, no reset-timestamp/anchor info, and (per section
|
||||
4) shouldn't be a dominant warning bar; it should be a subtle, explained badge.
|
||||
|
||||
### 9. Demo reset exists but is not discoverable as a demo feature
|
||||
`POST /api/v1/demo/reset` and its sidebar UI trigger (this session's earlier batch)
|
||||
work correctly and are Operations-Manager-gated, but reset isn't connected to any demo
|
||||
narrative (no "prepare demo again" framing, not reachable from an About page or guide,
|
||||
no post-reset scenario-integrity confirmation beyond the raw counts).
|
||||
|
||||
### 10. No demo manifest / no single source of "what's on right now"
|
||||
Nothing exposes demo mode, scenario list, integration status and guidance availability
|
||||
as one small structured payload the frontend can render consistently (badge tooltip,
|
||||
guide, scenario overview, About page all currently would have to duplicate this
|
||||
knowledge if built independently).
|
||||
|
||||
### 11. Everything is in English; the brief asks for consistent Dutch demo text
|
||||
All existing UI copy is English. The brief explicitly asks for consistent Dutch product
|
||||
text for the demo-facing additions. Recorded decision (no existing doc answers this,
|
||||
and translating the entire already-shipped, already-tested Control Rail UI is out of
|
||||
proportion to "add guidance," would touch dozens of files and contradicts "this is not
|
||||
a redesign"): **new demo-productization surfaces are written in Dutch** (demo entry
|
||||
copy, Demo Guide, scenario overview, About-this-demo page, the demo badge/tooltip,
|
||||
plain-language integration-status labels). **Existing operational screens (Dashboard,
|
||||
Vehicles, Bookings, Data Quality workbench, Audit, Automation internals) stay in
|
||||
English** — they are already shipped, tested, and section 16 explicitly forbids
|
||||
rebuilding working screens without functional reason. This mirrors how a real bilingual
|
||||
guided demo works: narration in one language, software in whatever it already ships in.
|
||||
|
||||
## Scope reminder (not gaps — explicitly out of scope per the brief)
|
||||
|
||||
- No RAGcore or MCP Hub live implementation.
|
||||
- No new business logic in the 5 resolution flows — only explanatory framing around
|
||||
the existing ones.
|
||||
- No full redesign, new component library, new color palette, or full retranslation.
|
||||
|
||||
## Plan (implementation batches, matches task list)
|
||||
|
||||
1. Fix date anchoring in `seed_loader.py` (shift all seeded dates by `today − anchor`
|
||||
at every reset); switch `dashboard.py::_today()` to real wall-clock date; add a
|
||||
`demo_data_seeded` audit marker recording the anchor/reset moment.
|
||||
2. `GET /api/v1/demo/manifest` (org name, demo mode, scenarios, integration status,
|
||||
reset timestamp) + Dutch demo entry screen + permanent demo badge with popover.
|
||||
3. Demo Guide (collapsible panel/bottom sheet, 8 steps) + scenario overview (5 cards)
|
||||
on the dashboard.
|
||||
4. Layer plain-language Dutch explanation onto the return flow, the 5 DQ panels and the
|
||||
knowledge assistant (fix the RAGcore-naming bug), without touching their logic.
|
||||
5. Plain-language integration status labels, richer audit narration for guide-linked
|
||||
events, "Over deze demo" page, reset wired into guide/About/OM menu with a
|
||||
post-reset scenario-integrity check.
|
||||
6. Full guided-demo Playwright test + targeted tests per section 19.
|
||||
7. Clean-checkout demo drill, docs, Unraid deploy, final evidence with screenshots.
|
||||
Reference in New Issue
Block a user