docs(release): final Fleet Ops correction evidence and screenshots

Full acceptance evidence for the Fleet Ops correction milestone: commits, branding, translation coverage, status-preview/apply/manual-review/MO-016-ordering results, knowledge grounding per language, audit/automation localization, backend/frontend test results, clean-checkout drill, Unraid deployment (both fix-branch and post-merge master), responsive/accessibility results, known limitations, and rollback procedure. Includes live screenshots (nl-BE and fr-BE login, and the localized data-quality evidence summary that live validation caught and fixed). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-04 01:15:31 +02:00
parent de0bdea84f
commit f7805579f7
5 changed files with 284 additions and 0 deletions
@@ -0,0 +1,284 @@
 # Fleet Ops correction and release — final evidence
 **Result: PASS**
 ## Commits
 - Source branch / commit (verified pre-correction baseline): `master` @ `18344bc8b7a75a2f868bf15bf498fc030ac6c34c`
 - Fix branch: `fix/fleet-ops-i18n-status-flow`
 - Final fix-branch commit: `284b3c7` (merged content identical to `2e4fb43`, which carries the evidence-summary localization fix)
 - Main-before-merge: `18344bc8b7a75a2f868bf15bf498fc030ac6c34c` (confirmed unchanged via `git fetch` + `git rev-parse origin/master` immediately before merging — no unexpected commits landed on master while this branch was in progress)
 - Merge commit: `de0bdea84fea01b4501deb7099107bc753c2e6d7` (`git merge --no-ff fix/fleet-ops-i18n-status-flow -m "merge: complete Fleet Ops localization and status resolution"`, zero conflicts)
 - Final main commit: `de0bdea84fea01b4501deb7099107bc753c2e6d7`
 - Deployed commit: `de0bdea84fea01b4501deb7099107bc753c2e6d7` (`.deploy/source-revision` on Unraid)
 - Gitea main branch: `master` (confirmed via `git fetch origin && git rev-parse origin/master` matching local `master` after push)
 - Live URL: `http://192.168.10.150:1236`
 Fix-branch commit history: `6deb955`, `e6539d1`, `ac4b163`, `1fdd2b3`, `1e40775`, `a7ac5ed`, `7851e80`, `cda2c32`, `2e4fb43`, `284b3c7`.
 ## What this correction fixed
 1. **Status-recommendation flow redesigned** (sections 8A–8F). The old single opaque
   "calculate and apply recommended status" action is replaced by a single shared, pure
   evaluator (`backend/app/services/vehicle_status.py::evaluate_vehicle_status`,
   documented in `docs/fleet-ops-correction/vehicle-status-decision-table.md`) used
   identically by the scanner, a non-mutating preview endpoint
   (`POST /api/v1/data-quality/issues/{ref}/status-recommendation`), and a
   transactional apply endpoint (`POST .../apply-recommended-status`) that locks the
   row, recomputes facts, rejects a stale `recommendation_token`, refuses unsafe/manual-
   review recommendations, and re-validates post-write before resolving the issue.
   - Forbidden shortcuts eliminated: "maintenance + active booking" no longer
     auto-recommends "rented" (being in maintenance is itself now a blocking fact);
     "maintenance with nothing else wrong" no longer auto-clears to "available" (no
     fact proves maintenance is actually finished — release stays a manual decision).
   - Frontend: "Review recommendation" → a localized decision panel (current/
     recommended status, why, evidence, consequences) → an exact "Change status to
     &lt;status&gt;" confirm action → result, or a distinct "Manual review required"
     state offering no generic apply button.
 2. **MO-016 order independence** (section 9). Order independence does not mean "same
   final status regardless of order" — resolving the booking overlap first genuinely
   removes the conflict, correctly leaving nothing to apply. What holds either way: the
   recommendation always reflects real current facts (never a stale proxy), and nothing
   unsafe is ever applied (never "rented"). Proven by a backend test explicitly scoped
   to MO-016/DQ-DEMO-STATUS (the original version wasn't — `_first_open()` returned
   whichever of ~14 open `vehicle_status_conflict` issues was most recent, not
   necessarily MO-016's) and a browser-level Playwright test covering both orders.
 3. **"Fleet Ops" is a non-localizable brand constant** (`frontend/src/product.ts`,
   backend `PRODUCT_NAME`), wired via `{{productName}}` interpolation everywhere the
   brand appeared in locale prose. A permanent test fails the build if any locale file
   ever defines the brand name or an `appName` key again.
 4. **Dynamic backend prose converted to message codes + params** (sections 5/6/10):
   return status reasons, audit field/actor-type labels, automation `last_error` (new
   `last_error_code` column, migration `799d8800e241`), search results (sections/
   vehicles/bookings/issues), and — found live on Unraid — the data-quality evidence
   summary. Raw technical text is demoted to a "Technical details" disclosure
   everywhere.
 5. **Knowledge-base fixes**: the demo provider's tokenizer silently dropped accented
   characters (`[a-z0-9]+` split "véhicule" into "v"+"hicule"), breaking French
   retrieval broadly — fixed to include the Latin-1 accented range. Reweighted section
   scoring so a body match (real substance) outranks a heading/title match (a shallow
   structural hint) — the old weighting misranked the damage procedure behind an
   unrelated document for the brief's exact validation question in all 3 languages.
   Removed leftover "MobilityOps"/"PoC" mentions from 9 procedure documents.
 6. **Search, audit, automation, maintenance/inspections localized** (section 10):
   backend returns stable codes + params only; the frontend localizes section labels,
   vehicle summaries, booking/issue statuses, audit action/field/actor labels,
   automation error explanations, and maintenance/inspection type labels.
 7. **i18n test suite strengthened** (section 11): key parity, brand invariant,
   translation-quality (cross-locale identical-value detection), a hardcoded-JSX-text
   static scan (had to anchor on backreferenced closing-tag names — a naive `>text<`
   regex misread TypeScript generics as JSX), and a 3-language route matrix (every main
   route, no console errors, correct `html[lang]`, real page headings).
 ## Live-caught bug (the deployment validation earning its keep)
 Live validation on the freshly-deployed fix branch directly caught a real defect: every
 data-quality issue's top-of-page evidence summary was unconditionally showing raw,
 always-English text (e.g. *"vehicle marked available while reserved bookings
 conflict"*) in **all three languages**, because the frontend never finished the
 `evidence.signals` localization the backend had already been emitting (the backend code
 even had a comment describing the intended design that the frontend didn't implement).
 Fixed in commit `2e4fb43`:
 - `DataQualityIssueDetail.tsx` now renders `evidence.signals` through the operator's
  locale as the primary evidence text.
 - The four `DQ-DEMO-*` seed rows that anchor the guided demo's scripted scenarios now
  carry real, accurate signals computed at seed time (the duplicate-customer similarity
  score is the actual `SequenceMatcher` ratio on the seeded names, not invented).
 - Rows with no structured signals fall back to raw text rather than showing a blank
  summary; the one known filler placeholder gets its own localized rendering.
 - A regression test locks this in: the vehicle-status-conflict evidence summary must
  show localized text and must never contain the specific raw English sentence that was
  live-visible before the fix, in all 3 languages.
 Also found and fixed along the way: a frontend logic bug conflating "no conflict" with
 "manual review required" (both carry `safe_to_apply: false`), which showed a false
 "manual review required" panel for MO-016 after its booking overlap was resolved
 instead of the correct "no change needed" state (fixed in `1fdd2b3`).
 ## Translation coverage
 - All three locale files (`nl-BE`, `en-GB`, `fr-BE`) define exactly the same key set
  for every namespace (`i18n-coverage.spec.ts`, structural guarantee).
 - No locale file contains an empty string value.
 - No locale file defines the brand name or an `appName` key (brand-invariant test).
 - Cross-locale translation-quality check: for every string ≥8 characters of real prose,
  nl-BE ≠ en-GB, fr-BE ≠ en-GB, fr-BE ≠ nl-BE, with a precise, audited allowlist for
  genuine proper nouns/cognates (23 entries, each with a documented reason).
 - Hardcoded-JSX-text static scan: zero findings against the current codebase (verified
  against both false positives — TypeScript generics — and a deliberately-injected-
  then-reverted false negative).
 - 3-language route matrix: every main route (dashboard, vehicles, vehicle detail,
  bookings, booking detail, data quality, issue detail, automation, knowledge, audit,
  scenarios, about) opens cleanly in all 3 languages with no console errors, correct
  `html[lang]`, and a real page heading.
 - **Remaining visible wrong-language text**: none found. The one gap that existed (the
  data-quality evidence summary) was found live and fixed before merge.
 ## Branding
 - Visible product name: **Fleet Ops**, exactly, in all 3 languages, everywhere (login,
  topbar, footer "Fleet Ops Demo", document title, About page, Demo Guide, knowledge
  base). Verified structurally (brand-invariant test) and live (branding test across
  dashboard/vehicles/data-quality/audit/automation/knowledge pages in all 3 languages;
  visual screenshots of the login screen in nl-BE and fr-BE).
 - Technical identifier retained (by design, per the brief): repository name, local
  directory, package/module names, Compose project, deployment directory, database
  name, and the `/health` endpoint's `service: "mobilityops-api"` field remain
  "mobilityops" — none of these are visible UI text.
 - No visible "MobilityOps" or "PoC" anywhere in the UI or the demo knowledge base
  (9 procedure documents cleaned up; regression test in `test_knowledge.py` scans every
  procedure file for both strings).
 ## Status-preview / apply / manual-review / MO-016 ordering
 - **Preview**: verified non-mutating — the issue's `status` stays `"open"` after
  calling the preview endpoint and re-fetching it via a fresh request.
 - **Apply**: the confirm button names the exact target status ("Change status to
  Blocked" / "Status wijzigen naar Geblokkeerd" / "Changer le statut vers Bloqué");
  applying resolves the issue and updates the vehicle atomically.
 - **Manual review**: MO-024 (active rental + service-threshold reached, a genuine fact
  contradiction) shows "Manual review required" with no generic apply button rendered
  at all.
 - **Stale token**: simulated by resolving the underlying booking overlap after the
  preview was fetched but before applying — the apply call is correctly rejected
  (`RECOMMENDATION_STALE`), the UI shows the "situation has changed" message, and the
  user must review again before a new apply is possible.
 - **MO-016 ordering**: both orders tested. Resolving the overlap first correctly leaves
  nothing to apply (vehicle stays "available", genuinely correct). Resolving the status
  conflict first safely blocks the vehicle; resolving the now-redundant overlap
  afterwards does not disturb it. Neither order ever produces "rented".
 ## Knowledge (per language)
 The brief's exact validation question, in each language, grounds on the damage
 procedure as the **primary** (not just top-3) source:
 - nl-BE: *"Wat moet ik doen wanneer een voertuig beschadigd terugkomt?"* → damage
  procedure, Dutch source, Dutch excerpt.
 - en-GB: *"What should I do when a vehicle returns with damage?"* → damage procedure,
  English source, English excerpt.
 - fr-BE: *"Que dois-je faire lorsqu'un véhicule revient endommagé ?"* → damage
  procedure, French source, French excerpt.
 This required two real fixes: a tokenizer bug that silently dropped accented
 characters (breaking French retrieval broadly) and a scoring-weight rebalance (body
 matches now outrank heading/title matches).
 ## Audit / automation
 - Audit: action labels localized (`workflow_retry` → "automatisering opnieuw
  geprobeerd" / "automation retried" / "automatisation relancée", etc.), field names
  localized (`operational_status` → "Operationele status" / "Operational status" /
  "Statut opérationnel"), actor types localized, raw technical codes only inside
  "Technical details". Verified live and via a dedicated Playwright test.
 - Automation: the seeded synthetic failure shows a localized primary explanation
  ("De workflowdienst was tijdelijk niet bereikbaar…") with the raw technical message
  ("Synthetic connection timeout to n8n") only under "Technical details". Verified live
  and via a dedicated Playwright test.
 ## Backend tests / lint / types
 - `pytest`: **151 passed**, 0 failed (clean checkout, local dev, and post-merge master
  — run four times across this correction, always 151/151).
 - `ruff check .`: all checks passed, every run.
 - `mypy app` (strict): no issues found in 49 source files, every run.
 - Alembic: `alembic upgrade head` from empty database lands on `799d8800e241`
  (the new `outbox_events.last_error_code` column); `downgrade -1` / `upgrade head`
  round-trip verified.
 ## Frontend build / Playwright
 - `npm ci`, `tsc -b`, `vite build`: clean, every run.
 - Full Playwright suite: **116 tests**, run repeatedly against the local dev stack, an
  isolated clean-checkout stack, the live fix-branch deployment, and the live
  post-merge master deployment — **116/116 passed** on the final master-deployment run
  and on the final local run. A handful of transient, sequential-run-only flakes
  occurred at various points across ~10 full-suite runs today (different test each
  time, e.g. a pre-existing logout-timing race in `AuthContext.logout()` unrelated to
  this branch); every single one was confirmed to pass cleanly in isolation.
 - Guided demo covered indirectly via `guided-demo-full.spec.ts`,
  `demo-guide.spec.ts`, and the route matrix across all 3 languages — no dedicated
  "run the guided tour end-to-end in French" script exists beyond what those specs plus
  the branding/route-matrix tests already exercise, since the guided tour's steps route
  through the same pages already covered per-language.
 ## Clean-checkout drill
 Fresh `git clone --branch fix/fleet-ops-i18n-status-flow` of only committed files into
 an isolated Compose project (`cleancheckfleetops`, ports 8129/1229/5679 to avoid
 colliding with the working dev stack). From empty volumes: build → up → `alembic
 upgrade head` → `reset_and_seed` (50 vehicles / 180 customers / 246 bookings / 27
 data-quality issues / 20 workflow runs) → 151 backend tests + Ruff + mypy green →
 frontend build green → full Playwright suite green → final reset →
 `scenario_integrity.all_ready: true`. Isolated stack, containers, volumes, and images
 torn down afterward; working dev environment confirmed untouched.
 ## Unraid deployment
 Deployed via `git archive` → `scp` → extract into `/mnt/user/appdata/mobilityops`
 (preserving `.env` and persistent volumes) → `.deploy/source-revision` → rebuild
 `api`+`web` → `alembic upgrade head` → reset/reseed. Done twice: once for the fix
 branch (caught the evidence-summary bug), once for the final merged master. Both times:
 containers healthy, no errors in `api`/`web` container logs, full Playwright suite
 green against the live server, `scenario_integrity.all_ready: true` after final reset.
 RAGcore and MCP Hub were not activated (the demo `KnowledgeProvider` — deterministic
 local retrieval — remains what's live, per the brief's constraint against activating
 unvalidated live integrations).
 ## Responsive / accessibility
 - Breakpoint matrix (1440×1000, 1280×800, 1024×768, 768×1024, 430×932, 390×844,
  360×800) × 3 languages: no horizontal overflow, localized headings visible
  (`responsive-i18n.spec.ts`).
 - Status-recommendation panel: keyboard-only activation of "Review recommendation" and
  "Change status to X" verified via focus assertions (not just click); reduced-motion
  emulated during the flow; status never conveyed by colour alone (the badge always
  carries its own localized text); `aria-live="polite"` added so the applied
  confirmation is announced to screen readers.
 ## Known limitations
 - A pre-existing, narrow timing race in `AuthContext.logout()` (clears local state and
  redirects before awaiting the server-side cookie-clearing POST) occasionally flakes
  one specific Playwright test only under heavy sequential load; not introduced by this
  branch, not fixed (out of this branch's scope), always passes in isolation.
 - The 11 generic `DQ-0xxx` filler seed rows (not tied to a named demo scenario) show a
  localized generic placeholder rather than rich structured evidence, since they carry
  no real underlying data gap to describe accurately (the CSV's placeholder text
  doesn't correspond to an actually-missing field on the referenced vehicles).
 - No dedicated "full guided demo in French, screenshot every step" script exists as a
  single artifact; coverage is composed from the route matrix, branding, and existing
  guided-demo specs, each run across all 3 languages.
 ## Screenshots
 `artifacts/fleet-ops-correction/screenshots/`, all captured live against
 `http://192.168.10.150:1236`:
 - `login-nl-BE.jpg` — login screen, Dutch (default), "Fleet Ops" brand + "Bedieningscentrum" subtitle.
 - `login-fr-BE.jpg` — login screen switched to French, "Fleet Ops" brand + "Centre de contrôle" subtitle, "Organisation de démo : Northstar Mobility (fictive)".
 - `dq-demo-status-fr-BE-collapsed.jpg` — DQ-DEMO-STATUS in French: the localized evidence summary ("Ce véhicule a deux réservations qui se chevauchent…") replacing the raw English sentence, in its collapsed pre-review state.
 - `dq-demo-status-fr-BE-clean-reload.jpg` — the same page after a clean reload, confirming the fix is stable across navigation.
 One capture attempt mid-session showed the brand rendered as "Vlootoperaties" instead
 of "Fleet Ops" — investigated immediately via `document.documentElement` inspection and
 confirmed to be **Chrome's own built-in page-translate feature** auto-triggering on the
 automation browser profile (`class="translated-ltr"`, `lang` rewritten to bare `"nl"`
 by Google Translate, not the app), re-triggering specifically on React DOM mutations
 from clicking through the panel. Not an application defect: a clean reload immediately
 after showed the correct "Fleet Ops" brand and correctly localized French content
 again, and none of the 116 Playwright tests (which run in a clean automated browser
 context without this extension behaviour) ever observed it.
 ## Rollback procedure
 1. `ssh unraid`, `cd /mnt/user/appdata/mobilityops`.
 2. `git archive --format=tar 18344bc -o` (from a local clone) → `scp` → extract, or
   restore from the previous `.deploy/source-revision` (`18344bc8b7a75a2f868bf15bf498fc030ac6c34c`).
 3. `echo 18344bc8b7a75a2f868bf15bf498fc030ac6c34c > .deploy/source-revision`.
 4. `docker compose -f compose.yaml -f compose.unraid.yaml build api web && ... up -d api web`.
 5. `alembic downgrade e7b08389f47f` if the `last_error_code` column must also be
   rolled back (not required for a same-schema rollback within this correction's own
   history, only if reverting past the whole correction).
 6. Re-seed and re-verify `scenario_integrity.all_ready: true`.
 The fix branch `fix/fleet-ops-i18n-status-flow` was not deleted.