Files
MobilityOps/artifacts/fleet-ops-correction/final-summary.md
T
NuklearRabbitandClaude Sonnet 5 f7805579f7 docs(release): final Fleet Ops correction evidence and screenshots
Full acceptance evidence for the Fleet Ops correction milestone: commits, branding,
translation coverage, status-preview/apply/manual-review/MO-016-ordering results,
knowledge grounding per language, audit/automation localization, backend/frontend
test results, clean-checkout drill, Unraid deployment (both fix-branch and
post-merge master), responsive/accessibility results, known limitations, and
rollback procedure. Includes live screenshots (nl-BE and fr-BE login, and the
localized data-quality evidence summary that live validation caught and fixed).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-04 01:15:31 +02:00

18 KiB
Raw Blame History

Fleet Ops correction and release — final evidence

Result: PASS

Commits

  • Source branch / commit (verified pre-correction baseline): master @ 18344bc8b7a75a2f868bf15bf498fc030ac6c34c
  • Fix branch: fix/fleet-ops-i18n-status-flow
  • Final fix-branch commit: 284b3c7 (merged content identical to 2e4fb43, which carries the evidence-summary localization fix)
  • Main-before-merge: 18344bc8b7a75a2f868bf15bf498fc030ac6c34c (confirmed unchanged via git fetch + git rev-parse origin/master immediately before merging — no unexpected commits landed on master while this branch was in progress)
  • Merge commit: de0bdea84fea01b4501deb7099107bc753c2e6d7 (git merge --no-ff fix/fleet-ops-i18n-status-flow -m "merge: complete Fleet Ops localization and status resolution", zero conflicts)
  • Final main commit: de0bdea84fea01b4501deb7099107bc753c2e6d7
  • Deployed commit: de0bdea84fea01b4501deb7099107bc753c2e6d7 (.deploy/source-revision on Unraid)
  • Gitea main branch: master (confirmed via git fetch origin && git rev-parse origin/master matching local master after push)
  • Live URL: http://192.168.10.150:1236

Fix-branch commit history: 6deb955, e6539d1, ac4b163, 1fdd2b3, 1e40775, a7ac5ed, 7851e80, cda2c32, 2e4fb43, 284b3c7.

What this correction fixed

  1. Status-recommendation flow redesigned (sections 8A8F). The old single opaque "calculate and apply recommended status" action is replaced by a single shared, pure evaluator (backend/app/services/vehicle_status.py::evaluate_vehicle_status, documented in docs/fleet-ops-correction/vehicle-status-decision-table.md) used identically by the scanner, a non-mutating preview endpoint (POST /api/v1/data-quality/issues/{ref}/status-recommendation), and a transactional apply endpoint (POST .../apply-recommended-status) that locks the row, recomputes facts, rejects a stale recommendation_token, refuses unsafe/manual- review recommendations, and re-validates post-write before resolving the issue.
    • Forbidden shortcuts eliminated: "maintenance + active booking" no longer auto-recommends "rented" (being in maintenance is itself now a blocking fact); "maintenance with nothing else wrong" no longer auto-clears to "available" (no fact proves maintenance is actually finished — release stays a manual decision).
    • Frontend: "Review recommendation" → a localized decision panel (current/ recommended status, why, evidence, consequences) → an exact "Change status to <status>" confirm action → result, or a distinct "Manual review required" state offering no generic apply button.
  2. MO-016 order independence (section 9). Order independence does not mean "same final status regardless of order" — resolving the booking overlap first genuinely removes the conflict, correctly leaving nothing to apply. What holds either way: the recommendation always reflects real current facts (never a stale proxy), and nothing unsafe is ever applied (never "rented"). Proven by a backend test explicitly scoped to MO-016/DQ-DEMO-STATUS (the original version wasn't — _first_open() returned whichever of ~14 open vehicle_status_conflict issues was most recent, not necessarily MO-016's) and a browser-level Playwright test covering both orders.
  3. "Fleet Ops" is a non-localizable brand constant (frontend/src/product.ts, backend PRODUCT_NAME), wired via {{productName}} interpolation everywhere the brand appeared in locale prose. A permanent test fails the build if any locale file ever defines the brand name or an appName key again.
  4. Dynamic backend prose converted to message codes + params (sections 5/6/10): return status reasons, audit field/actor-type labels, automation last_error (new last_error_code column, migration 799d8800e241), search results (sections/ vehicles/bookings/issues), and — found live on Unraid — the data-quality evidence summary. Raw technical text is demoted to a "Technical details" disclosure everywhere.
  5. Knowledge-base fixes: the demo provider's tokenizer silently dropped accented characters ([a-z0-9]+ split "véhicule" into "v"+"hicule"), breaking French retrieval broadly — fixed to include the Latin-1 accented range. Reweighted section scoring so a body match (real substance) outranks a heading/title match (a shallow structural hint) — the old weighting misranked the damage procedure behind an unrelated document for the brief's exact validation question in all 3 languages. Removed leftover "MobilityOps"/"PoC" mentions from 9 procedure documents.
  6. Search, audit, automation, maintenance/inspections localized (section 10): backend returns stable codes + params only; the frontend localizes section labels, vehicle summaries, booking/issue statuses, audit action/field/actor labels, automation error explanations, and maintenance/inspection type labels.
  7. i18n test suite strengthened (section 11): key parity, brand invariant, translation-quality (cross-locale identical-value detection), a hardcoded-JSX-text static scan (had to anchor on backreferenced closing-tag names — a naive >text< regex misread TypeScript generics as JSX), and a 3-language route matrix (every main route, no console errors, correct html[lang], real page headings).

Live-caught bug (the deployment validation earning its keep)

Live validation on the freshly-deployed fix branch directly caught a real defect: every data-quality issue's top-of-page evidence summary was unconditionally showing raw, always-English text (e.g. "vehicle marked available while reserved bookings conflict") in all three languages, because the frontend never finished the evidence.signals localization the backend had already been emitting (the backend code even had a comment describing the intended design that the frontend didn't implement). Fixed in commit 2e4fb43:

  • DataQualityIssueDetail.tsx now renders evidence.signals through the operator's locale as the primary evidence text.
  • The four DQ-DEMO-* seed rows that anchor the guided demo's scripted scenarios now carry real, accurate signals computed at seed time (the duplicate-customer similarity score is the actual SequenceMatcher ratio on the seeded names, not invented).
  • Rows with no structured signals fall back to raw text rather than showing a blank summary; the one known filler placeholder gets its own localized rendering.
  • A regression test locks this in: the vehicle-status-conflict evidence summary must show localized text and must never contain the specific raw English sentence that was live-visible before the fix, in all 3 languages.

Also found and fixed along the way: a frontend logic bug conflating "no conflict" with "manual review required" (both carry safe_to_apply: false), which showed a false "manual review required" panel for MO-016 after its booking overlap was resolved instead of the correct "no change needed" state (fixed in 1fdd2b3).

Translation coverage

  • All three locale files (nl-BE, en-GB, fr-BE) define exactly the same key set for every namespace (i18n-coverage.spec.ts, structural guarantee).
  • No locale file contains an empty string value.
  • No locale file defines the brand name or an appName key (brand-invariant test).
  • Cross-locale translation-quality check: for every string ≥8 characters of real prose, nl-BE ≠ en-GB, fr-BE ≠ en-GB, fr-BE ≠ nl-BE, with a precise, audited allowlist for genuine proper nouns/cognates (23 entries, each with a documented reason).
  • Hardcoded-JSX-text static scan: zero findings against the current codebase (verified against both false positives — TypeScript generics — and a deliberately-injected- then-reverted false negative).
  • 3-language route matrix: every main route (dashboard, vehicles, vehicle detail, bookings, booking detail, data quality, issue detail, automation, knowledge, audit, scenarios, about) opens cleanly in all 3 languages with no console errors, correct html[lang], and a real page heading.
  • Remaining visible wrong-language text: none found. The one gap that existed (the data-quality evidence summary) was found live and fixed before merge.

Branding

  • Visible product name: Fleet Ops, exactly, in all 3 languages, everywhere (login, topbar, footer "Fleet Ops Demo", document title, About page, Demo Guide, knowledge base). Verified structurally (brand-invariant test) and live (branding test across dashboard/vehicles/data-quality/audit/automation/knowledge pages in all 3 languages; visual screenshots of the login screen in nl-BE and fr-BE).
  • Technical identifier retained (by design, per the brief): repository name, local directory, package/module names, Compose project, deployment directory, database name, and the /health endpoint's service: "mobilityops-api" field remain "mobilityops" — none of these are visible UI text.
  • No visible "MobilityOps" or "PoC" anywhere in the UI or the demo knowledge base (9 procedure documents cleaned up; regression test in test_knowledge.py scans every procedure file for both strings).

Status-preview / apply / manual-review / MO-016 ordering

  • Preview: verified non-mutating — the issue's status stays "open" after calling the preview endpoint and re-fetching it via a fresh request.
  • Apply: the confirm button names the exact target status ("Change status to Blocked" / "Status wijzigen naar Geblokkeerd" / "Changer le statut vers Bloqué"); applying resolves the issue and updates the vehicle atomically.
  • Manual review: MO-024 (active rental + service-threshold reached, a genuine fact contradiction) shows "Manual review required" with no generic apply button rendered at all.
  • Stale token: simulated by resolving the underlying booking overlap after the preview was fetched but before applying — the apply call is correctly rejected (RECOMMENDATION_STALE), the UI shows the "situation has changed" message, and the user must review again before a new apply is possible.
  • MO-016 ordering: both orders tested. Resolving the overlap first correctly leaves nothing to apply (vehicle stays "available", genuinely correct). Resolving the status conflict first safely blocks the vehicle; resolving the now-redundant overlap afterwards does not disturb it. Neither order ever produces "rented".

Knowledge (per language)

The brief's exact validation question, in each language, grounds on the damage procedure as the primary (not just top-3) source:

  • nl-BE: "Wat moet ik doen wanneer een voertuig beschadigd terugkomt?" → damage procedure, Dutch source, Dutch excerpt.
  • en-GB: "What should I do when a vehicle returns with damage?" → damage procedure, English source, English excerpt.
  • fr-BE: "Que dois-je faire lorsqu'un véhicule revient endommagé ?" → damage procedure, French source, French excerpt.

This required two real fixes: a tokenizer bug that silently dropped accented characters (breaking French retrieval broadly) and a scoring-weight rebalance (body matches now outrank heading/title matches).

Audit / automation

  • Audit: action labels localized (workflow_retry → "automatisering opnieuw geprobeerd" / "automation retried" / "automatisation relancée", etc.), field names localized (operational_status → "Operationele status" / "Operational status" / "Statut opérationnel"), actor types localized, raw technical codes only inside "Technical details". Verified live and via a dedicated Playwright test.
  • Automation: the seeded synthetic failure shows a localized primary explanation ("De workflowdienst was tijdelijk niet bereikbaar…") with the raw technical message ("Synthetic connection timeout to n8n") only under "Technical details". Verified live and via a dedicated Playwright test.

Backend tests / lint / types

  • pytest: 151 passed, 0 failed (clean checkout, local dev, and post-merge master — run four times across this correction, always 151/151).
  • ruff check .: all checks passed, every run.
  • mypy app (strict): no issues found in 49 source files, every run.
  • Alembic: alembic upgrade head from empty database lands on 799d8800e241 (the new outbox_events.last_error_code column); downgrade -1 / upgrade head round-trip verified.

Frontend build / Playwright

  • npm ci, tsc -b, vite build: clean, every run.
  • Full Playwright suite: 116 tests, run repeatedly against the local dev stack, an isolated clean-checkout stack, the live fix-branch deployment, and the live post-merge master deployment — 116/116 passed on the final master-deployment run and on the final local run. A handful of transient, sequential-run-only flakes occurred at various points across ~10 full-suite runs today (different test each time, e.g. a pre-existing logout-timing race in AuthContext.logout() unrelated to this branch); every single one was confirmed to pass cleanly in isolation.
  • Guided demo covered indirectly via guided-demo-full.spec.ts, demo-guide.spec.ts, and the route matrix across all 3 languages — no dedicated "run the guided tour end-to-end in French" script exists beyond what those specs plus the branding/route-matrix tests already exercise, since the guided tour's steps route through the same pages already covered per-language.

Clean-checkout drill

Fresh git clone --branch fix/fleet-ops-i18n-status-flow of only committed files into an isolated Compose project (cleancheckfleetops, ports 8129/1229/5679 to avoid colliding with the working dev stack). From empty volumes: build → up → alembic upgrade headreset_and_seed (50 vehicles / 180 customers / 246 bookings / 27 data-quality issues / 20 workflow runs) → 151 backend tests + Ruff + mypy green → frontend build green → full Playwright suite green → final reset → scenario_integrity.all_ready: true. Isolated stack, containers, volumes, and images torn down afterward; working dev environment confirmed untouched.

Unraid deployment

Deployed via git archivescp → extract into /mnt/user/appdata/mobilityops (preserving .env and persistent volumes) → .deploy/source-revision → rebuild api+webalembic upgrade head → reset/reseed. Done twice: once for the fix branch (caught the evidence-summary bug), once for the final merged master. Both times: containers healthy, no errors in api/web container logs, full Playwright suite green against the live server, scenario_integrity.all_ready: true after final reset. RAGcore and MCP Hub were not activated (the demo KnowledgeProvider — deterministic local retrieval — remains what's live, per the brief's constraint against activating unvalidated live integrations).

Responsive / accessibility

  • Breakpoint matrix (1440×1000, 1280×800, 1024×768, 768×1024, 430×932, 390×844, 360×800) × 3 languages: no horizontal overflow, localized headings visible (responsive-i18n.spec.ts).
  • Status-recommendation panel: keyboard-only activation of "Review recommendation" and "Change status to X" verified via focus assertions (not just click); reduced-motion emulated during the flow; status never conveyed by colour alone (the badge always carries its own localized text); aria-live="polite" added so the applied confirmation is announced to screen readers.

Known limitations

  • A pre-existing, narrow timing race in AuthContext.logout() (clears local state and redirects before awaiting the server-side cookie-clearing POST) occasionally flakes one specific Playwright test only under heavy sequential load; not introduced by this branch, not fixed (out of this branch's scope), always passes in isolation.
  • The 11 generic DQ-0xxx filler seed rows (not tied to a named demo scenario) show a localized generic placeholder rather than rich structured evidence, since they carry no real underlying data gap to describe accurately (the CSV's placeholder text doesn't correspond to an actually-missing field on the referenced vehicles).
  • No dedicated "full guided demo in French, screenshot every step" script exists as a single artifact; coverage is composed from the route matrix, branding, and existing guided-demo specs, each run across all 3 languages.

Screenshots

artifacts/fleet-ops-correction/screenshots/, all captured live against http://192.168.10.150:1236:

  • login-nl-BE.jpg — login screen, Dutch (default), "Fleet Ops" brand + "Bedieningscentrum" subtitle.
  • login-fr-BE.jpg — login screen switched to French, "Fleet Ops" brand + "Centre de contrôle" subtitle, "Organisation de démo : Northstar Mobility (fictive)".
  • dq-demo-status-fr-BE-collapsed.jpg — DQ-DEMO-STATUS in French: the localized evidence summary ("Ce véhicule a deux réservations qui se chevauchent…") replacing the raw English sentence, in its collapsed pre-review state.
  • dq-demo-status-fr-BE-clean-reload.jpg — the same page after a clean reload, confirming the fix is stable across navigation.

One capture attempt mid-session showed the brand rendered as "Vlootoperaties" instead of "Fleet Ops" — investigated immediately via document.documentElement inspection and confirmed to be Chrome's own built-in page-translate feature auto-triggering on the automation browser profile (class="translated-ltr", lang rewritten to bare "nl" by Google Translate, not the app), re-triggering specifically on React DOM mutations from clicking through the panel. Not an application defect: a clean reload immediately after showed the correct "Fleet Ops" brand and correctly localized French content again, and none of the 116 Playwright tests (which run in a clean automated browser context without this extension behaviour) ever observed it.

Rollback procedure

  1. ssh unraid, cd /mnt/user/appdata/mobilityops.
  2. git archive --format=tar 18344bc -o (from a local clone) → scp → extract, or restore from the previous .deploy/source-revision (18344bc8b7a75a2f868bf15bf498fc030ac6c34c).
  3. echo 18344bc8b7a75a2f868bf15bf498fc030ac6c34c > .deploy/source-revision.
  4. docker compose -f compose.yaml -f compose.unraid.yaml build api web && ... up -d api web.
  5. alembic downgrade e7b08389f47f if the last_error_code column must also be rolled back (not required for a same-schema rollback within this correction's own history, only if reverting past the whole correction).
  6. Re-seed and re-verify scenario_integrity.all_ready: true.

The fix branch fix/fleet-ops-i18n-status-flow was not deleted.