Files
MobilityOps/artifacts/fleet-ops-correction/final-summary.md
T
NuklearRabbitandClaude Sonnet 5 f7805579f7 docs(release): final Fleet Ops correction evidence and screenshots
Full acceptance evidence for the Fleet Ops correction milestone: commits, branding,
translation coverage, status-preview/apply/manual-review/MO-016-ordering results,
knowledge grounding per language, audit/automation localization, backend/frontend
test results, clean-checkout drill, Unraid deployment (both fix-branch and
post-merge master), responsive/accessibility results, known limitations, and
rollback procedure. Includes live screenshots (nl-BE and fr-BE login, and the
localized data-quality evidence summary that live validation caught and fixed).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-04 01:15:31 +02:00

285 lines
18 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Fleet Ops correction and release — final evidence
**Result: PASS**
## Commits
- Source branch / commit (verified pre-correction baseline): `master` @ `18344bc8b7a75a2f868bf15bf498fc030ac6c34c`
- Fix branch: `fix/fleet-ops-i18n-status-flow`
- Final fix-branch commit: `284b3c7` (merged content identical to `2e4fb43`, which carries the evidence-summary localization fix)
- Main-before-merge: `18344bc8b7a75a2f868bf15bf498fc030ac6c34c` (confirmed unchanged via `git fetch` + `git rev-parse origin/master` immediately before merging — no unexpected commits landed on master while this branch was in progress)
- Merge commit: `de0bdea84fea01b4501deb7099107bc753c2e6d7` (`git merge --no-ff fix/fleet-ops-i18n-status-flow -m "merge: complete Fleet Ops localization and status resolution"`, zero conflicts)
- Final main commit: `de0bdea84fea01b4501deb7099107bc753c2e6d7`
- Deployed commit: `de0bdea84fea01b4501deb7099107bc753c2e6d7` (`.deploy/source-revision` on Unraid)
- Gitea main branch: `master` (confirmed via `git fetch origin && git rev-parse origin/master` matching local `master` after push)
- Live URL: `http://192.168.10.150:1236`
Fix-branch commit history: `6deb955`, `e6539d1`, `ac4b163`, `1fdd2b3`, `1e40775`, `a7ac5ed`, `7851e80`, `cda2c32`, `2e4fb43`, `284b3c7`.
## What this correction fixed
1. **Status-recommendation flow redesigned** (sections 8A8F). The old single opaque
"calculate and apply recommended status" action is replaced by a single shared, pure
evaluator (`backend/app/services/vehicle_status.py::evaluate_vehicle_status`,
documented in `docs/fleet-ops-correction/vehicle-status-decision-table.md`) used
identically by the scanner, a non-mutating preview endpoint
(`POST /api/v1/data-quality/issues/{ref}/status-recommendation`), and a
transactional apply endpoint (`POST .../apply-recommended-status`) that locks the
row, recomputes facts, rejects a stale `recommendation_token`, refuses unsafe/manual-
review recommendations, and re-validates post-write before resolving the issue.
- Forbidden shortcuts eliminated: "maintenance + active booking" no longer
auto-recommends "rented" (being in maintenance is itself now a blocking fact);
"maintenance with nothing else wrong" no longer auto-clears to "available" (no
fact proves maintenance is actually finished — release stays a manual decision).
- Frontend: "Review recommendation" → a localized decision panel (current/
recommended status, why, evidence, consequences) → an exact "Change status to
&lt;status&gt;" confirm action → result, or a distinct "Manual review required"
state offering no generic apply button.
2. **MO-016 order independence** (section 9). Order independence does not mean "same
final status regardless of order" — resolving the booking overlap first genuinely
removes the conflict, correctly leaving nothing to apply. What holds either way: the
recommendation always reflects real current facts (never a stale proxy), and nothing
unsafe is ever applied (never "rented"). Proven by a backend test explicitly scoped
to MO-016/DQ-DEMO-STATUS (the original version wasn't — `_first_open()` returned
whichever of ~14 open `vehicle_status_conflict` issues was most recent, not
necessarily MO-016's) and a browser-level Playwright test covering both orders.
3. **"Fleet Ops" is a non-localizable brand constant** (`frontend/src/product.ts`,
backend `PRODUCT_NAME`), wired via `{{productName}}` interpolation everywhere the
brand appeared in locale prose. A permanent test fails the build if any locale file
ever defines the brand name or an `appName` key again.
4. **Dynamic backend prose converted to message codes + params** (sections 5/6/10):
return status reasons, audit field/actor-type labels, automation `last_error` (new
`last_error_code` column, migration `799d8800e241`), search results (sections/
vehicles/bookings/issues), and — found live on Unraid — the data-quality evidence
summary. Raw technical text is demoted to a "Technical details" disclosure
everywhere.
5. **Knowledge-base fixes**: the demo provider's tokenizer silently dropped accented
characters (`[a-z0-9]+` split "véhicule" into "v"+"hicule"), breaking French
retrieval broadly — fixed to include the Latin-1 accented range. Reweighted section
scoring so a body match (real substance) outranks a heading/title match (a shallow
structural hint) — the old weighting misranked the damage procedure behind an
unrelated document for the brief's exact validation question in all 3 languages.
Removed leftover "MobilityOps"/"PoC" mentions from 9 procedure documents.
6. **Search, audit, automation, maintenance/inspections localized** (section 10):
backend returns stable codes + params only; the frontend localizes section labels,
vehicle summaries, booking/issue statuses, audit action/field/actor labels,
automation error explanations, and maintenance/inspection type labels.
7. **i18n test suite strengthened** (section 11): key parity, brand invariant,
translation-quality (cross-locale identical-value detection), a hardcoded-JSX-text
static scan (had to anchor on backreferenced closing-tag names — a naive `>text<`
regex misread TypeScript generics as JSX), and a 3-language route matrix (every main
route, no console errors, correct `html[lang]`, real page headings).
## Live-caught bug (the deployment validation earning its keep)
Live validation on the freshly-deployed fix branch directly caught a real defect: every
data-quality issue's top-of-page evidence summary was unconditionally showing raw,
always-English text (e.g. *"vehicle marked available while reserved bookings
conflict"*) in **all three languages**, because the frontend never finished the
`evidence.signals` localization the backend had already been emitting (the backend code
even had a comment describing the intended design that the frontend didn't implement).
Fixed in commit `2e4fb43`:
- `DataQualityIssueDetail.tsx` now renders `evidence.signals` through the operator's
locale as the primary evidence text.
- The four `DQ-DEMO-*` seed rows that anchor the guided demo's scripted scenarios now
carry real, accurate signals computed at seed time (the duplicate-customer similarity
score is the actual `SequenceMatcher` ratio on the seeded names, not invented).
- Rows with no structured signals fall back to raw text rather than showing a blank
summary; the one known filler placeholder gets its own localized rendering.
- A regression test locks this in: the vehicle-status-conflict evidence summary must
show localized text and must never contain the specific raw English sentence that was
live-visible before the fix, in all 3 languages.
Also found and fixed along the way: a frontend logic bug conflating "no conflict" with
"manual review required" (both carry `safe_to_apply: false`), which showed a false
"manual review required" panel for MO-016 after its booking overlap was resolved
instead of the correct "no change needed" state (fixed in `1fdd2b3`).
## Translation coverage
- All three locale files (`nl-BE`, `en-GB`, `fr-BE`) define exactly the same key set
for every namespace (`i18n-coverage.spec.ts`, structural guarantee).
- No locale file contains an empty string value.
- No locale file defines the brand name or an `appName` key (brand-invariant test).
- Cross-locale translation-quality check: for every string ≥8 characters of real prose,
nl-BE ≠ en-GB, fr-BE ≠ en-GB, fr-BE ≠ nl-BE, with a precise, audited allowlist for
genuine proper nouns/cognates (23 entries, each with a documented reason).
- Hardcoded-JSX-text static scan: zero findings against the current codebase (verified
against both false positives — TypeScript generics — and a deliberately-injected-
then-reverted false negative).
- 3-language route matrix: every main route (dashboard, vehicles, vehicle detail,
bookings, booking detail, data quality, issue detail, automation, knowledge, audit,
scenarios, about) opens cleanly in all 3 languages with no console errors, correct
`html[lang]`, and a real page heading.
- **Remaining visible wrong-language text**: none found. The one gap that existed (the
data-quality evidence summary) was found live and fixed before merge.
## Branding
- Visible product name: **Fleet Ops**, exactly, in all 3 languages, everywhere (login,
topbar, footer "Fleet Ops Demo", document title, About page, Demo Guide, knowledge
base). Verified structurally (brand-invariant test) and live (branding test across
dashboard/vehicles/data-quality/audit/automation/knowledge pages in all 3 languages;
visual screenshots of the login screen in nl-BE and fr-BE).
- Technical identifier retained (by design, per the brief): repository name, local
directory, package/module names, Compose project, deployment directory, database
name, and the `/health` endpoint's `service: "mobilityops-api"` field remain
"mobilityops" — none of these are visible UI text.
- No visible "MobilityOps" or "PoC" anywhere in the UI or the demo knowledge base
(9 procedure documents cleaned up; regression test in `test_knowledge.py` scans every
procedure file for both strings).
## Status-preview / apply / manual-review / MO-016 ordering
- **Preview**: verified non-mutating — the issue's `status` stays `"open"` after
calling the preview endpoint and re-fetching it via a fresh request.
- **Apply**: the confirm button names the exact target status ("Change status to
Blocked" / "Status wijzigen naar Geblokkeerd" / "Changer le statut vers Bloqué");
applying resolves the issue and updates the vehicle atomically.
- **Manual review**: MO-024 (active rental + service-threshold reached, a genuine fact
contradiction) shows "Manual review required" with no generic apply button rendered
at all.
- **Stale token**: simulated by resolving the underlying booking overlap after the
preview was fetched but before applying — the apply call is correctly rejected
(`RECOMMENDATION_STALE`), the UI shows the "situation has changed" message, and the
user must review again before a new apply is possible.
- **MO-016 ordering**: both orders tested. Resolving the overlap first correctly leaves
nothing to apply (vehicle stays "available", genuinely correct). Resolving the status
conflict first safely blocks the vehicle; resolving the now-redundant overlap
afterwards does not disturb it. Neither order ever produces "rented".
## Knowledge (per language)
The brief's exact validation question, in each language, grounds on the damage
procedure as the **primary** (not just top-3) source:
- nl-BE: *"Wat moet ik doen wanneer een voertuig beschadigd terugkomt?"* → damage
procedure, Dutch source, Dutch excerpt.
- en-GB: *"What should I do when a vehicle returns with damage?"* → damage procedure,
English source, English excerpt.
- fr-BE: *"Que dois-je faire lorsqu'un véhicule revient endommagé ?"* → damage
procedure, French source, French excerpt.
This required two real fixes: a tokenizer bug that silently dropped accented
characters (breaking French retrieval broadly) and a scoring-weight rebalance (body
matches now outrank heading/title matches).
## Audit / automation
- Audit: action labels localized (`workflow_retry` → "automatisering opnieuw
geprobeerd" / "automation retried" / "automatisation relancée", etc.), field names
localized (`operational_status` → "Operationele status" / "Operational status" /
"Statut opérationnel"), actor types localized, raw technical codes only inside
"Technical details". Verified live and via a dedicated Playwright test.
- Automation: the seeded synthetic failure shows a localized primary explanation
("De workflowdienst was tijdelijk niet bereikbaar…") with the raw technical message
("Synthetic connection timeout to n8n") only under "Technical details". Verified live
and via a dedicated Playwright test.
## Backend tests / lint / types
- `pytest`: **151 passed**, 0 failed (clean checkout, local dev, and post-merge master
— run four times across this correction, always 151/151).
- `ruff check .`: all checks passed, every run.
- `mypy app` (strict): no issues found in 49 source files, every run.
- Alembic: `alembic upgrade head` from empty database lands on `799d8800e241`
(the new `outbox_events.last_error_code` column); `downgrade -1` / `upgrade head`
round-trip verified.
## Frontend build / Playwright
- `npm ci`, `tsc -b`, `vite build`: clean, every run.
- Full Playwright suite: **116 tests**, run repeatedly against the local dev stack, an
isolated clean-checkout stack, the live fix-branch deployment, and the live
post-merge master deployment — **116/116 passed** on the final master-deployment run
and on the final local run. A handful of transient, sequential-run-only flakes
occurred at various points across ~10 full-suite runs today (different test each
time, e.g. a pre-existing logout-timing race in `AuthContext.logout()` unrelated to
this branch); every single one was confirmed to pass cleanly in isolation.
- Guided demo covered indirectly via `guided-demo-full.spec.ts`,
`demo-guide.spec.ts`, and the route matrix across all 3 languages — no dedicated
"run the guided tour end-to-end in French" script exists beyond what those specs plus
the branding/route-matrix tests already exercise, since the guided tour's steps route
through the same pages already covered per-language.
## Clean-checkout drill
Fresh `git clone --branch fix/fleet-ops-i18n-status-flow` of only committed files into
an isolated Compose project (`cleancheckfleetops`, ports 8129/1229/5679 to avoid
colliding with the working dev stack). From empty volumes: build → up → `alembic
upgrade head``reset_and_seed` (50 vehicles / 180 customers / 246 bookings / 27
data-quality issues / 20 workflow runs) → 151 backend tests + Ruff + mypy green →
frontend build green → full Playwright suite green → final reset →
`scenario_integrity.all_ready: true`. Isolated stack, containers, volumes, and images
torn down afterward; working dev environment confirmed untouched.
## Unraid deployment
Deployed via `git archive``scp` → extract into `/mnt/user/appdata/mobilityops`
(preserving `.env` and persistent volumes) → `.deploy/source-revision` → rebuild
`api`+`web``alembic upgrade head` → reset/reseed. Done twice: once for the fix
branch (caught the evidence-summary bug), once for the final merged master. Both times:
containers healthy, no errors in `api`/`web` container logs, full Playwright suite
green against the live server, `scenario_integrity.all_ready: true` after final reset.
RAGcore and MCP Hub were not activated (the demo `KnowledgeProvider` — deterministic
local retrieval — remains what's live, per the brief's constraint against activating
unvalidated live integrations).
## Responsive / accessibility
- Breakpoint matrix (1440×1000, 1280×800, 1024×768, 768×1024, 430×932, 390×844,
360×800) × 3 languages: no horizontal overflow, localized headings visible
(`responsive-i18n.spec.ts`).
- Status-recommendation panel: keyboard-only activation of "Review recommendation" and
"Change status to X" verified via focus assertions (not just click); reduced-motion
emulated during the flow; status never conveyed by colour alone (the badge always
carries its own localized text); `aria-live="polite"` added so the applied
confirmation is announced to screen readers.
## Known limitations
- A pre-existing, narrow timing race in `AuthContext.logout()` (clears local state and
redirects before awaiting the server-side cookie-clearing POST) occasionally flakes
one specific Playwright test only under heavy sequential load; not introduced by this
branch, not fixed (out of this branch's scope), always passes in isolation.
- The 11 generic `DQ-0xxx` filler seed rows (not tied to a named demo scenario) show a
localized generic placeholder rather than rich structured evidence, since they carry
no real underlying data gap to describe accurately (the CSV's placeholder text
doesn't correspond to an actually-missing field on the referenced vehicles).
- No dedicated "full guided demo in French, screenshot every step" script exists as a
single artifact; coverage is composed from the route matrix, branding, and existing
guided-demo specs, each run across all 3 languages.
## Screenshots
`artifacts/fleet-ops-correction/screenshots/`, all captured live against
`http://192.168.10.150:1236`:
- `login-nl-BE.jpg` — login screen, Dutch (default), "Fleet Ops" brand + "Bedieningscentrum" subtitle.
- `login-fr-BE.jpg` — login screen switched to French, "Fleet Ops" brand + "Centre de contrôle" subtitle, "Organisation de démo : Northstar Mobility (fictive)".
- `dq-demo-status-fr-BE-collapsed.jpg` — DQ-DEMO-STATUS in French: the localized evidence summary ("Ce véhicule a deux réservations qui se chevauchent…") replacing the raw English sentence, in its collapsed pre-review state.
- `dq-demo-status-fr-BE-clean-reload.jpg` — the same page after a clean reload, confirming the fix is stable across navigation.
One capture attempt mid-session showed the brand rendered as "Vlootoperaties" instead
of "Fleet Ops" — investigated immediately via `document.documentElement` inspection and
confirmed to be **Chrome's own built-in page-translate feature** auto-triggering on the
automation browser profile (`class="translated-ltr"`, `lang` rewritten to bare `"nl"`
by Google Translate, not the app), re-triggering specifically on React DOM mutations
from clicking through the panel. Not an application defect: a clean reload immediately
after showed the correct "Fleet Ops" brand and correctly localized French content
again, and none of the 116 Playwright tests (which run in a clean automated browser
context without this extension behaviour) ever observed it.
## Rollback procedure
1. `ssh unraid`, `cd /mnt/user/appdata/mobilityops`.
2. `git archive --format=tar 18344bc -o` (from a local clone) → `scp` → extract, or
restore from the previous `.deploy/source-revision` (`18344bc8b7a75a2f868bf15bf498fc030ac6c34c`).
3. `echo 18344bc8b7a75a2f868bf15bf498fc030ac6c34c > .deploy/source-revision`.
4. `docker compose -f compose.yaml -f compose.unraid.yaml build api web && ... up -d api web`.
5. `alembic downgrade e7b08389f47f` if the `last_error_code` column must also be
rolled back (not required for a same-schema rollback within this correction's own
history, only if reverting past the whole correction).
6. Re-seed and re-verify `scenario_integrity.all_ready: true`.
The fix branch `fix/fleet-ops-i18n-status-flow` was not deleted.