MobilityOps/artifacts/final-integrations/final-summary.md

# Fleet Ops final integrations — evidence summary

Session date: 2026-08-05. Branch `feat/fleet-ops-final-integrations`.

## Repository state

| Repo | Start | End | Branch | Notes |
|---|---|---|---|---|
| Fleet Ops (MobilityOps) | `3ebca9e` (from `feat/live-n8n-ragcore-integration`) | `727c19a` (+ e2e test fixes, uncommitted at write time) | `feat/fleet-ops-final-integrations`, pushed to `origin` | 3 commits: `34df66d`, `2ae2044`, `727c19a` |
| RAGcore | `64a908a` | `64a908a` (+1 isolated commit `ce0ad56`) | `main` | Only a backlog handoff entry committed; no code changes (36-file concurrent-session collision — see below) |
| ITWorx MCP Hub | not modified this session | — | `feature/wp240-final-acceptance` | Connector already live in production before this session started; not touched |

## Deployed revisions

- Fleet Ops: `http://192.168.10.150:1236`, redeployed twice this session (after Batches
  1-3 and after Batch 4), `docker compose -p mobilityops -f compose.yaml -f
  compose.unraid.yaml up --build -d db api web`, `.deploy/source-revision` = `727c19a...`.
- RAGcore: `http://192.168.10.150:1237`, `ragcore-app-1`. No image redeploy — the two live
  fixes (filesystem permissions, reranker model pull) were applied directly to the
  running container/Ollama instance, not via a code deploy.
- ITWorx MCP Hub: `http://192.168.10.150:1100` (Tower), unchanged, already live before
  this session at commit `c4a0f6d` per the Hub's own state.

## GUI polish (Batch 1)

- Dashboard Attention Queue: curated severity mix (grouped "Handle now / Follow up
  today / Review later"), replacing pure severity-sort that let `high` crowd out
  everything else.
- Today's Movements: seed data curated (`seed/bookings.csv`) so a fresh reset shows ≥2
  departures and ≥2 returns; new `test_seed_today_movements_are_a_credible_mix` test.
  Live-verified after a real demo reset: 2 returns + 2 departures shown.
- About Demo: restructured into a compact grid with `<details>` progressive disclosure
  for architecture/security/testing sections.
- Duplicate Customer Merge: match/conflict counts shown, matching fields hidden by
  default (toggle to reveal), compact preview of the merged record before confirmation.
- Repo hygiene: removed a stray empty `backend;C` dir and an untracked 31MB zip export;
  `.gitignore` now excludes future archive exports.
- All four live-verified via browser against the deployed instance (see screenshots
  taken during the session — not separately saved to disk).

## n8n (Batch 2)

- 4 canonical workflows confirmed live: Vehicle Return Orchestration, Scheduled Data
  Quality Scan, RAGcore Procedure Sync, Workflow Error Handler.
- Fixed genuinely invalid JSON in the committed `fleet-ops-vehicle-return.json` (a
  missing `},` between two node objects — the file could not be parsed).
- Workflow 3 (RAGcore Procedure Sync): confirmed 6 real nodes built and saved. Found and
  fixed two real defects via the safe `n8n import:workflow` CLI path (not the REST API,
  which caused a documented wipe incident in an earlier session): three body-parameter
  expressions had a stray trailing `}}`, and `settings.errorWorkflow` was unset. Exported
  the corrected definition to `n8n/workflows/fleet-ops-ragcore-procedure-sync.json`,
  added to `MANIFEST.md` and `check_drift.py`.
- **Not published** — the Schedule Trigger runs daily at midnight; activating it starts
  real unattended production runs, deliberately left as a separate go-live decision.
- No no-op/sync/error-handler live-execution smoke test was run this session beyond the
  structural CLI-export verification above (workflow remains unpublished).

## RAGcore (Batch 3)

- **Root cause found and fixed, live, user-approved**: the "zero retrieval candidates"
  bug was a filesystem permission bug (`/workspace/.state/models/embedding_profiles.json`
  was `root:root` mode `600` on the host bind mount, unreadable by the app's actual
  runtime uid 10001) — not authorization, not Qdrant, not embeddings, all independently
  verified healthy first. Fixed via `chown`/`chmod`; re-verified in-process (5 real hits,
  up from 0).
- **Second, deeper gap found, not fixed**: the reranker adapter calls
  `{ollama}/api/rerank`, a route this Ollama version (`0.32.5`) does not serve (404).
  Pulled a working model (`xitao/bge-reranker-v2-m3:latest`, 1.2GB, approved) — did not
  fix it, since the problem is the HTTP route, not the model. `/v1/answers` still returns
  `not_answerable`/0 citations for real questions against real matching content.
- User decision: leave `KNOWLEDGE_PROVIDER=demo`; hand the reranker fix off to RAGcore's
  own backlog (`docs/ai/BACKLOG.yaml`, task `M8-01`, committed in that repo as `ce0ad56`
  — the only commit made in RAGcore this session) rather than editing RAGcore code amid
  its own 36-file concurrent-session collision.
- Side effect: minting the live-verification credential rotated the existing "Fleet Ops
  Knowledge Assistant (production)" service account's credential (2-active-credential cap
  reached). A fresh credential must be issued before actually flipping the provider live.

## MCP Hub (Batch 4)

- Confirmed the Fleet Ops connector is already live in production on the Hub side
  (Tower, commit `c4a0f6d`), with a real contract fix already applied there
  (`vehicle.get`'s wire parameter normalized to camelCase `vehicleRef`).
- Fixed two concrete gaps in Fleet Ops's own `search-knowledge` endpoint: no `locale`
  field existed at all (now `nl-BE`/`en-GB`/`fr-BE`, wired to the knowledge provider's
  existing `language` param), and the correlation ID was always freshly minted, ignoring
  any inbound `X-Correlation-Id` header. Added `get_correlation_id`, applied to all four
  MCP endpoints.
- `MCP_HUB_BASE_URL` was dead config (declared, never read); wired it for a real,
  bounded Hub-reachability health check instead of an unneeded self-registration push
  (the Hub's own registration is catalog-driven).
- Renamed Fleet Ops's own internal audit tool labels `mobilityops_*` → `fleet_ops_*`
  (mirrored in `contracts/mcp-tools.json`, `mobilityops_*` kept as deprecated aliases).
  The live Hub connector's own dotted tool namespace (`mobilityops.operations.summary`
  etc.) is a separate, Hub-owned naming layer, deliberately not touched.
- Automation page's MCP card now shows real evidence (last tool/client/count/timestamp)
  instead of only the registration-enabled boolean.

## AI Operations Brief (Batch 5)

Real MCP-client-shaped run via the live ITWorx MCP Hub connector's own
`MobilityOpsClient` class against production Fleet Ops. Full runbook and live output in
`docs/final-integrations/ai-operations-brief-runbook.md`. Summary:

- Real operations summary (21 available / 11 rented / 6 cleaning / 5 maintenance /
  7 blocked; 23 open quality issues).
- Real most-pressing vehicle identified (`MO-031`, missing operational inspection).
- Real vehicle detail lookup.
- Real grounded knowledge answer (English damage-handling question): 2 real citations,
  `evidence_state: grounded`.
- Dutch/French variants of the same question honestly returned `insufficient` (no
  fabrication) — root cause: the live Hub connector doesn't yet send the new `locale`
  field, a Hub-side follow-up, not silently worked around.
- Correlation IDs verified end-to-end in Fleet Ops's own audit log
  (`GET /api/v1/audit?action=mcp_tool_request`), matching the response payloads exactly.
- No write actions performed at any point.

## Testing per batch

- Backend: **176 passed**, `ruff check .` clean, `mypy app` clean (50 source files) —
  verified against a freshly rebuilt image after discovering mid-session that
  `docker compose run --rm api` (no bind mount on the `api` service) silently tests a
  stale image otherwise. One genuinely stale test assertion found and fixed as a result.
- Frontend: `tsc -b && vite build` clean.
- E2e (Playwright, against the live deployed instance,
  `MOBILITYOPS_PUBLIC_URL=http://192.168.10.150:1236`): every spec file run this
  session passed — `demo.spec.ts`, `interactive-elements.spec.ts` (26),
  `responsive-i18n.spec.ts` + `demo-accessibility.spec.ts` + `guided-demo-full.spec.ts`
  (28), `i18n-coverage.spec.ts` + `error-messages.spec.ts` + `clickable-rows.spec.ts` +
  `demo-guide.spec.ts` + `demo-entry.spec.ts` + `demo-legibility.spec.ts` +
  `fleet-ops-correction.spec.ts` + `ui-redesign.spec.ts` + `greeting.spec.ts` +
  `greeting-live.spec.ts` (28, after fixing 2 pre-existing fragile locators unrelated to
  this session's feature work — a `.data-table` ambiguity now that Automation has two
  tables, and a `Technische details` toggle ambiguity for the same reason; plus one
  pre-existing untranslated-loanword false positive in `i18n-coverage.spec.ts`).

## Known limitations, stated plainly

- `KNOWLEDGE_PROVIDER` is still `demo`, not `ragcore` — blocked on RAGcore's own
  reranker gap (handed off, not fixed this session).
- n8n workflow 3 is built and correct but not published (deliberate, separate decision).
- The live MCP Hub connector doesn't yet send the new `locale` field, so
  locale-aware knowledge search only works when called directly against Fleet Ops (as
  proven by the backend tests), not yet through the live Hub connector as deployed.
- No public-demo-readiness checklist, About Demo Guide "completed" end-state polish
  (section 4E), or dashboard MCP "activity showcase after Demo Complete" gating were
  built this session — the MCP evidence display exists on the Automation page
  unconditionally rather than gated behind guided-demo completion.
- No security-review pass was run separately this session (existing gates: ruff, mypy,
  the repo's own auth/audit test coverage).

## Rollback

- Fleet Ops: prior working revision `0571a40` remains in `.deploy/` as
  `source-0571a40.tar.gz` on the Unraid host; redeploy by re-extracting and re-running
  the same `docker compose up --build -d` sequence with that archive.
- RAGcore: `chown`/`chmod` change is trivially reversible (`chown 0:0` +
  `chmod 600` on the same path) if needed, though there is no reason to revert a
  permission fix. Ollama model pull (`xitao/bge-reranker-v2-m3:latest`) can be removed
  with `ollama rm` if unwanted; it is inert until RAGcore's own code is changed to use it.
- MCP Hub: not modified this session.