Files
MobilityOps/artifacts/final-integrations/final-summary.md
NuklearRabbitandClaude Sonnet 5 57992bf153 M10: AI Operations Brief runbook, full live e2e regression, final evidence
Ran a real AI Operations Brief via the live ITWorx MCP Hub connector's own
MobilityOpsClient against production Fleet Ops: real operations summary, real
most-pressing vehicle, real grounded knowledge answer with citations, real
correlation IDs verified end-to-end in Fleet Ops's own audit log. No write
actions performed. Runbook and full output in
docs/final-integrations/ai-operations-brief-runbook.md.

Ran the full Playwright e2e suite against the live deployed instance and fixed
two pre-existing fragile locators unrelated to this session's feature work
(both broke because Automation now legitimately has two tables sharing the
same generic selectors, exposed by running the full suite rather than
individual files) plus one pre-existing untranslated-loanword false positive.
All specs pass.

artifacts/final-integrations/final-summary.md has the complete evidence
write-up: repository/deployment state, what was fixed vs. handed off, test
results, and known limitations stated plainly.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-05 13:49:23 +02:00

10 KiB

Fleet Ops final integrations — evidence summary

Session date: 2026-08-05. Branch feat/fleet-ops-final-integrations.

Repository state

Repo Start End Branch Notes
Fleet Ops (MobilityOps) 3ebca9e (from feat/live-n8n-ragcore-integration) 727c19a (+ e2e test fixes, uncommitted at write time) feat/fleet-ops-final-integrations, pushed to origin 3 commits: 34df66d, 2ae2044, 727c19a
RAGcore 64a908a 64a908a (+1 isolated commit ce0ad56) main Only a backlog handoff entry committed; no code changes (36-file concurrent-session collision — see below)
ITWorx MCP Hub not modified this session feature/wp240-final-acceptance Connector already live in production before this session started; not touched

Deployed revisions

  • Fleet Ops: http://192.168.10.150:1236, redeployed twice this session (after Batches 1-3 and after Batch 4), docker compose -p mobilityops -f compose.yaml -f compose.unraid.yaml up --build -d db api web, .deploy/source-revision = 727c19a....
  • RAGcore: http://192.168.10.150:1237, ragcore-app-1. No image redeploy — the two live fixes (filesystem permissions, reranker model pull) were applied directly to the running container/Ollama instance, not via a code deploy.
  • ITWorx MCP Hub: http://192.168.10.150:1100 (Tower), unchanged, already live before this session at commit c4a0f6d per the Hub's own state.

GUI polish (Batch 1)

  • Dashboard Attention Queue: curated severity mix (grouped "Handle now / Follow up today / Review later"), replacing pure severity-sort that let high crowd out everything else.
  • Today's Movements: seed data curated (seed/bookings.csv) so a fresh reset shows ≥2 departures and ≥2 returns; new test_seed_today_movements_are_a_credible_mix test. Live-verified after a real demo reset: 2 returns + 2 departures shown.
  • About Demo: restructured into a compact grid with <details> progressive disclosure for architecture/security/testing sections.
  • Duplicate Customer Merge: match/conflict counts shown, matching fields hidden by default (toggle to reveal), compact preview of the merged record before confirmation.
  • Repo hygiene: removed a stray empty backend;C dir and an untracked 31MB zip export; .gitignore now excludes future archive exports.
  • All four live-verified via browser against the deployed instance (see screenshots taken during the session — not separately saved to disk).

n8n (Batch 2)

  • 4 canonical workflows confirmed live: Vehicle Return Orchestration, Scheduled Data Quality Scan, RAGcore Procedure Sync, Workflow Error Handler.
  • Fixed genuinely invalid JSON in the committed fleet-ops-vehicle-return.json (a missing }, between two node objects — the file could not be parsed).
  • Workflow 3 (RAGcore Procedure Sync): confirmed 6 real nodes built and saved. Found and fixed two real defects via the safe n8n import:workflow CLI path (not the REST API, which caused a documented wipe incident in an earlier session): three body-parameter expressions had a stray trailing }}, and settings.errorWorkflow was unset. Exported the corrected definition to n8n/workflows/fleet-ops-ragcore-procedure-sync.json, added to MANIFEST.md and check_drift.py.
  • Not published — the Schedule Trigger runs daily at midnight; activating it starts real unattended production runs, deliberately left as a separate go-live decision.
  • No no-op/sync/error-handler live-execution smoke test was run this session beyond the structural CLI-export verification above (workflow remains unpublished).

RAGcore (Batch 3)

  • Root cause found and fixed, live, user-approved: the "zero retrieval candidates" bug was a filesystem permission bug (/workspace/.state/models/embedding_profiles.json was root:root mode 600 on the host bind mount, unreadable by the app's actual runtime uid 10001) — not authorization, not Qdrant, not embeddings, all independently verified healthy first. Fixed via chown/chmod; re-verified in-process (5 real hits, up from 0).
  • Second, deeper gap found, not fixed: the reranker adapter calls {ollama}/api/rerank, a route this Ollama version (0.32.5) does not serve (404). Pulled a working model (xitao/bge-reranker-v2-m3:latest, 1.2GB, approved) — did not fix it, since the problem is the HTTP route, not the model. /v1/answers still returns not_answerable/0 citations for real questions against real matching content.
  • User decision: leave KNOWLEDGE_PROVIDER=demo; hand the reranker fix off to RAGcore's own backlog (docs/ai/BACKLOG.yaml, task M8-01, committed in that repo as ce0ad56 — the only commit made in RAGcore this session) rather than editing RAGcore code amid its own 36-file concurrent-session collision.
  • Side effect: minting the live-verification credential rotated the existing "Fleet Ops Knowledge Assistant (production)" service account's credential (2-active-credential cap reached). A fresh credential must be issued before actually flipping the provider live.

MCP Hub (Batch 4)

  • Confirmed the Fleet Ops connector is already live in production on the Hub side (Tower, commit c4a0f6d), with a real contract fix already applied there (vehicle.get's wire parameter normalized to camelCase vehicleRef).
  • Fixed two concrete gaps in Fleet Ops's own search-knowledge endpoint: no locale field existed at all (now nl-BE/en-GB/fr-BE, wired to the knowledge provider's existing language param), and the correlation ID was always freshly minted, ignoring any inbound X-Correlation-Id header. Added get_correlation_id, applied to all four MCP endpoints.
  • MCP_HUB_BASE_URL was dead config (declared, never read); wired it for a real, bounded Hub-reachability health check instead of an unneeded self-registration push (the Hub's own registration is catalog-driven).
  • Renamed Fleet Ops's own internal audit tool labels mobilityops_*fleet_ops_* (mirrored in contracts/mcp-tools.json, mobilityops_* kept as deprecated aliases). The live Hub connector's own dotted tool namespace (mobilityops.operations.summary etc.) is a separate, Hub-owned naming layer, deliberately not touched.
  • Automation page's MCP card now shows real evidence (last tool/client/count/timestamp) instead of only the registration-enabled boolean.

AI Operations Brief (Batch 5)

Real MCP-client-shaped run via the live ITWorx MCP Hub connector's own MobilityOpsClient class against production Fleet Ops. Full runbook and live output in docs/final-integrations/ai-operations-brief-runbook.md. Summary:

  • Real operations summary (21 available / 11 rented / 6 cleaning / 5 maintenance / 7 blocked; 23 open quality issues).
  • Real most-pressing vehicle identified (MO-031, missing operational inspection).
  • Real vehicle detail lookup.
  • Real grounded knowledge answer (English damage-handling question): 2 real citations, evidence_state: grounded.
  • Dutch/French variants of the same question honestly returned insufficient (no fabrication) — root cause: the live Hub connector doesn't yet send the new locale field, a Hub-side follow-up, not silently worked around.
  • Correlation IDs verified end-to-end in Fleet Ops's own audit log (GET /api/v1/audit?action=mcp_tool_request), matching the response payloads exactly.
  • No write actions performed at any point.

Testing per batch

  • Backend: 176 passed, ruff check . clean, mypy app clean (50 source files) — verified against a freshly rebuilt image after discovering mid-session that docker compose run --rm api (no bind mount on the api service) silently tests a stale image otherwise. One genuinely stale test assertion found and fixed as a result.
  • Frontend: tsc -b && vite build clean.
  • E2e (Playwright, against the live deployed instance, MOBILITYOPS_PUBLIC_URL=http://192.168.10.150:1236): every spec file run this session passed — demo.spec.ts, interactive-elements.spec.ts (26), responsive-i18n.spec.ts + demo-accessibility.spec.ts + guided-demo-full.spec.ts (28), i18n-coverage.spec.ts + error-messages.spec.ts + clickable-rows.spec.ts + demo-guide.spec.ts + demo-entry.spec.ts + demo-legibility.spec.ts + fleet-ops-correction.spec.ts + ui-redesign.spec.ts + greeting.spec.ts + greeting-live.spec.ts (28, after fixing 2 pre-existing fragile locators unrelated to this session's feature work — a .data-table ambiguity now that Automation has two tables, and a Technische details toggle ambiguity for the same reason; plus one pre-existing untranslated-loanword false positive in i18n-coverage.spec.ts).

Known limitations, stated plainly

  • KNOWLEDGE_PROVIDER is still demo, not ragcore — blocked on RAGcore's own reranker gap (handed off, not fixed this session).
  • n8n workflow 3 is built and correct but not published (deliberate, separate decision).
  • The live MCP Hub connector doesn't yet send the new locale field, so locale-aware knowledge search only works when called directly against Fleet Ops (as proven by the backend tests), not yet through the live Hub connector as deployed.
  • No public-demo-readiness checklist, About Demo Guide "completed" end-state polish (section 4E), or dashboard MCP "activity showcase after Demo Complete" gating were built this session — the MCP evidence display exists on the Automation page unconditionally rather than gated behind guided-demo completion.
  • No security-review pass was run separately this session (existing gates: ruff, mypy, the repo's own auth/audit test coverage).

Rollback

  • Fleet Ops: prior working revision 0571a40 remains in .deploy/ as source-0571a40.tar.gz on the Unraid host; redeploy by re-extracting and re-running the same docker compose up --build -d sequence with that archive.
  • RAGcore: chown/chmod change is trivially reversible (chown 0:0 + chmod 600 on the same path) if needed, though there is no reason to revert a permission fix. Ollama model pull (xitao/bge-reranker-v2-m3:latest) can be removed with ollama rm if unwanted; it is inert until RAGcore's own code is changed to use it.
  • MCP Hub: not modified this session.