Files
MobilityOps/docs/final-integrations/ai-operations-brief-runbook.md
NuklearRabbitandClaude Sonnet 5 57992bf153 M10: AI Operations Brief runbook, full live e2e regression, final evidence
Ran a real AI Operations Brief via the live ITWorx MCP Hub connector's own
MobilityOpsClient against production Fleet Ops: real operations summary, real
most-pressing vehicle, real grounded knowledge answer with citations, real
correlation IDs verified end-to-end in Fleet Ops's own audit log. No write
actions performed. Runbook and full output in
docs/final-integrations/ai-operations-brief-runbook.md.

Ran the full Playwright e2e suite against the live deployed instance and fixed
two pre-existing fragile locators unrelated to this session's feature work
(both broke because Automation now legitimately has two tables sharing the
same generic selectors, exposed by running the full suite rather than
individual files) plus one pre-existing untranslated-loanword false positive.
All specs pass.

artifacts/final-integrations/final-summary.md has the complete evidence
write-up: repository/deployment state, what was fixed vs. handed off, test
results, and known limitations stated plainly.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-05 13:49:23 +02:00

5.0 KiB

AI Operations Brief — runbook and live evidence

Answers, using a real MCP client through the real ITWorx MCP Hub connector against live production Fleet Ops: "Which vehicles need the most attention today, why, and which internal procedure should be followed for the most important issue?"

What this is not

Not a chatbot. No write actions exist under /api/v1/integrations/mcp/* (verified by test_no_write_endpoints_exist_under_mcp_namespace). Every call below is a plain read-only tool invocation, exactly as the deployed Hub connector performs them.

Reproducible command

Run from inside the live itworx-mcp-hub-connector-mobilityops-1 container (has the real MOBILITYOPS_ENDPOINT, MOBILITYOPS_SERVICE_TOKEN_FILE and the real packages.connector_kit.mobilityops.MobilityOpsClient already available — the same client class the deployed connector uses):

docker exec -e PYTHONPATH=/opt/hub -w /opt/hub \
  itworx-mcp-hub-connector-mobilityops-1 python3 - <<'PY'
from packages.connector_kit.mobilityops import FileTokenProvider, MobilityOpsClient, MobilityOpsSettings
import os, json

settings = MobilityOpsSettings(
    base_url=os.environ["MOBILITYOPS_ENDPOINT"],
    token_reference=os.environ["MOBILITYOPS_SERVICE_TOKEN_FILE"],
)
client = MobilityOpsClient(settings, FileTokenProvider())
client_id = "ai-ops-brief"

summary = client.operations_summary(client_id)
attention = client.attention_vehicles(client_id, minimum_severity="high", date=None, limit=20)
top_ref = attention[0]["vehicle_ref"] if attention else None
details = client.vehicle_details(client_id, top_ref) if top_ref else None
answer = client.search_knowledge(client_id, "What must I do when a vehicle returns with damage?", max_sources=2)

print(json.dumps({"summary": summary, "top_attention": attention[0] if attention else None, "vehicle": details, "answer": answer}, indent=2))
client.close()
PY

This calls, in order: fleet_ops_get_operations_summaryfleet_ops_list_attention_vehiclesfleet_ops_get_vehicle_detailsfleet_ops_search_knowledge. No result is invented — every field printed is exactly what Fleet Ops's live API returned.

Live run, 2026-08-05

  • Operations summary: 21 available, 11 rented, 6 cleaning, 5 maintenance, 7 blocked; 23 open quality issues; 1 pending/failed workflow.
  • Most pressing vehicle: MO-031missing_required_field, "near-future booking; required operational inspection missing", detected 2026-08-05T11:08:33Z. (16 vehicles currently carry a high-severity open issue; MO-031 was the earliest-detected.)
  • Vehicle detail (MO-031): Adria Matrix, 2022, Geel, operational_status: blocked, 41,149 km (service due at 50,000 km), no current booking.
  • Grounded procedure (English): Damage handling procedure v1.3, section "1. Immediate actions" — "When a vehicle returns with visible or reported damage, mark damage in the return inspection, add a concise factual description and keep the vehicle blocked. Do not promise the customer a repair cost or liability decision." Second source: Vehicle return procedure v2.0, section "3. Determine next state". evidence_state: grounded, 2 real citations, provider: demo.
  • Dutch and French variants of the damage question (Wat moet ik doen wanneer een voertuig beschadigd terugkomt?, Que dois-je faire lorsqu'un véhicule revient endommagé ?) both returned evidence_state: insufficient — an honest, non-fabricated "no match" rather than a wrong or invented answer. Root cause: the currently-deployed Hub connector does not yet send the new locale field this session added to search-knowledge (Fleet Ops defaults to en-GB), so non-English question text doesn't match the demo provider's English-tokenized index. A real fix needs a Hub-side connector update to pass locale, tracked as a follow-up, not silently worked around.
  • Correlation ID, end to end, verified: each call's response correlation_id (e.g. abd643a7-2aeb-4af3-806a-da04acb3e444 for the English grounded answer) appears verbatim in Fleet Ops's own audit log (GET /api/v1/audit?action=mcp_tool_request), alongside actor_label: ai-ops-brief-2026-08-05 and the real tool name (fleet_ops_search_knowledge). No write actions were performed; only mcp_tool_request audit rows were created, matching every other live MCP call this integration makes.

Known limitation, stated plainly

The demo knowledge provider (not RAGcore — KNOWLEDGE_PROVIDER=demo, see docs/final-integrations/current-state-audit.md for why) is what grounds the English answer here. It is deterministic, extractive, and never fabricates — but it is not the live RAGcore integration the brief brief for this task set out to exercise; that remains blocked on RAGcore's own reranker gap. This run is honest about that: it proves the full MCP-client → Hub → Fleet Ops → knowledge-provider → audit chain works for real, live, in production, with real data and real citations — using the knowledge provider that is actually configured live today.