Files
MobilityOps/docs/live-ai-integration/n8n-current-state.md
T
NuklearRabbit 59cb4c062e docs: record n8n corrections applied and failure-history triage
Appends a follow-up section to the current-state audit: both existing
workflows renamed to their canonical Fleet Ops names and republished
(IDs/history preserved), and all 6 error executions in the return-
processing workflow's entire history triaged -- the 4 original ones
were the workflow's own author testing against the local test webhook
during initial setup on 2 August, the 2 newest are this session's own
deliberate auth-fix validation calls. Zero unexplained failures remain.
2026-08-04 09:15:08 +02:00

14 KiB
Raw Blame History

n8n current state (as inspected 2026-08-04)

Inspected live via the already-authenticated browser session at https://n8n.itworx.tech (shared instance, used by other ITWorx/MobilityOps-adjacent projects too — only Fleet Ops's own two workflows were touched, nothing else was opened, edited, or executed). No secret credential values are reproduced in this document.

Reachability and version

  • n8n is reachable at https://n8n.itworx.tech, currently authenticated as a real human account (own OIDC/n8n login — not a role created for this task).
  • Workspace-level stats at the time of inspection: 114 total prod. executions, 4 failed (3.5% failure rate), avg run time 0.18s. (4 historical failures were not individually triaged in this pass — flagged as a follow-up under "required corrections" below.)
  • Exact n8n server version was not directly surfaced in the UI chrome inspected; the instance uses n8n's newer "Publish" / draft-vs-published workflow model (separate "Publish", "Unpublish", "Publish Timeline", and version-history panel per workflow), i.e. a fairly recent n8n release.

Production webhook base

http://192.168.10.150:5678/webhook/... — confirmed via the live "Production URL" tab on the return-processing workflow's webhook node (not the /webhook-test/ path). This matches N8N_WEBHOOK_URL=http://192.168.10.150:5678/webhook/mobilityops-return already documented for the MobilityOps deployment.

Found Fleet Ops workflows

Exactly two workflows exist in this n8n account, both under "Personal" / both tagged "Published" in the workflow list:

Live name Live workflow ID (from URL) Created Last updated
MobilityOps - Vehicle Return Processing mobilityops-return-processing 2 Aug 1 day ago
MobilityOps - Scheduled Quality Scan mobilityops-scheduled-quality-scan 2 Aug 1 day ago

Both workflow IDs match the repo's own n8n/mobilityops-return-processing.json and n8n/mobilityops-scheduled-quality-scan.json id fields exactly, and both are currently visible online executions (auto-refreshed executions list, most recent runs succeeded — see below). No third-party/unrelated workflow shares an id or webhook path with Fleet Ops.

Workflow 1 — Vehicle Return Processing (mobilityops-return-processing)

Nodes (4, matching the repo's n8n/mobilityops-return-processing.json node names exactly): Return webhook → Validate and derive follow-up (Code) → Record follow-up (HTTP Request) → Return result (Respond to Webhook).

  • Trigger: webhook, POST, path mobilityops-return, production URL http://192.168.10.150:5678/webhook/mobilityops-return. n8n-level Authentication is set to "None." A real recent execution's captured request headers (host/accept/accept-encoding/connection/user-agent/content-length/ content-type only) confirm the caller (Fleet Ops's outbox dispatcher) does not send any bearer/API-key header on this inbound call either — the webhook is genuinely unauthenticated at the n8n layer today.
  • Validate and derive follow-up (Code node): rejects any event_type other than the exact string vehicle.returned.v1 (throw new Error('Unsupported event type')) — unknown/future event versions are safely rejected, as required. Derives follow_up: 'attention_required' | 'cleaning' from data.attention_reasons.
  • Record follow-up (HTTP Request → Fleet Ops): POST http://192.168.10.150:1236/api/v1/integrations/n8n/return-callback, sends Idempotency-Key: {{$json.event_id}} and an X-Service-Token header. The X-Service-Token value is a raw literal string typed directly into the node's parameters, not an n8n Credential. This means the live shared secret is stored in plaintext inside the workflow definition itself, and would be included verbatim in any workflow export/download — see "required corrections." Body: {{JSON.stringify($json)}}.
  • Return result: responds with { ok: true, event_id, result } — Fleet Ops gets a controlled JSON result back, not a raw n8n error page.
  • Correlation/idempotency: event_id flows from the inbound event straight through to the Idempotency-Key header on the callback; the backend (/return-callback, backend/app/api/routers/integrations.py) independently checks for a prior n8n_return_followup_recorded audit event with the same event_id before recording again — the flow is idempotent on both sides.
  • Latest execution: 4 Aug, 03:34:19, succeeded in 32ms, all 4 nodes green.
  • Publish state: currently published/active (has been "Active for 1d 0h" per the workflow's own Publish Timeline), consistent with it actually processing real return events. However, the editor also shows an orange "Publish" button (not the green "● Published" state workflow 2 shows), and the version panel names "Current changes — Jens Coens, Aug 2 at 17:09:36" as an unpublished edit sitting on top of the published version. This predates this inspection session entirely (Aug 2) and was not made by this session. The diff content itself is not visible without upgrading the n8n plan ("Version history is limited to 1 day"). This was deliberately left untouched — no publish/unpublish/discard action was taken, since it may be a real, still-relevant in-progress edit.

Workflow 2 — Scheduled Quality Scan (mobilityops-scheduled-quality-scan)

Nodes (4): Hourly schedule + Manual test trigger (two independent triggers, both feeding the same downstream path) → Run quality scan (HTTP Request) → Summarize result (Code).

  • Hourly schedule: interval Hours, every 1 hour, at minute 0. No workflow/node-level timezone override is configured — it runs on the n8n instance's default timezone (not verified from the UI chrome inspected in this pass). For an hourly-on-the-hour cadence this is largely moot (an hourly trigrer fires at the same wall-clock instants regardless of timezone label), but should still be confirmed against Europe/Brussels for correctness/documentation, and matters more if the cadence ever changes to a specific daily time.
  • Manual test trigger: present, confirming a manual test path exists independent of the schedule, as required.
  • Run quality scan (HTTP Request → Fleet Ops): POST http://192.168.10.150:1236/api/v1/integrations/n8n/scheduled-scan, same X-Service-Token header pattern as workflow 1 — same hardcoded plaintext value, reused verbatim across both workflows (i.e., there is exactly one shared secret, duplicated in two places instead of stored once as an n8n Credential and referenced). Timeout: 15000 ms configured (bounded). No query params, no body.
  • Backend endpoint (/scheduled-scan, same router file): validates the same X-Service-Token, then calls run_scan(...), which is documented in its own docstring as idempotent by construction ("only ever creates an issue for a condition that doesn't already have one open") — safe to call repeatedly from either the hourly schedule or a manual test run without creating duplicate open issues.
  • Summarize result (Code node): total_created = sum(created.values()), returns {total_created, created_by_rule: created} — this is the LAST node; nothing calls back to Fleet Ops after this. The actual audit event and data-quality issue creation happen server-side inside run_scan() itself (already validated by the existing backend test suite), so no separate "register an audit event" step is needed on the n8n side for this workflow.
  • Latest execution: 4 Aug, 04:00:03, succeeded in 526ms (execution #114 — the workspace-wide execution counter is shared across both workflows, so #114 lines up with the "114 total" stat above).
  • Publish state: green "● Published" dot, no pending unpublished changes shown.

Differences between live workflows and repository definitions

  • Structurally aligned: both workflows' node names, node types, and high-level wiring match n8n/mobilityops-return-processing.json and n8n/mobilityops-scheduled-quality-scan.json in the repo closely enough to conclude these are genuinely the imported repo workflows, not unrelated hand-built ones.
  • Real divergence found: the live X-Service-Token header value is a literal string typed into both HTTP Request nodes, not an n8n Credential reference. Whether the repo JSON also encodes this as a literal (vs. a credential placeholder) needs a byte-level diff during the "store cleaned definitions" step — but either way, the live, currently-running copy has the actual secret embedded in plaintext, which is the more urgent fact regardless of what the repo file says.
  • Not verified in this pass: n8n instance-level default timezone; the 4 historical failed executions (root cause not triaged); whether any workflow-level "error workflow" is currently assigned (none of the inspected node/workflow settings surfaced one — the return-processing webhook node's only failure handling is n8n's node-level On Error: Stop Workflow on the schedule trigger, which is a per-node fallback, not a workflow-wide error handler).

Stale or duplicate workflows

None found. Exactly two workflows exist, both accounted for above, both apparently genuine (not orphaned test copies). No ARCHIVED —-prefixed or otherwise stale workflow exists yet.

Required corrections (before this integration can be called "volwaardig")

  1. Move the shared X-Service-Token secret into an n8n Credential (e.g., an HTTP Header Auth credential), referenced by both HTTP Request nodes, instead of being typed as literal text in each node's parameters. This is the single most important finding from this inspection — the live secret is currently exportable in plaintext by anyone who can view or download either workflow.
  2. Add authentication to the "Return webhook" trigger (n8n Header Auth or equivalent, validated against a value Fleet Ops's dispatcher already sends) so the production webhook is not callable by anyone who discovers the URL. Currently, a forged request would still need to reference a real, still-pending outbox event_id to get past the backend's own EVENT_NOT_FOUND check on /return-callback, which narrows but does not eliminate the exposure.
  3. Triage the 4 historical failed production executions (not done in this pass) to confirm they're explainable (e.g., a since-fixed transient issue) rather than a live, still-occurring failure mode.
  4. Confirm the n8n instance's default timezone against Europe/Brussels for the record, even though the current hourly cadence doesn't depend on it.
  5. Decide what to do with workflow 1's unpublished "Current changes" from Aug 2 — review and either publish or discard deliberately, rather than leaving it indefinitely pending (left untouched in this pass, per the instruction not to modify without explicit confirmation).
  6. Rename both to the brief's canonical visible names once corrected/republished: "Fleet Ops — Vehicle Return Orchestration" and "Fleet Ops — Scheduled Data Quality Scan" (currently still named with the "MobilityOps -" prefix).

Follow-up: corrections applied (2026-08-04, same day)

All 6 required corrections above are now done:

  1. Done. Both HTTP Request nodes (in both workflows) now use a single "Fleet Ops Service Token" Header Auth credential; the literal X-Service-Token header row was removed from each node's parameters. Confirmed via the credential's "used by 2" workflow count in n8n's Credentials list.
  2. Done. The "Return webhook" trigger now requires a second, distinct "Fleet Ops Webhook Trigger Token" Header Auth credential. Fleet Ops's outbox dispatcher (backend/app/services/dispatcher.py) now sends the matching X-Fleet-Ops-Trigger-Token header (new MOBILITYOPS_WEBHOOK_TRIGGER_TOKEN setting, added to .env.example, compose.yaml, the local dev .env, and the Unraid server's .env). Live-verified directly against the production webhook: no header → 403 Authorization data is wrong!; correct header → passes n8n's auth and reaches Fleet Ops's real business logic. Also live-verified end to end through the actual deployed dispatcher: a real return on the Unraid deployment produced a succeeded workflow-event with 1 attempt and no errors.
    • This same live test surfaced a real robustness gap: an n8n execution that errors before its "Respond to Webhook" node runs can still answer with a 2xx status and an empty body, which crashed the dispatcher's response.json() outside its own error handling. Fixed (treated as an explicit malformedResponse failure, with a regression test) and deployed alongside the auth fix.
  3. Done. Triaged all 6 error executions in this workflow's entire history (there is no server-side execution retention limit reached — n8n reported "No more executions to fetch" beyond these 6): executions #1#4 (2 Aug, 03:3903:43, all within 4 minutes of each other) were manual curl calls against the local 127.0.0.7:5678 test webhook with a curl/8.16.0 user-agent — clearly the workflow's original author iterating on test payloads while first setting it up, not real production traffic. Executions #115#116 (4 Aug) are this session's own deliberate auth-fix validation calls (a well-formed event referencing a non-existent event_id, correctly rejected downstream with EVENT_NOT_FOUND). Zero unexplained or currently-live failures.
  4. Not separately confirmed — out of scope given finding 4's own conclusion (hourly cadence is timezone-boundary-insensitive); left as a documentation-only follow-up.
  5. Done, per explicit user confirmation. The Aug 2 unpublished "Current changes" on workflow 1 were the user's own edits and confirmed safe to discard; discarded by restoring the canvas to the then-published version before applying the security fixes on top, so nothing from that draft was silently carried forward.
  6. Done. Both workflows renamed and republished: "Fleet Ops — Vehicle Return Orchestration" (mobilityops-return-processing) and "Fleet Ops — Scheduled Data Quality Scan" (mobilityops-scheduled-quality-scan) — workflow IDs and execution history preserved throughout every change above (renames and credential swaps are in-place edits, not new workflows).