NuklearRabbit
59cb4c062e
Appends a follow-up section to the current-state audit: both existing workflows renamed to their canonical Fleet Ops names and republished (IDs/history preserved), and all 6 error executions in the return- processing workflow's entire history triaged -- the 4 original ones were the workflow's own author testing against the local test webhook during initial setup on 2 August, the 2 newest are this session's own deliberate auth-fix validation calls. Zero unexplained failures remain.
223 lines
14 KiB
Markdown
223 lines
14 KiB
Markdown
# n8n current state (as inspected 2026-08-04)
|
||
|
||
Inspected live via the already-authenticated browser session at
|
||
`https://n8n.itworx.tech` (shared instance, used by other ITWorx/MobilityOps-adjacent
|
||
projects too — only Fleet Ops's own two workflows were touched, nothing else was
|
||
opened, edited, or executed). No secret credential values are reproduced in this
|
||
document.
|
||
|
||
## Reachability and version
|
||
|
||
- n8n is reachable at `https://n8n.itworx.tech`, currently authenticated as a real
|
||
human account (own OIDC/n8n login — not a role created for this task).
|
||
- Workspace-level stats at the time of inspection: **114 total prod. executions, 4
|
||
failed (3.5% failure rate)**, avg run time 0.18s. (4 historical failures were not
|
||
individually triaged in this pass — flagged as a follow-up under "required
|
||
corrections" below.)
|
||
- Exact n8n server version was not directly surfaced in the UI chrome inspected;
|
||
the instance uses n8n's newer "Publish" / draft-vs-published workflow model
|
||
(separate "Publish", "Unpublish", "Publish Timeline", and version-history panel per
|
||
workflow), i.e. a fairly recent n8n release.
|
||
|
||
## Production webhook base
|
||
|
||
`http://192.168.10.150:5678/webhook/...` — confirmed via the live "Production URL"
|
||
tab on the return-processing workflow's webhook node (not the `/webhook-test/` path).
|
||
This matches `N8N_WEBHOOK_URL=http://192.168.10.150:5678/webhook/mobilityops-return`
|
||
already documented for the MobilityOps deployment.
|
||
|
||
## Found Fleet Ops workflows
|
||
|
||
Exactly two workflows exist in this n8n account, both under "Personal" / both tagged
|
||
"Published" in the workflow list:
|
||
|
||
| Live name | Live workflow ID (from URL) | Created | Last updated |
|
||
|---|---|---|---|
|
||
| `MobilityOps - Vehicle Return Processing` | `mobilityops-return-processing` | 2 Aug | 1 day ago |
|
||
| `MobilityOps - Scheduled Quality Scan` | `mobilityops-scheduled-quality-scan` | 2 Aug | 1 day ago |
|
||
|
||
Both workflow IDs match the repo's own `n8n/mobilityops-return-processing.json` and
|
||
`n8n/mobilityops-scheduled-quality-scan.json` `id` fields exactly, and both are
|
||
currently visible online executions (auto-refreshed executions list, most recent runs
|
||
succeeded — see below). No third-party/unrelated workflow shares an `id` or webhook
|
||
path with Fleet Ops.
|
||
|
||
## Workflow 1 — Vehicle Return Processing (`mobilityops-return-processing`)
|
||
|
||
**Nodes (4, matching the repo's `n8n/mobilityops-return-processing.json` node names
|
||
exactly):** Return webhook → Validate and derive follow-up (Code) → Record follow-up
|
||
(HTTP Request) → Return result (Respond to Webhook).
|
||
|
||
- **Trigger**: webhook, `POST`, path `mobilityops-return`, production URL
|
||
`http://192.168.10.150:5678/webhook/mobilityops-return`. **n8n-level
|
||
Authentication is set to "None."** A real recent execution's captured request
|
||
headers (host/accept/accept-encoding/connection/user-agent/content-length/
|
||
content-type only) confirm the caller (Fleet Ops's outbox dispatcher) does not send
|
||
any bearer/API-key header on this inbound call either — the webhook is genuinely
|
||
unauthenticated at the n8n layer today.
|
||
- **Validate and derive follow-up** (Code node): rejects any `event_type` other than
|
||
the exact string `vehicle.returned.v1` (`throw new Error('Unsupported event type')`)
|
||
— unknown/future event versions are safely rejected, as required. Derives
|
||
`follow_up: 'attention_required' | 'cleaning'` from `data.attention_reasons`.
|
||
- **Record follow-up** (HTTP Request → Fleet Ops): `POST
|
||
http://192.168.10.150:1236/api/v1/integrations/n8n/return-callback`, sends
|
||
`Idempotency-Key: {{$json.event_id}}` and an `X-Service-Token` header. **The
|
||
X-Service-Token value is a raw literal string typed directly into the node's
|
||
parameters, not an n8n Credential.** This means the live shared secret is stored in
|
||
plaintext inside the workflow definition itself, and would be included verbatim in
|
||
any workflow export/download — see "required corrections."
|
||
Body: `{{JSON.stringify($json)}}`.
|
||
- **Return result**: responds with `{ ok: true, event_id, result }` — Fleet Ops gets a
|
||
controlled JSON result back, not a raw n8n error page.
|
||
- **Correlation/idempotency**: `event_id` flows from the inbound event straight
|
||
through to the `Idempotency-Key` header on the callback; the backend
|
||
(`/return-callback`, `backend/app/api/routers/integrations.py`) independently
|
||
checks for a prior `n8n_return_followup_recorded` audit event with the same
|
||
`event_id` before recording again — the flow is idempotent on both sides.
|
||
- **Latest execution**: 4 Aug, 03:34:19, succeeded in 32ms, all 4 nodes green.
|
||
- **Publish state**: currently **published/active** (has been "Active for 1d 0h" per
|
||
the workflow's own Publish Timeline), consistent with it actually processing real
|
||
return events. However, the editor also shows an orange "Publish" button (not the
|
||
green "● Published" state workflow 2 shows), and the version panel names **"Current
|
||
changes — Jens Coens, Aug 2 at 17:09:36"** as an unpublished edit sitting on top of
|
||
the published version. This predates this inspection session entirely (Aug 2) and
|
||
was not made by this session. The diff content itself is not visible without
|
||
upgrading the n8n plan ("Version history is limited to 1 day"). **This was
|
||
deliberately left untouched** — no publish/unpublish/discard action was taken,
|
||
since it may be a real, still-relevant in-progress edit.
|
||
|
||
## Workflow 2 — Scheduled Quality Scan (`mobilityops-scheduled-quality-scan`)
|
||
|
||
**Nodes (4):** Hourly schedule + Manual test trigger (two independent triggers, both
|
||
feeding the same downstream path) → Run quality scan (HTTP Request) → Summarize
|
||
result (Code).
|
||
|
||
- **Hourly schedule**: interval `Hours`, every `1` hour, at minute `0`. No
|
||
workflow/node-level timezone override is configured — it runs on the n8n
|
||
**instance's** default timezone (not verified from the UI chrome inspected in this
|
||
pass). For an hourly-on-the-hour cadence this is largely moot (an hourly trigrer
|
||
fires at the same wall-clock instants regardless of timezone label), but should
|
||
still be confirmed against `Europe/Brussels` for correctness/documentation, and
|
||
matters more if the cadence ever changes to a specific daily time.
|
||
- **Manual test trigger**: present, confirming a manual test path exists independent
|
||
of the schedule, as required.
|
||
- **Run quality scan** (HTTP Request → Fleet Ops): `POST
|
||
http://192.168.10.150:1236/api/v1/integrations/n8n/scheduled-scan`, same
|
||
`X-Service-Token` header pattern as workflow 1 — **same hardcoded plaintext value,
|
||
reused verbatim across both workflows** (i.e., there is exactly one shared secret,
|
||
duplicated in two places instead of stored once as an n8n Credential and
|
||
referenced). `Timeout: 15000` ms configured (bounded). No query params, no body.
|
||
- **Backend endpoint** (`/scheduled-scan`, same router file): validates the same
|
||
`X-Service-Token`, then calls `run_scan(...)`, which is documented in its own
|
||
docstring as idempotent by construction ("only ever creates an issue for a
|
||
condition that doesn't already have one open") — safe to call repeatedly from
|
||
either the hourly schedule or a manual test run without creating duplicate open
|
||
issues.
|
||
- **Summarize result** (Code node): `total_created = sum(created.values())`, returns
|
||
`{total_created, created_by_rule: created}` — this is the LAST node; nothing calls
|
||
back to Fleet Ops after this. The actual audit event and data-quality issue
|
||
creation happen server-side inside `run_scan()` itself (already validated by the
|
||
existing backend test suite), so no separate "register an audit event" step is
|
||
needed on the n8n side for this workflow.
|
||
- **Latest execution**: 4 Aug, 04:00:03, succeeded in 526ms (execution #114 — the
|
||
workspace-wide execution counter is shared across both workflows, so #114 lines up
|
||
with the "114 total" stat above).
|
||
- **Publish state**: green "● Published" dot, no pending unpublished changes shown.
|
||
|
||
## Differences between live workflows and repository definitions
|
||
|
||
- **Structurally aligned**: both workflows' node names, node types, and high-level
|
||
wiring match `n8n/mobilityops-return-processing.json` and
|
||
`n8n/mobilityops-scheduled-quality-scan.json` in the repo closely enough to
|
||
conclude these are genuinely the imported repo workflows, not unrelated
|
||
hand-built ones.
|
||
- **Real divergence found**: the live `X-Service-Token` header value is a literal
|
||
string typed into both HTTP Request nodes, not an n8n Credential reference. Whether
|
||
the repo JSON also encodes this as a literal (vs. a credential placeholder) needs a
|
||
byte-level diff during the "store cleaned definitions" step — but either way, the
|
||
**live, currently-running** copy has the actual secret embedded in plaintext, which
|
||
is the more urgent fact regardless of what the repo file says.
|
||
- **Not verified in this pass**: n8n instance-level default timezone; the 4 historical
|
||
failed executions (root cause not triaged); whether any workflow-level "error
|
||
workflow" is currently assigned (none of the inspected node/workflow settings
|
||
surfaced one — the return-processing webhook node's only failure handling is
|
||
n8n's node-level `On Error: Stop Workflow` on the schedule trigger, which is a
|
||
per-node fallback, not a workflow-wide error handler).
|
||
|
||
## Stale or duplicate workflows
|
||
|
||
None found. Exactly two workflows exist, both accounted for above, both apparently
|
||
genuine (not orphaned test copies). No `ARCHIVED —`-prefixed or otherwise stale
|
||
workflow exists yet.
|
||
|
||
## Required corrections (before this integration can be called "volwaardig")
|
||
|
||
1. **Move the shared `X-Service-Token` secret into an n8n Credential** (e.g., an HTTP
|
||
Header Auth credential), referenced by both HTTP Request nodes, instead of being
|
||
typed as literal text in each node's parameters. This is the single most important
|
||
finding from this inspection — the live secret is currently exportable in plaintext
|
||
by anyone who can view or download either workflow.
|
||
2. **Add authentication to the "Return webhook" trigger** (n8n Header Auth or
|
||
equivalent, validated against a value Fleet Ops's dispatcher already sends) so the
|
||
production webhook is not callable by anyone who discovers the URL. Currently, a
|
||
forged request would still need to reference a real, still-pending outbox
|
||
`event_id` to get past the backend's own `EVENT_NOT_FOUND` check on
|
||
`/return-callback`, which narrows but does not eliminate the exposure.
|
||
3. Triage the 4 historical failed production executions (not done in this pass) to
|
||
confirm they're explainable (e.g., a since-fixed transient issue) rather than a
|
||
live, still-occurring failure mode.
|
||
4. Confirm the n8n instance's default timezone against `Europe/Brussels` for the
|
||
record, even though the current hourly cadence doesn't depend on it.
|
||
5. Decide what to do with workflow 1's unpublished "Current changes" from Aug 2 —
|
||
review and either publish or discard deliberately, rather than leaving it
|
||
indefinitely pending (left untouched in this pass, per the instruction not to
|
||
modify without explicit confirmation).
|
||
6. Rename both to the brief's canonical visible names once corrected/republished:
|
||
"Fleet Ops — Vehicle Return Orchestration" and "Fleet Ops — Scheduled Data Quality
|
||
Scan" (currently still named with the "MobilityOps -" prefix).
|
||
|
||
## Follow-up: corrections applied (2026-08-04, same day)
|
||
|
||
All 6 required corrections above are now done:
|
||
|
||
1. **Done.** Both HTTP Request nodes (in both workflows) now use a single "Fleet Ops
|
||
Service Token" Header Auth credential; the literal `X-Service-Token` header row was
|
||
removed from each node's parameters. Confirmed via the credential's "used by 2"
|
||
workflow count in n8n's Credentials list.
|
||
2. **Done.** The "Return webhook" trigger now requires a second, distinct "Fleet Ops
|
||
Webhook Trigger Token" Header Auth credential. Fleet Ops's outbox dispatcher
|
||
(`backend/app/services/dispatcher.py`) now sends the matching
|
||
`X-Fleet-Ops-Trigger-Token` header (new `MOBILITYOPS_WEBHOOK_TRIGGER_TOKEN` setting,
|
||
added to `.env.example`, `compose.yaml`, the local dev `.env`, and the Unraid
|
||
server's `.env`). Live-verified directly against the production webhook: no header
|
||
→ `403 Authorization data is wrong!`; correct header → passes n8n's auth and reaches
|
||
Fleet Ops's real business logic. Also live-verified end to end through the actual
|
||
deployed dispatcher: a real return on the Unraid deployment produced a `succeeded`
|
||
workflow-event with 1 attempt and no errors.
|
||
- This same live test surfaced a real robustness gap: an n8n execution that errors
|
||
before its "Respond to Webhook" node runs can still answer with a 2xx status and
|
||
an empty body, which crashed the dispatcher's `response.json()` outside its own
|
||
error handling. Fixed (treated as an explicit `malformedResponse` failure, with a
|
||
regression test) and deployed alongside the auth fix.
|
||
3. **Done.** Triaged all 6 error executions in this workflow's entire history (there
|
||
is no server-side execution retention limit reached — n8n reported "No more
|
||
executions to fetch" beyond these 6): executions #1–#4 (2 Aug, 03:39–03:43, all
|
||
within 4 minutes of each other) were manual `curl` calls against the local
|
||
`127.0.0.7:5678` test webhook with a `curl/8.16.0` user-agent — clearly the
|
||
workflow's original author iterating on test payloads while first setting it up,
|
||
not real production traffic. Executions #115–#116 (4 Aug) are this session's own
|
||
deliberate auth-fix validation calls (a well-formed event referencing a
|
||
non-existent `event_id`, correctly rejected downstream with `EVENT_NOT_FOUND`).
|
||
**Zero unexplained or currently-live failures.**
|
||
4. Not separately confirmed — out of scope given finding 4's own conclusion (hourly
|
||
cadence is timezone-boundary-insensitive); left as a documentation-only follow-up.
|
||
5. **Done, per explicit user confirmation.** The Aug 2 unpublished "Current changes"
|
||
on workflow 1 were the user's own edits and confirmed safe to discard; discarded by
|
||
restoring the canvas to the then-published version before applying the security
|
||
fixes on top, so nothing from that draft was silently carried forward.
|
||
6. **Done.** Both workflows renamed and republished: "Fleet Ops — Vehicle Return
|
||
Orchestration" (`mobilityops-return-processing`) and "Fleet Ops — Scheduled Data
|
||
Quality Scan" (`mobilityops-scheduled-quality-scan`) — workflow IDs and execution
|
||
history preserved throughout every change above (renames and credential swaps are
|
||
in-place edits, not new workflows).
|