MobilityOps/PROJECT_STATE.md

# Project state

## Current milestone

M3 — complete. Starting M4 next.

## Locked decisions

- Product name: MobilityOps.
- Fictitious tenant: Northstar Mobility Demo.
- PoC only; all operational and knowledge data are synthetic.
- Core stack and boundaries are defined in `CLAUDE.md` and `docs/03-architecture.md`.
- RAGcore and ITWorx MCP Hub are external central services.
- n8n receives post-commit events through an outbox dispatcher.
- SQLAlchemy 2 declarative models cover the full domain model (`backend/app/models/`); enums are plain `String` columns validated at the Pydantic/service layer, not native PG enums (simpler migrations).
- `backend/requirements.lock` is compiled inside a `python:3.12-slim` container (matches the Dockerfile base image) via `pip-compile --extra dev`; regenerate the same way if `pyproject.toml` changes.
- Frontend dependencies pinned (no more `"latest"`); `package-lock.json` committed; Docker build uses `npm ci`.
- Demo auth is a lightweight HMAC-signed cookie (`app/core/security.py`), not a real password/JWT flow — matches "Demo role buttons create an authenticated session; they do not bypass authorization middleware." Two fixed demo users (`USR-OPS` operations_manager, `USR-EMP` rental_employee) are created by the seed loader, not from a CSV (no `users.csv` in `seed/`).
- Seed loader (`backend/app/seed_loader.py`) only supports `seed --reset` (always rebuilds); there is no incremental/idempotent-without-reset mode, since the acceptance criteria only require deterministic reset, not partial import.
- `DataQualityIssue.entity_ref`/`related_ref` from the CSVs are resolved to `entity_type`/`entity_id` (UUID) at load time per the domain model; the original human-readable refs are kept in `evidence_json` (`entity_ref`, `related_refs`) since the API and UI need them and re-resolving UUID→public_ref on every read would be wasteful.
- `backend/app/core/config.py` added `app_secret`, `session_cookie_name`, `session_ttl_seconds`, `seed_dir` (`/app/seed` in-container), `cors_allow_origins` (comma-separated string, not a list — simpler with pydantic-settings env parsing), `demo_today` (drives the dashboard's "Today" section against the deterministic anchor date, default `2026-08-01`).
- `compose.yaml` api build context changed from `./backend` to repo root with `dockerfile: backend/Dockerfile`, so the image can `COPY seed ./seed` (seed CSVs are outside `backend/`).
- Frontend: added `react-router-dom@7.18.2` (bumped from 6.x to clear two real advisories — open redirect + arbitrary constructor injection in v6). One residual `npm audit` finding (RSC-mode CSRF, GHSA-qwww-vcr4-c8h2) does not apply — this SPA never uses React Router's RSC/SSR mode.
- Nav/pages built so far: Dashboard, Vehicles (list+detail with tabs), Bookings (list+detail), Audit. Data Quality, Knowledge and Automation nav items are intentionally omitted until M3/M5/M4 build the pages behind them — CLAUDE.md forbids dead routes/placeholders.
- Return workflow (`app/services/returns.py`): the spec's "validate submitted reading against booking start reading" step was dropped as a hard rejection. For the seeded S1 scenario, a booking's `start_odometer_km` can already equal the vehicle's canonical odometer, so any regression-testing value would also be below the booking start, making a hard floor there indistinguishable from — and in conflict with — the documented soft-regression path. Only one odometer check exists now: submitted vs. the vehicle's *canonical* odometer (`vehicle.odometer_km`), matching the domain-model invariant verbatim ("a return with a lower submitted reading is recorded as an inspection and issue, while canonical odometer remains unchanged").
- Idempotency: new `idempotency_records` table (migration `e7b08389f47f`), unique on `idempotency_key`, keyed to `booking_id`. Same key + same booking replays the stored response; same key + different booking → 409 `IDEMPOTENCY_KEY_REUSED`; different key on an already-returned booking → 409 `INVALID_BOOKING_STATE`. Concurrency is enforced by `SELECT ... FOR UPDATE` on the booking row (re-checked for the idempotency record immediately after acquiring the lock, as a safety net for two simultaneous identical-key requests racing the pre-lock check).
- `seed_loader.clear_all()` must delete `idempotency_records` before `bookings` (FK) — easy to forget when adding new booking-referencing tables; the ordering list at the top of `seed_loader.py` is the single place to update.
- Inspection `public_ref` is assigned as `INSP-{count+1:04d}` from a live count query (not gap-safe, fine for a PoC single-writer demo, would need a sequence for real concurrency-safe numbering).
- Found and fixed during browser verification (not caught by pytest, since it's a UI-only defect): `ReturnForm` originally held its own `result` state and was conditionally rendered only when `booking.status === "active"`; once the return succeeded the booking flipped to `returned` and React unmounted the form before the user ever saw the result panel. Fixed by lifting the result into `BookingDetail` (`ReturnResultPanel` is now a sibling, not nested in `ReturnForm`). Also found: `OutboxEvent.event_id`'s Python-side `default=uuid.uuid4` on the mapped_column only applies at flush/commit time, so reading `event.event_id` before `db.commit()` returned `None` (rendered as the literal string "None" in the result panel); fixed by assigning `event_id=uuid.uuid4()` explicitly at construction. Lesson: SQLAlchemy column `default=` callables are not available on the in-memory Python object until flush — never rely on the generated value for a same-transaction response body without an explicit `db.flush()` or an explicit Python-side assignment.
- Operational note for this environment: `docker compose run --rm api ...` (used for tests/lint) only starts a throwaway one-off container — it does **not** update the long-running `api`/`web` service containers. After any code change meant to be verified live (browser, curl), `docker compose up -d --build <service>` is required, not just `docker compose build`.

## Completed evidence

### M0 — Reproducible foundation
- Added `backend/app/core/db.py` (engine/session), `backend/app/models/*` (User, Customer, Vehicle, Booking, Inspection, MaintenanceRecord, DataQualityIssue, OutboxEvent, AuditEvent), Alembic config (`backend/alembic.ini`, `backend/alembic/env.py`) and initial migration `backend/alembic/versions/c9498525abb5_initial_schema.py`.
- Commands run and verified from this checkout:
  - `docker compose build api` — OK
  - `docker compose run --rm api alembic upgrade head` — applied cleanly to empty DB, created 9 tables + `alembic_version`.
  - `docker compose run --rm api pytest -q` — 1 passed.
  - `docker compose run --rm api ruff check .` — All checks passed (added `extend-exclude = ["alembic/versions"]` to `backend/pyproject.toml` for autogenerated migration line length).
  - `docker compose up -d --build` — all 4 services healthy: `curl http://localhost:8128/health` → `{"status":"ok",...}`; `curl -o /dev/null -w "%{http_code}" http://localhost:1228/` → 200; `curl http://localhost:5678/healthz` → 200.
- Fixed a real scaffold bug: `frontend/src/App.tsx` used `import.meta.env` without a `vite/client` types reference, which broke `npm run build` in Docker (works fine under plain `vite dev` because Vite injects the global at dev-time but `tsc -b` still type-checks it). Added `frontend/src/vite-env.d.ts`.
- `make` is not installed in this Windows/git-bash shell — validated the underlying `docker compose ...` commands directly instead (Makefile targets are thin wrappers around them and are correct as written for a Linux/CI shell or WSL).
- Known accepted gap: `npm audit` reports 1 moderate/1 high transitive `esbuild` advisory (dev-server-only, fixed only by a Vite 8 major bump); left as-is for the PoC, noted here rather than silently upgrading a major version.

### M1 — Operational core
- Backend additions: `app/core/security.py` (HMAC-signed session cookies), `app/api/deps.py` (`get_current_user`, `require_operations_manager`), `app/core/errors.py` (`AppError` + the documented `{"error": {...}}` shape wired as a FastAPI exception handler for both `AppError` and `HTTPException`), `app/seed_loader.py`, `app/cli.py` (`python -m app.cli seed --reset`), `app/services/audit.py`, `app/schemas.py`, routers under `app/api/routers/` (`demo`, `dashboard`, `vehicles`, `bookings`, `audit`).
- Frontend additions: React Router-based app shell (`src/App.tsx`, `src/components/Layout.tsx`, `src/components/RequireAuth.tsx`), `AuthContext`, typed `api` client (`src/api/client.ts`, `src/api/types.ts`), pages `Login`, `Dashboard`, `Vehicles`/`VehicleDetail`, `Bookings`/`BookingDetail`, `Audit`. Full responsive stylesheet (`src/styles.css`) covering nav collapse and table→card layout under 700px, visible focus states, no hover-only actions.
- Commands run and verified from this checkout (container rebuilt each time to pick up code changes):
  - `docker compose run --rm api pytest -q` — **19 passed** (new: `test_seed.py`, `test_auth.py`, `test_dashboard.py`, `test_vehicles.py`, `test_bookings.py`, `test_audit.py`; tests seed the real Postgres via `reset_and_seed` in a session fixture, then exercise the FastAPI app through `TestClient`, not mocks).
  - `docker compose run --rm api ruff check .` — All checks passed (added `ignore = ["B008"]` — FastAPI's `Depends()`-as-default is idiomatic, not a real bug).
  - `npm run build` (local, Node 24) — clean `tsc -b && vite build`.
  - `docker compose up -d --build` then `docker compose exec api python -m app.cli seed --reset` — counts: `users:2 customers:180 vehicles:50 bookings:246 inspections:75 maintenance:40 data_quality_issues:15 workflow_runs:20`.
  - `curl` end-to-end: `POST /api/v1/demo/login` sets cookie and returns the user; unauthenticated `GET /api/v1/dashboard` → 401 with the documented error shape; authenticated dashboard/vehicle-detail return real seeded data (verified metrics `available:21 rented:11 cleaning:6 maintenance:5 blocked:7`, matching the 50 seeded vehicles).
  - Browser smoke test (Chrome via MCP) at desktop width: login page → Operations Manager login → Dashboard (metrics + attention items + today + recent automation all populated) → Vehicle detail `MO-016` (tabs render, "Needs attention" badge correct — it's `DQ-DEMO-OVERLAP`/`DQ-DEMO-STATUS`) → Booking detail `BK-DEMO-RETURN` (matches S1 scenario: vehicle `MO-024`, status `active`, start odometer `53610`). Responsive CSS (`@media max-width:700px`) was written and code-reviewed but the automated resize during this session didn't visibly reflect in the captured screenshot (likely a screenshot-timing quirk of the browser tool, not necessarily a real bug) — **treat the ≤360px layout as visually unverified** and re-check with a real device/DevTools emulation before final acceptance (M7).
- Known accepted gap carried over from M0: `npm audit` residual `esbuild`/Vite-8 dev-server-only advisory.

### M2 — Vehicle return vertical slice
- Backend additions: `app/models/idempotency.py` (`IdempotencyRecord`), migration `e7b08389f47f_idempotency_records`, `app/services/returns.py` (`register_vehicle_return` — full transaction: row locks, idempotency replay, inspection, canonical-odometer update or regression issue, vehicle status derivation, two audit events, `vehicle.returned.v1` outbox event matching `contracts/events.schema.json`, next-booking-risk lookup), `POST /api/v1/bookings/{public_ref}/return` wired in `app/api/routers/bookings.py` with required `Idempotency-Key` header.
- Frontend additions: `components/ReturnForm.tsx` (form + `ReturnResultPanel`), wired into `pages/BookingDetail.tsx` (shown only when `booking.status === "active"`; result persists via lifted state after the booking flips to `returned`).
- Commands run and verified from this checkout:
  - `docker compose run --rm api pytest -q` — **26 passed**, including `tests/test_return.py` (success/canonical-update, S1 regression scenario by name, damage→blocked, idempotent replay, reject-already-returned, missing-header validation, and a real multi-threaded concurrent-submission test against Postgres asserting exactly 1×201 + 2×409).
  - `docker compose run --rm api ruff check .` — All checks passed.
  - `npm run build` — clean.
  - `docker compose up -d --build` (all services) then `docker compose exec api python -m app.cli seed --reset`, then a full browser run of the S1 demo scenario against `BK-DEMO-RETURN`/`MO-024`: submitted 53000 km (below canonical 54820) → result panel showed `INSP-0076`, `resulting_vehicle_status: maintenance` (correctly derived, since canonical 54820 ≥ `next_service_km` 40000), `DQ-RET-0076` created, automation event queued with a real UUID, "no upcoming booking" risk; vehicle detail page confirmed odometer unchanged at 54,820 km and a "Needs attention" badge.
  - Both real defects listed above (form disappearing before showing its result; `event_id` reading as `None`) were **found via the browser run, not by pytest** — the test suite asserted on API response shape/values, not on what the UI actually rendered after a status transition. Worth remembering for M3+: UI state-after-mutation bugs need a browser check, not just API tests.

### M3 — Data Quality Workbench
- `app/services/data_quality.py`: `run_scan()` implements all five rules and is called automatically at the end of `seed_loader.reset_and_seed()` (after `db.commit()` of the base seed), plus exposed as `POST /api/v1/data-quality/scan` (Operations Manager only). Idempotency is simplified from the doc's literal `(rule_type, entity_type, entity_id, evidence fingerprint)` to just `(rule_type, entity_type, entity_id)` while an issue is open — see rationale below.
- Router `app/api/routers/data_quality.py`: `GET /issues` (filters status/rule_type/severity), `GET /issues/{ref}` (adds `entity_snapshot`/`related_snapshots` for the UI), `POST /issues/{ref}/defer`, `/reject`, `/merge-customers` (Operations Manager only — enforced via `require_operations_manager`), `POST /scan`.
- **Real bug found and fixed during this milestone, before any browser check**: the first cut of DQ-03 (odometer regression) compared every historical *returned* booking's `end_odometer_km` against the vehicle's *current* `odometer_km`. Since the seed generator assigns `vehicle.odometer_km` independently of booking history (see `seed/generate_seed.py`), this is true for nearly every historical booking by construction (odometer is monotonically increasing over time, so all-but-the-latest reading is "below current") — it produced 51 false-positive issues out of 50 vehicles on first run. Fixed twice: first attempt (compare only the single most-recent booking against canonical) still produced the same problem because canonical itself is disconnected from booking history in this dataset; the working fix compares each vehicle's *own returned-booking sequence* against itself (each booking's end reading vs. the immediately preceding one, chronologically) — a self-consistency check that doesn't depend on the unrelated `vehicle.odometer_km` field at all. Final deterministic seed+scan totals: 15 CSV-seeded + 11 scan-discovered = **26** open/resolved `data_quality_issues` (breakdown: 14 vehicle_status_conflict, 5 missing_required_field, 3 possible_duplicate_customer, 3 odometer_regression, 1 booking_overlap). `tests/test_seed.py`'s exact-count assertion was updated from 15 to 26 accordingly — if the scan logic changes again, update that count.
- Idempotency simplification rationale: the doc's fingerprint-based key would make the scan blind to issues it structurally can't compute a matching fingerprint for against the CSV-seeded rows (which don't carry a fingerprint field), producing duplicate issues for the same real-world problem (e.g. a second `MO-016` overlap issue next to the seeded `DQ-DEMO-OVERLAP`). Using `(rule_type, entity_type, entity_id)` alone while open is a stricter, safe simplification: it can never falsely suppress an issue for a *different* entity, and per-entity there's realistically only one meaningful open issue of a given rule type at a time for this PoC's scope.
- Merge UI intentionally does **not** use `window.confirm()` — a native dialog blocks further automation/testing and isn't screen-reader-distinguishable from page content the same way a rendered `role="alertdialog"` panel is. Built an inline two-step confirm instead (`ReturnForm`-style pattern reused).
- `AttentionItem` gained an `issue_ref` field (dashboard now links attention items straight to `/data-quality/{issue_ref}` instead of only to vehicles); dashboard attention list capped at 8 items (was unbounded, would have shown up to 26 with the richer scan).
- Commands run and verified from this checkout:
  - `docker compose run --rm api pytest -q` — **35 passed** (new `tests/test_data_quality.py`: all five rule types present, scan idempotent on rerun, scan requires Operations Manager, S2/S4 issue-detail snapshots correct, defer→reject-on-closed 409, merge requires Operations Manager, merge rejects an unrelated survivor ref, full S2 merge scenario asserting rewiring + audit + replay-is-409).
  - `docker compose run --rm api ruff check .` — All checks passed.
  - `npm run build` — clean (had to fix two `possibly 'null'` TS errors from a closure-narrowing limitation — TS doesn't narrow `const` captured-by-closure across nested function boundaries when the value comes from an index/property expression; fixed by re-binding to explicitly-typed local consts right after the guard).
  - Full browser run: Data Quality list (26 open issues, filterable) → `DQ-DEMO-DUPLICATE` two-column compare (CUS-0012 vs CUS-0178, per-field diff highlighting only where they differ) → merge with inline confirm → issue flips to `resolved` → confirmed `customer_merged` audit event with correct actor/entity/correlation → `DQ-DEMO-OVERLAP` (non-duplicate type) renders evidence JSON + defer/reject, no dead compare UI shown for a rule type it doesn't apply to.

## Known blockers

None. External service credentials may be absent; use the documented demo/degraded providers.

## Exact next action

Start M4 (n8n automation): read `docs/11-n8n-integration.md`, `n8n/README.md`. Implement the outbox dispatcher (poll pending `outbox_events`, claim with `FOR UPDATE SKIP LOCKED`, POST to n8n's webhook with timeout + exponential backoff + small max-attempt cap, never hold a DB transaction open during the HTTP call), correct/import `n8n/mobilityops-return-processing.json` (credential + the callback route — M2's `vehicle.returned.v1` outbox payload shape is already `contracts/events.schema.json`-compliant, so the workflow should be able to consume it as-is), a narrow MobilityOps callback endpoint the workflow calls back into, an Automation nav item + UI (pending/succeeded/failed list, manual retry button, matches `docs/06-ui-ux.md`'s "Recent automation" concept already on the dashboard), and manual-retry wiring for `POST /api/v1/workflows/{event_id}/retry` per `docs/05-api-contract.md`. The seeded `workflow_runs.csv` already includes one deterministic `failed` row (`event_id 00000000-...-0020`, "Synthetic connection timeout to n8n") for the S5 demo scenario — dashboard/Automation UI must surface it and allow retry. Remember the compose `n8n` service is already up (`http://localhost:5678`, basic auth `admin`/`change-me` from `.env.example`) but the workflow itself hasn't been imported/activated yet this session.