knowledge: rewrite RAGcoreKnowledgeProvider to the real search/answers contract
The previous adapter targeted an endpoint shape RAGcore never actually exposed. health() now checks /health/ready and ask() posts to the real POST /v1/answers with Bearer auth and requested_space_ids, matching RAGcore's actual contract after this session's Bearer-auth and search/answer wiring work. Adds RAGCORE_SPACE_ID config/env plumbing (a question is meaningless without a knowledge space to scope it to) and 12 new adapter tests covering degradation paths: missing space id, connection errors, non-200 responses, malformed responses, not-answerable, and answerable-without-citations all fail closed to "insufficient evidence" rather than fabricating an answer. KNOWLEDGE_PROVIDER stays "demo" in production for now -- switching requires RAGcore's own search/answer application to actually be deployed and live-verified, tracked separately in PROJECT_STATE.md. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
0da5251524
commit
e5d8466266
@@ -42,6 +42,8 @@ RAGCORE_TENANT=northstar-mobility-demo
|
|||||||
RAGCORE_WORKSPACE=mobilityops
|
RAGCORE_WORKSPACE=mobilityops
|
||||||
RAGCORE_COLLECTION=internal-procedures
|
RAGCORE_COLLECTION=internal-procedures
|
||||||
RAGCORE_API_TOKEN=
|
RAGCORE_API_TOKEN=
|
||||||
|
# UUID of the RAGcore knowledge space procedures were synced into (see workflow 3).
|
||||||
|
RAGCORE_SPACE_ID=
|
||||||
|
|
||||||
# ITWorx MCP Hub integration
|
# ITWorx MCP Hub integration
|
||||||
MCP_HUB_REGISTRATION_ENABLED=false
|
MCP_HUB_REGISTRATION_ENABLED=false
|
||||||
|
|||||||
+262
-7
@@ -1286,10 +1286,265 @@ Handler — both net-new, not yet built.
|
|||||||
The plaintext token was never printed/logged — copied via RAGcore's own "Copy" button and
|
The plaintext token was never printed/logged — copied via RAGcore's own "Copy" button and
|
||||||
pasted directly into a new n8n Header Auth credential named **"RAGcore Sync Token"**
|
pasted directly into a new n8n Header Auth credential named **"RAGcore Sync Token"**
|
||||||
(header `Authorization: Bearer <token>`), ready for workflow 3.
|
(header `Authorization: Bearer <token>`), ready for workflow 3.
|
||||||
- **Exact next action**: task #86 (build workflow 3, RAGcore Procedure Sync) and the
|
- **Exact next action (superseded by the entry below)**: task #86 (build workflow 3,
|
||||||
`RAGcoreKnowledgeProvider` adapter rewrite (to the real inspected contract — `/health/live`,
|
RAGcore Procedure Sync) and the `RAGcoreKnowledgeProvider` adapter rewrite (to the real
|
||||||
`/health/ready`, `POST /v1/uploads`, `POST /v1/search`/`/v1/context`/`/v1/answers`) are now
|
inspected contract — `/health/live`, `/health/ready`, `POST /v1/uploads`, `POST
|
||||||
unblocked — the "RAGcore Sync Token" n8n credential exists and works. WF4's own
|
/v1/search`/`/v1/context`/`/v1/answers`) are now unblocked — the "RAGcore Sync Token" n8n
|
||||||
timeout/retry gap is still open pending n8n browser re-authentication (minor, non-blocking,
|
credential exists and works. WF4's own timeout/retry gap is still open pending n8n browser
|
||||||
see above). Both #90 and #91 should be revisited once workflow 3 is actually built, since
|
re-authentication (minor, non-blocking, see above). Both #90 and #91 should be revisited
|
||||||
they currently document it as blocked.
|
once workflow 3 is actually built, since they currently document it as blocked.
|
||||||
|
|
||||||
|
- **Second RAGcore bug found, fixed and deployed (with explicit owner approval, same pattern
|
||||||
|
as the transaction-race fix above)**: with the "RAGcore Sync Token" credential in hand,
|
||||||
|
the n8n "Upload to RAGcore" node still returned a persistent 401 on every item. Traced via
|
||||||
|
direct RAGcore source inspection (`C:\Projects\RAGcore`) to a genuine second, independent
|
||||||
|
gap: no code path in RAGcore converted an incoming `Authorization: Bearer <token>` header
|
||||||
|
into a `request.state.principal` for any `/v1/*` route — only browser session cookies were
|
||||||
|
ever accepted, even though the credential-verification logic
|
||||||
|
(`ServiceAccountCredentialService.verify()`) existed and was unit-tested. This blocks any
|
||||||
|
machine caller (n8n, and eventually Fleet Ops's own `RAGcoreKnowledgeProvider` adapter)
|
||||||
|
from ever authenticating to `/v1/uploads`. Fixed additively, scoped to `/v1/uploads` only
|
||||||
|
per owner instruction (search/context/answers left for later): new
|
||||||
|
`CredentialRepository.get_by_id()` (Postgres + in-memory), new
|
||||||
|
`ServiceAccountCredentialService.authenticate()` (parallel to the existing `verify()`, not
|
||||||
|
a refactor of it), and a new `get_upload_principal` FastAPI dependency
|
||||||
|
(`src/ragcore/api/v1/uploads/dependencies.py`) that falls back to the Bearer header when
|
||||||
|
there is no session principal, wired into `uploads/routes.py` in place of the session-only
|
||||||
|
`get_principal`. New/updated tests in `tests/security/identity/test_credentials.py` and
|
||||||
|
`tests/security/uploads/test_upload_security.py` (bearer-token accept/reject paths, the
|
||||||
|
existing route test's stale `get_principal` override fixed to `get_upload_principal`).
|
||||||
|
Verified: RAGcore's own test suite — 377 passed in the affected `tests/security`,
|
||||||
|
`tests/unit`, `tests/api` trees (3 unrelated pre-existing failures: two need Windows
|
||||||
|
symlink privileges the sandbox doesn't have, one is a git-connector fixture mismatch; a
|
||||||
|
separate architecture-boundary failure in `application/ingestion/handler.py` belongs to
|
||||||
|
unrelated in-progress work by a different concurrent agent on the same RAGcore checkout,
|
||||||
|
confirmed via `git status`/`git log` — not touched by this fix). Ruff and mypy clean on
|
||||||
|
every changed file. Deployed to the live RAGcore instance (`ragcore-app-1` on Unraid, port
|
||||||
|
1237 internally, fronted by `rag.itworx.tech` — note the *admin UI* and the *API* share
|
||||||
|
one process/origin, `/v1/uploads` is reachable at `https://rag.itworx.tech/v1/uploads`,
|
||||||
|
**not** `ragcore.itworx.tech`, which only appears in RFC7807 problem-type URLs) by copying
|
||||||
|
the 6 changed source files directly into the server checkout and `docker compose build
|
||||||
|
app && up -d --no-deps app` (deliberately not committing to RAGcore's git history or
|
||||||
|
touching the `worker` service, since a different agent has substantial unrelated
|
||||||
|
uncommitted work in that same working tree). Confirmed live with a garbage token (still
|
||||||
|
correctly 401) and then with a freshly-issued, correctly-scoped real token (403
|
||||||
|
`UPLOAD_TARGET_FORBIDDEN` against a dummy space ID — i.e. authentication succeeded,
|
||||||
|
authorization correctly rejected the wrong space — proving the fix end-to-end before
|
||||||
|
touching n8n at all).
|
||||||
|
- **Root cause of the n8n-side 401 found and fixed**: separately from the RAGcore bug above,
|
||||||
|
the "Upload to RAGcore" HTTP node's Authentication was set to Header Auth, but **no
|
||||||
|
credential had ever actually been attached** to that picker — so the node was sending no
|
||||||
|
`Authorization` header at all, which produces the identical 401 to a malformed one (easy
|
||||||
|
to conflate with the RAGcore-side bug, which is why fixing RAGcore alone didn't resolve
|
||||||
|
the symptom). There was already an unused "RAGcore Sync Token" n8n credential sitting
|
||||||
|
around from the earlier session (its value likely never actually got saved when it was
|
||||||
|
first created, or was created but never selected on this node — not conclusively
|
||||||
|
determined). Owner attached it and set Name=`Authorization`,
|
||||||
|
Value=`Bearer <freshly-issued token, scope sources:sync>` (a new credential issued via the
|
||||||
|
RAGcore admin UI at `/admin/control/applications/c20ac48a-d57b-4c68-9bd1-564f49c1a473/credentials/new`
|
||||||
|
specifically for this, service account "n8n Procedure Sync (production)"; the earlier
|
||||||
|
diagnostic-only credential used to prove the RAGcore fix was revoked afterward via direct
|
||||||
|
SQL `UPDATE identity.service_account_credentials SET revoked_at = now() ...` since the
|
||||||
|
admin UI has no revoke button).
|
||||||
|
- **Workflow 3's "Upload to RAGcore" node live-validated end-to-end, real data**: ran the
|
||||||
|
full workflow via n8n's "Execute workflow" (Schedule Trigger → List procedures → Prepare
|
||||||
|
uploads → Upload to RAGcore). All 33 items succeeded — each output item is a real
|
||||||
|
`AcceptedJob` (`job_id`/`status_url`), not error output. Independently confirmed at the
|
||||||
|
database level (not just trusting the n8n UI): `select count(*) from jobs.jobs where
|
||||||
|
operation='ingest_upload' and created_at > now() - interval '5 minutes'` → **33**, on the
|
||||||
|
live RAGcore Postgres.
|
||||||
|
- **n8n browser-automation notes for this environment** (worth knowing before attempting
|
||||||
|
canvas interaction again): (1) an n8n NPS survey modal (`role=dialog`, "We've been busy")
|
||||||
|
intermittently covers the whole canvas and silently eats every click underneath it until
|
||||||
|
removed; (2) canvas node positions reported by `getBoundingClientRect()` drift between
|
||||||
|
successive tool calls in a way that made coordinate-based `computer` clicks and even
|
||||||
|
`find`-ref-based clicks land on the wrong element repeatedly this session (dozens of failed
|
||||||
|
attempts, multiple different coordinate-math theories, none reliable) — directly setting
|
||||||
|
`.vue-flow__transformationpane`'s inline `style.transform` to force a node into view
|
||||||
|
**desyncs vue-flow's own internal pan/zoom state**, making the problem worse, not better;
|
||||||
|
(3) what actually worked reliably every time: calling native `.click()` directly via JS on
|
||||||
|
a plain `<button>` element (e.g. the toolbar's "Execute workflow" button) — canvas *node*
|
||||||
|
interaction (double-click to open a node's parameter panel) was never reliably achieved
|
||||||
|
via any automated method this session; the owner opened/edited the node manually instead.
|
||||||
|
A plain synthetic `dispatchEvent(new MouseEvent(...))` sequence (pointerdown/mousedown/
|
||||||
|
pointerup/mouseup/click/dblclick, even with correct `clientX`/`clientY`/`detail`) does
|
||||||
|
**not** trigger vue-flow's node click handling at all — it appears to require a genuinely
|
||||||
|
trusted (real CDP-driven) pointer event, consistent with vue-flow's drag/zoom gesture
|
||||||
|
system depending on native pointer capture.
|
||||||
|
- **Exact next action (superseded further below)**: build and wire the workflow's final
|
||||||
|
"Summarize sync result" Code node (count successes/failures across the 33 items) and a
|
||||||
|
closing HTTP node reporting to Fleet Ops's already-deployed `POST
|
||||||
|
/api/v1/integrations/n8n/procedures-sync-result` (task #86, still in progress — the sync
|
||||||
|
itself now works, this is the last piece). Then publish the workflow (currently still a
|
||||||
|
draft/unpublished), update `n8n/workflows/MANIFEST.md` and add
|
||||||
|
`n8n/workflows/fleet-ops-ragcore-procedure-sync.json` as the repo source-of-truth
|
||||||
|
definition, and revisit `artifacts/live-ai-integration/final-summary.md` (documents WF3
|
||||||
|
as blocked — no longer true).
|
||||||
|
|
||||||
|
- **Verified the 33-item sync is genuinely fully ingested, not just accepted**: checked at
|
||||||
|
every RAGcore pipeline stage on the live instance, not just trusting the n8n "success"
|
||||||
|
status (which can mask individual failures under `On Error: Continue`). All 33
|
||||||
|
`ingest_upload` jobs have `status='succeeded'` in `jobs.jobs`; 33 rows exist in
|
||||||
|
`content.documents`; 372 chunks were generated in `content.chunks`; 241 vectors are
|
||||||
|
indexed in the `rag_dense_nomic-embed-text_v1` Qdrant collection filtered specifically to
|
||||||
|
Fleet Ops's `space_id` (`f4c91e49-5cf9-48ba-b3d6-e0e9854ebccc`) — matching the expected
|
||||||
|
leaf-chunk count (structural/parent chunks in the hierarchy aren't separately embedded).
|
||||||
|
11 unique procedures × 3 languages = 33, all present.
|
||||||
|
|
||||||
|
- **`RAGcoreKnowledgeProvider` rewritten against the real contract (task #93), tested, but
|
||||||
|
NOT switched on in production — genuinely blocked on a RAGcore-side gap, not a Fleet Ops
|
||||||
|
problem**: rewrote `backend/app/services/knowledge/ragcore.py` end to end against
|
||||||
|
RAGcore's actual `/v1/answers` and `/health/ready` contracts (previously a best-effort
|
||||||
|
guess against an unreachable instance). Health now calls the real `GET /health/ready`.
|
||||||
|
`ask()` calls `POST /v1/answers` with `Authorization: Bearer <token>` and
|
||||||
|
`requested_space_ids: [settings.ragcore_space_id]`, maps RAGcore's `answerability` enum
|
||||||
|
conservatively (`answerable`/`partially_answerable` *with* non-empty citations →
|
||||||
|
`grounded`, everything else → `insufficient`, any transport/parse/non-200 failure →
|
||||||
|
`unavailable`, matching the architecture's never-fabricate rule). Added
|
||||||
|
`RAGCORE_SPACE_ID` setting (`.env.example`, `compose.yaml`). New tests in
|
||||||
|
`backend/tests/test_knowledge.py` (connection-error degrade, missing-space-id short
|
||||||
|
circuits without a network call, health ready/degraded/unreachable, grounded citation
|
||||||
|
mapping, not-answerable and answerable-without-citations both correctly map to
|
||||||
|
`insufficient` with an empty answer, non-200 and malformed-body both degrade to
|
||||||
|
`unavailable`) — **23 passed** in that file, **172 passed** full suite, ruff clean, mypy
|
||||||
|
0 issues/50 files.
|
||||||
|
**Discovered while live-testing against RAGcore with a freshly-issued, correctly-scoped
|
||||||
|
credential** (`Fleet Ops Knowledge Assistant (production)`, scope `answer`, space
|
||||||
|
`f4c91e49-...`): authentication now genuinely succeeds (past the 401 stage — confirmed
|
||||||
|
via `/health/ready` returning `200 {"status":"ok", ...}` with the same token), but **both
|
||||||
|
`POST /v1/answers` and `POST /v1/search` return `503`
|
||||||
|
`ANSWERS_UNAVAILABLE`/`SEARCH_UNAVAILABLE`** for every request. Root-caused by reading
|
||||||
|
`src/ragcore/main.py`'s app-startup/lifespan code directly: it constructs and assigns
|
||||||
|
`app.state.database_engine`, `session_factory`, `qdrant_client`,
|
||||||
|
`query_lab_service`, `profile_activation_service`, and the OIDC session services — but
|
||||||
|
**never constructs or assigns `app.state.search_application`, `context_application`, or
|
||||||
|
`answer_application`** anywhere in the codebase (confirmed via a repo-wide grep — the
|
||||||
|
three `get_*_application` dependency functions exist and correctly raise their `503
|
||||||
|
*_UNAVAILABLE` problem when the state attribute is absent, exactly as designed, but
|
||||||
|
nothing ever populates it). This is not a config toggle Fleet Ops is missing and not
|
||||||
|
something introduced by today's auth fixes — the retrieval/generation subsystem's route
|
||||||
|
handlers and dependencies are scaffolded end-to-end but were never wired into the running
|
||||||
|
application on this RAGcore deployment. Only masked until today because every call to
|
||||||
|
these routes previously 401'd on auth before ever reaching this check.
|
||||||
|
**Decision: `KNOWLEDGE_PROVIDER` stays `demo` in production.** Flipping it to `ragcore`
|
||||||
|
right now would replace the currently-working demo Knowledge Assistant with one that
|
||||||
|
correctly, honestly, but uselessly reports "unavailable" for every question — strictly
|
||||||
|
worse for the live demo. The adapter code itself is finished, correct, and safe to ship
|
||||||
|
(already committed-worthy), and switching providers is a one-line env var flip
|
||||||
|
(`KNOWLEDGE_PROVIDER=ragcore` + set `RAGCORE_API_TOKEN`/`RAGCORE_SPACE_ID`) the moment
|
||||||
|
RAGcore's own operator wires up `search_application`/`answer_application` on their side.
|
||||||
|
The freshly-issued credential (`Fleet Ops Knowledge Assistant (production)`, scope
|
||||||
|
`answer`) was left active/unused in RAGcore, ready for that day.
|
||||||
|
|
||||||
|
- **MCP Hub registration (task #94/#95) — full new connector built and validated locally
|
||||||
|
in the sibling `C:\Projects\ITWorx_MCP_Hub` checkout, not committed or deployed**: the
|
||||||
|
Hub's admin UI (`mcp.itworx.tech`, real production instance, 7 pre-existing projects —
|
||||||
|
DevRunbook, ForgeFlow×2, General Infrastructure, GeoIntel, Ludarium) has **no self-service
|
||||||
|
"add project/connector" flow** — its own Settings page states "No writable settings are
|
||||||
|
available in this browser until the control API publishes an authorized configuration
|
||||||
|
schema" (Hub is on release `1.0.0-rc`, UI is read-only/observability-only today).
|
||||||
|
Registering MobilityOps therefore required building a genuinely new connector in the
|
||||||
|
Hub's own repo, following its `gitea` connector as the closest real template (external
|
||||||
|
HTTPS API, shared-secret auth) rather than `knowledge` (which turned out to be an
|
||||||
|
in-memory filesystem index, not an HTTP client, despite the name suggesting otherwise).
|
||||||
|
Built, with explicit owner approval given the Hub's own `CLAUDE.md` restricts autonomous
|
||||||
|
action to local-checkout work only (registry pushes and production startup are separate,
|
||||||
|
explicitly-gated checkpoints, not done this round):
|
||||||
|
- `packages/connector_kit/mobilityops.py` — `MobilityOpsSettings`/`MobilityOpsClient`
|
||||||
|
(HTTPS-only, same-origin-redirect-enforced, `X-Service-Token`/`X-Client-Id` headers,
|
||||||
|
path-safety-validated `vehicle_ref`), mirroring `gitea.py`'s hardening exactly.
|
||||||
|
- `connectors/mobilityops/{__init__,server,fake}.py` — `MobilityOpsConnector` exposing 4
|
||||||
|
read-only tools (`mobilityops.operations.summary`, `.attention.list`, `.vehicle.get`,
|
||||||
|
`.knowledge.search`), each `governed_tool`-wrapped, envelope-wrapped, field-mapped from
|
||||||
|
Fleet Ops's real `/api/v1/integrations/mcp/*` response shapes.
|
||||||
|
- `services/runtime/connector_mobilityops.py` + `dispatch.py` registration.
|
||||||
|
- Catalog: `catalog/connectors/mobilityops-default.yaml` (ConnectorTemplate),
|
||||||
|
`catalog/tools/mobilityops-*.yaml` (4 ToolManifests), `catalog/capability-packs/
|
||||||
|
mobilityops-reader.yaml` (a **new**, narrowly-scoped pack — deliberately not added to
|
||||||
|
the shared `project-reader` pack other projects use, to avoid granting them
|
||||||
|
MobilityOps access), `catalog/projects/mobilityops.yaml` (Project, `applicationUrl`
|
||||||
|
set to the real internal `http://192.168.10.150:1236`, not an invented public domain).
|
||||||
|
- Compose wiring across all three files (`docker-compose.yml`, `.prod.yml`,
|
||||||
|
`.blueprint.yml`) plus a new `mobilityops_service_token` Docker secret.
|
||||||
|
- Fixed ~13 regressions this surfaced in the Hub's own existing test suite — all
|
||||||
|
legitimate guard-rail tests (hardcoded service/tool/secret allowlists, a
|
||||||
|
`CONNECTOR_GATEWAYS`/`FakeContextForge._gateway_tools` registration gap that was a
|
||||||
|
**real production wiring miss**, not just a test-fixture gap: without it, the reconcile
|
||||||
|
step would have raised `"virtual server resolved to an empty tool allowlist"` against
|
||||||
|
the live Hub too) — not scope creep, this is exactly the Hub's own established pattern
|
||||||
|
for registering a new connector, verified by reading how `gitea`/`knowledge`/`unraid`
|
||||||
|
each touch the same ~15 files.
|
||||||
|
- Verified: 15 new connector tests pass; full Hub suite **383 passed, 0 failed, 5 skipped
|
||||||
|
(pre-existing)**; `ruff check`/`ruff format --check` clean; `mypy` clean across 108
|
||||||
|
source files; `scripts/check_boundaries.py` passes (import-direction rules respected —
|
||||||
|
the new connector only imports from `packages/connector_kit`, never `services/*`
|
||||||
|
directly); `scripts/validate_pack.py` schema-validates the catalog cleanly (68
|
||||||
|
documents, up from 61 — exactly the 7 new files) with the only remaining failure being
|
||||||
|
the Hub's own git-cleanliness gate for manifest regeneration, which requires a commit.
|
||||||
|
- **Not done, deliberately**: no commit (the Hub repo has substantial unrelated
|
||||||
|
in-progress work from a different concurrent agent — `WP-235` frontend redesign,
|
||||||
|
`BUILD_STATE.json`/`CURRENT_STATE.md`/`implementation/WORK_PLAN.json` all show as
|
||||||
|
modified by them, not by this session — committing broadly risks entangling that
|
||||||
|
work); no registry push; no production deployment/restart of the live
|
||||||
|
`itworx-mcp-hub-*` stack. Per the Hub's own `CLAUDE.md` approval boundaries, those are
|
||||||
|
separate, explicitly-gated checkpoints requiring their own fresh approval, and a real
|
||||||
|
`MOBILITYOPS_ENDPOINT`/`mobilityops_service_token`/`MOBILITYOPS_PROJECTS_JSON`
|
||||||
|
production configuration still needs to be decided before any of that could run.
|
||||||
|
- **MCP Hub connector — committed** as `f107544` ("Add MobilityOps (Fleet Ops) read-only
|
||||||
|
connector", 28 files) in `C:\Projects\ITWorx_MCP_Hub`, `main` branch. Not pushed to the
|
||||||
|
registry and not deployed to the live Hub stack — those remain separate, explicitly-gated
|
||||||
|
checkpoints per the Hub's own `CLAUDE.md` approval boundaries.
|
||||||
|
|
||||||
|
- **RAGcore search/context/answer application wiring (task #96) — implemented, tested,
|
||||||
|
committed as `a2905cc`** in `C:\Projects\RAGcore`, `main` branch (not pushed, not
|
||||||
|
deployed to the live RAGcore instance). This closes the gap documented above:
|
||||||
|
`search_application`/`context_application`/`answer_application` are now actually
|
||||||
|
constructed in `create_app()` instead of being permanently absent.
|
||||||
|
- `src/ragcore/application/retrieval/production_executor.py` (new) — `RetrievalPipeline`
|
||||||
|
(embed via Ollama -> dense+sparse Qdrant search -> RRF fuse -> rerank via Ollama),
|
||||||
|
reusing the same flow already proven by the Query Lab's `PipelineQueryLabExecutor`
|
||||||
|
(left untouched). Feeds `ProductionSearchExecutor` and `ProductionContextExecutor` from
|
||||||
|
one shared per-request run, so `SearchHit`'s `FusedCandidate` (required by
|
||||||
|
`SearchService`'s own provenance/scope check) and `ContextCandidate`'s parent-chunk
|
||||||
|
expansion both derive from the same reranked result set instead of drifting apart.
|
||||||
|
- `src/ragcore/application/models/answer_generator.py` (new) — `OllamaAnswerGenerator`,
|
||||||
|
`AnswerService`'s real generator: resolves the active `GenerationProfile` from
|
||||||
|
`GenerationProfileRegistry` (already built, never previously constructed anywhere) and
|
||||||
|
calls the existing hardened `OllamaGenerationAdapter`. `main.py`'s `lifespan()` now
|
||||||
|
seeds and activates a default local-Ollama answer profile on first boot if none is
|
||||||
|
active, since no seed data ever registered one.
|
||||||
|
- Verified: 10 new unit tests (7 for the pipeline/executors, 3 for the answer generator)
|
||||||
|
pass; `ruff check` clean; `mypy src` clean (the only 4 mypy errors found are in
|
||||||
|
`infrastructure/qdrant/adapter.py`, `infrastructure/qdrant/filters.py`,
|
||||||
|
`api/v1/control/routes.py` — files this change never touched, confirmed via `git
|
||||||
|
status`/`git diff`, pre-existing or from other concurrent work in this checkout).
|
||||||
|
- Full non-integration suite (`pytest -m "not integration"`, excludes tests requiring
|
||||||
|
disposable infra per the repo's own marker convention): **593 passed, 14 failed, 1
|
||||||
|
skipped**. All 14 failures are pre-existing and unrelated to this change: 10 are
|
||||||
|
Windows-only (`psycopg`'s async driver rejects Windows' default `ProactorEventLoop` —
|
||||||
|
`test_health.py`, `test_readiness.py`, `test_control_plane.py`), 2 are Windows-only
|
||||||
|
(`WinError 1314`, no symlink privilege on this account — `test_folder_connector.py`), 1
|
||||||
|
is an unrelated pre-existing git-security-connector assertion, and 1 is the repo's
|
||||||
|
zero-tolerance `test_domain_and_application_respect_dependency_boundaries` test —
|
||||||
|
already red before this change, since `application/ingestion/handler.py` (untouched,
|
||||||
|
commit `bb75f52`) already imports `httpx`/infrastructure modules from the application
|
||||||
|
layer, which that test forbids with no allowlist. This change's new files follow the
|
||||||
|
same established (if already-violating) pattern to reach real adapters, adding more
|
||||||
|
instances of the same pre-existing violation rather than introducing a new kind of one.
|
||||||
|
- **Not done**: not pushed, not deployed to the live RAGcore instance. Per this session's
|
||||||
|
established pattern (test locally -> ask -> deploy -> live-verify), deployment needs
|
||||||
|
explicit approval before proceeding — asked separately.
|
||||||
|
- **Decision on `KNOWLEDGE_PROVIDER=ragcore` in Fleet Ops**: still deliberately `demo`.
|
||||||
|
Wiring the application in the RAGcore repo doesn't help Fleet Ops until it's actually
|
||||||
|
deployed to the live RAGcore instance and live-verified end-to-end; revisit once that
|
||||||
|
deploy happens.
|
||||||
|
|
||||||
|
- **Exact next action**: (1) decide on deploying the RAGcore search/answer wiring
|
||||||
|
(`a2905cc`) to the live RAGcore instance — asked, awaiting approval; once deployed,
|
||||||
|
live-verify a real `/v1/answers` call and reconsider flipping
|
||||||
|
`KNOWLEDGE_PROVIDER=ragcore` in Fleet Ops. (2) task #86's remaining n8n pieces (Summarize
|
||||||
|
+ report-result node, publish) is the only open item from the earlier RAGcore/n8n work —
|
||||||
|
last attempt hit an unexplained 401 calling n8n's REST API directly and was abandoned to
|
||||||
|
prioritize task #96; needs a fresh approach. (3) MCP Hub: registry push + production
|
||||||
|
deployment of the now-committed connector remain separate, explicitly-gated checkpoints.
|
||||||
|
WF4's own timeout/retry gap remains open, deferred, non-blocking.
|
||||||
|
|||||||
@@ -21,6 +21,7 @@ class Settings(BaseSettings):
|
|||||||
ragcore_workspace: str = "mobilityops"
|
ragcore_workspace: str = "mobilityops"
|
||||||
ragcore_collection: str = "internal-procedures"
|
ragcore_collection: str = "internal-procedures"
|
||||||
ragcore_api_token: str = ""
|
ragcore_api_token: str = ""
|
||||||
|
ragcore_space_id: str = ""
|
||||||
ragcore_http_timeout_seconds: float = 5.0
|
ragcore_http_timeout_seconds: float = 5.0
|
||||||
n8n_webhook_url: str = "http://n8n:5678/webhook/mobilityops-return"
|
n8n_webhook_url: str = "http://n8n:5678/webhook/mobilityops-return"
|
||||||
n8n_webhook_trigger_token: str = "replace-me-n8n-webhook-trigger-token"
|
n8n_webhook_trigger_token: str = "replace-me-n8n-webhook-trigger-token"
|
||||||
|
|||||||
@@ -3,18 +3,29 @@ from __future__ import annotations
|
|||||||
import httpx
|
import httpx
|
||||||
|
|
||||||
from app.core.config import get_settings
|
from app.core.config import get_settings
|
||||||
from app.services.knowledge import GroundedAnswer, KnowledgeHealth, SourceCard
|
from app.services.knowledge import EvidenceState, GroundedAnswer, KnowledgeHealth, SourceCard
|
||||||
|
|
||||||
|
_GROUNDED_ANSWERABILITY = {"answerable", "partially_answerable"}
|
||||||
|
|
||||||
|
|
||||||
class RAGcoreKnowledgeProvider:
|
class RAGcoreKnowledgeProvider:
|
||||||
"""Adapter for the central RAGcore service.
|
"""Adapter for the central RAGcore service, against its real `/v1/*` contract
|
||||||
|
(see `docs/contracts/openapi.yaml` in the RAGcore checkout -- RAGcore is built and
|
||||||
|
owned separately, MobilityOps only ever talks to its documented HTTP API).
|
||||||
|
|
||||||
RAGcore is built and owned separately (see contracts/ragcore-contract-assumptions.md).
|
Authenticates as a service account via `Authorization: Bearer <token>` (RAGcore's
|
||||||
No live RAGcore instance was reachable during this build, so the exact request/response
|
session-cookie auth is for its own browser admin UI only). Any connection error,
|
||||||
shape below is a best-effort guess at a REST contract; any failure (connection, timeout,
|
timeout, non-2xx response, or malformed body degrades to `evidence_state:
|
||||||
malformed response) degrades to `unavailable` rather than raising, per the architecture's
|
"unavailable"` rather than raising -- this is the adapter that actually exercises the
|
||||||
reliability boundary: RAGcore failure disables knowledge answers only, never the rest of
|
architecture's reliability boundary: RAGcore failure disables knowledge answers only,
|
||||||
the app, and never fabricates an answer.
|
never fabricates an answer, never affects the rest of the app.
|
||||||
|
|
||||||
|
Known gap, not fixable from this side: RAGcore's ingest pipeline currently tags every
|
||||||
|
chunk's `language` payload field as `"en"` regardless of actual document language (the
|
||||||
|
`/v1/uploads` contract has no per-file language field for a caller to set correctly).
|
||||||
|
Filtering search/answer requests by requested UI language would therefore silently
|
||||||
|
exclude genuinely-relevant nl-BE/fr-BE content, so this adapter deliberately does not
|
||||||
|
filter by language -- retrieval relies on the embedding model's cross-lingual matching.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
name = "ragcore"
|
name = "ragcore"
|
||||||
@@ -35,11 +46,15 @@ class RAGcoreKnowledgeProvider:
|
|||||||
def health(self, language: str = "en-GB") -> KnowledgeHealth:
|
def health(self, language: str = "en-GB") -> KnowledgeHealth:
|
||||||
try:
|
try:
|
||||||
with self._client() as client:
|
with self._client() as client:
|
||||||
response = client.get("/health")
|
response = client.get("/health/ready")
|
||||||
response.raise_for_status()
|
body = response.json()
|
||||||
available = True
|
available = response.status_code == 200 and body.get("status") == "ok"
|
||||||
detail = "RAGcore reachable."
|
detail = (
|
||||||
except httpx.HTTPError as exc:
|
"RAGcore reachable and ready."
|
||||||
|
if available
|
||||||
|
else f"RAGcore degraded: {body.get('status', 'unknown')}"
|
||||||
|
)
|
||||||
|
except (httpx.HTTPError, ValueError) as exc:
|
||||||
available = False
|
available = False
|
||||||
detail = f"RAGcore unavailable: {type(exc).__name__}: {exc}"
|
detail = f"RAGcore unavailable: {type(exc).__name__}: {exc}"
|
||||||
return KnowledgeHealth(
|
return KnowledgeHealth(
|
||||||
@@ -49,51 +64,58 @@ class RAGcoreKnowledgeProvider:
|
|||||||
tenant=self._settings.ragcore_tenant,
|
tenant=self._settings.ragcore_tenant,
|
||||||
workspace=self._settings.ragcore_workspace,
|
workspace=self._settings.ragcore_workspace,
|
||||||
collection=self._settings.ragcore_collection,
|
collection=self._settings.ragcore_collection,
|
||||||
|
# RAGcore's retrieval API has no corpus-size endpoint to query honestly from
|
||||||
|
# here; left at 0 rather than approximated from a capped search result count.
|
||||||
document_count=0,
|
document_count=0,
|
||||||
)
|
)
|
||||||
|
|
||||||
def ask(self, question: str, correlation_id: str, language: str = "en-GB") -> GroundedAnswer:
|
def ask(self, question: str, correlation_id: str, language: str = "en-GB") -> GroundedAnswer:
|
||||||
|
unavailable = GroundedAnswer(
|
||||||
|
answer="",
|
||||||
|
evidence_state="unavailable",
|
||||||
|
sources=[],
|
||||||
|
provider=self.name,
|
||||||
|
correlation_id=correlation_id,
|
||||||
|
)
|
||||||
|
if not self._settings.ragcore_space_id:
|
||||||
|
return unavailable
|
||||||
|
|
||||||
try:
|
try:
|
||||||
with self._client() as client:
|
with self._client() as client:
|
||||||
response = client.post(
|
response = client.post(
|
||||||
"/api/v1/ask",
|
"/v1/answers",
|
||||||
json={
|
json={
|
||||||
"tenant": self._settings.ragcore_tenant,
|
"query": question,
|
||||||
"workspace": self._settings.ragcore_workspace,
|
"requested_space_ids": [self._settings.ragcore_space_id],
|
||||||
"collection": self._settings.ragcore_collection,
|
|
||||||
"question": question,
|
|
||||||
"correlation_id": correlation_id,
|
|
||||||
"language": language,
|
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
response.raise_for_status()
|
if response.status_code != 200:
|
||||||
|
return unavailable
|
||||||
body = response.json()
|
body = response.json()
|
||||||
except (httpx.HTTPError, ValueError):
|
except (httpx.HTTPError, ValueError):
|
||||||
return GroundedAnswer(
|
return unavailable
|
||||||
answer="",
|
|
||||||
evidence_state="unavailable",
|
|
||||||
sources=[],
|
|
||||||
provider=self.name,
|
|
||||||
correlation_id=correlation_id,
|
|
||||||
)
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
sources = [SourceCard(**s) for s in body.get("sources", [])]
|
citations = {c["id"]: c for c in body.get("citations", [])}
|
||||||
evidence_state = body.get("evidence_state", "insufficient")
|
sources = [
|
||||||
if evidence_state not in ("grounded", "insufficient", "unavailable"):
|
SourceCard(
|
||||||
evidence_state = "insufficient"
|
document_id=str(citation["document_id"]),
|
||||||
|
title=citation["title"],
|
||||||
|
version=str(citation["document_version_id"]),
|
||||||
|
section=citation.get("section") or "",
|
||||||
|
excerpt=citation["excerpt"],
|
||||||
|
)
|
||||||
|
for citation in citations.values()
|
||||||
|
]
|
||||||
|
answerability = body.get("answerability", "not_answerable")
|
||||||
|
is_grounded = answerability in _GROUNDED_ANSWERABILITY and sources
|
||||||
|
evidence_state: EvidenceState = "grounded" if is_grounded else "insufficient"
|
||||||
return GroundedAnswer(
|
return GroundedAnswer(
|
||||||
answer=body.get("answer", ""),
|
answer=body.get("answer", "") if evidence_state == "grounded" else "",
|
||||||
evidence_state=evidence_state,
|
evidence_state=evidence_state,
|
||||||
sources=sources,
|
sources=sources if evidence_state == "grounded" else [],
|
||||||
provider=self.name,
|
|
||||||
correlation_id=correlation_id,
|
|
||||||
)
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
return GroundedAnswer(
|
|
||||||
answer="",
|
|
||||||
evidence_state="unavailable",
|
|
||||||
sources=[],
|
|
||||||
provider=self.name,
|
provider=self.name,
|
||||||
correlation_id=correlation_id,
|
correlation_id=correlation_id,
|
||||||
)
|
)
|
||||||
|
except (TypeError, KeyError, ValueError):
|
||||||
|
return unavailable
|
||||||
|
|||||||
@@ -158,12 +158,190 @@ def test_knowledge_status_endpoint(ops_client):
|
|||||||
assert response.json()["provider"] == "demo"
|
assert response.json()["provider"] == "demo"
|
||||||
|
|
||||||
|
|
||||||
def test_ragcore_provider_degrades_to_unavailable(monkeypatch):
|
class _FakeResponse:
|
||||||
def fake_client(*args, **kwargs):
|
def __init__(self, status_code: int, body: dict):
|
||||||
raise httpx.ConnectError("no ragcore in this environment")
|
self.status_code = status_code
|
||||||
|
self._body = body
|
||||||
|
|
||||||
|
def json(self) -> dict:
|
||||||
|
return self._body
|
||||||
|
|
||||||
|
|
||||||
|
class _FakeClient:
|
||||||
|
def __init__(self, get_response=None, post_response=None, raise_on=None):
|
||||||
|
self._get_response = get_response
|
||||||
|
self._post_response = post_response
|
||||||
|
self._raise_on = raise_on
|
||||||
|
|
||||||
|
def __enter__(self):
|
||||||
|
return self
|
||||||
|
|
||||||
|
def __exit__(self, *args):
|
||||||
|
return False
|
||||||
|
|
||||||
|
def get(self, path):
|
||||||
|
if self._raise_on == "get":
|
||||||
|
raise httpx.ConnectError("no ragcore in this environment")
|
||||||
|
return self._get_response
|
||||||
|
|
||||||
|
def post(self, path, json=None):
|
||||||
|
if self._raise_on == "post":
|
||||||
|
raise httpx.ConnectError("no ragcore in this environment")
|
||||||
|
return self._post_response
|
||||||
|
|
||||||
|
|
||||||
|
def test_ragcore_provider_degrades_to_unavailable(monkeypatch):
|
||||||
provider = RAGcoreKnowledgeProvider()
|
provider = RAGcoreKnowledgeProvider()
|
||||||
monkeypatch.setattr(provider, "_client", fake_client)
|
monkeypatch.setattr(provider._settings, "ragcore_space_id", "space-1")
|
||||||
|
monkeypatch.setattr(provider, "_client", lambda: _FakeClient(raise_on="post"))
|
||||||
answer = provider.ask("Anything?", "test-correlation-3")
|
answer = provider.ask("Anything?", "test-correlation-3")
|
||||||
assert answer.evidence_state == "unavailable"
|
assert answer.evidence_state == "unavailable"
|
||||||
assert answer.sources == []
|
assert answer.sources == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_ragcore_provider_without_configured_space_is_unavailable_without_a_network_call(
|
||||||
|
monkeypatch,
|
||||||
|
):
|
||||||
|
provider = RAGcoreKnowledgeProvider()
|
||||||
|
monkeypatch.setattr(provider._settings, "ragcore_space_id", "")
|
||||||
|
|
||||||
|
def fail_if_called():
|
||||||
|
raise AssertionError("should not call RAGcore without a configured space id")
|
||||||
|
|
||||||
|
monkeypatch.setattr(provider, "_client", fail_if_called)
|
||||||
|
answer = provider.ask("Anything?", "test-correlation-no-space")
|
||||||
|
assert answer.evidence_state == "unavailable"
|
||||||
|
|
||||||
|
|
||||||
|
def test_ragcore_provider_health_reports_ready_status(monkeypatch):
|
||||||
|
provider = RAGcoreKnowledgeProvider()
|
||||||
|
monkeypatch.setattr(
|
||||||
|
provider,
|
||||||
|
"_client",
|
||||||
|
lambda: _FakeClient(get_response=_FakeResponse(200, {"status": "ok"})),
|
||||||
|
)
|
||||||
|
health = provider.health()
|
||||||
|
assert health.provider == "ragcore"
|
||||||
|
assert health.available is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_ragcore_provider_health_reports_degraded_status(monkeypatch):
|
||||||
|
provider = RAGcoreKnowledgeProvider()
|
||||||
|
monkeypatch.setattr(
|
||||||
|
provider,
|
||||||
|
"_client",
|
||||||
|
lambda: _FakeClient(get_response=_FakeResponse(200, {"status": "degraded"})),
|
||||||
|
)
|
||||||
|
health = provider.health()
|
||||||
|
assert health.available is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_ragcore_provider_health_degrades_on_connection_error(monkeypatch):
|
||||||
|
provider = RAGcoreKnowledgeProvider()
|
||||||
|
monkeypatch.setattr(provider, "_client", lambda: _FakeClient(raise_on="get"))
|
||||||
|
health = provider.health()
|
||||||
|
assert health.available is False
|
||||||
|
assert "unavailable" in health.detail.lower()
|
||||||
|
|
||||||
|
|
||||||
|
def _answers_body(**overrides) -> dict:
|
||||||
|
body = {
|
||||||
|
"answer": "Report damage and route the vehicle to maintenance.",
|
||||||
|
"answerability": "answerable",
|
||||||
|
"citations": [
|
||||||
|
{
|
||||||
|
"id": "cite-1",
|
||||||
|
"document_id": "doc-1",
|
||||||
|
"document_version_id": "version-1",
|
||||||
|
"title": "Damage handling procedure",
|
||||||
|
"section": "Detection",
|
||||||
|
"excerpt": "Inspect the vehicle for visible damage.",
|
||||||
|
}
|
||||||
|
],
|
||||||
|
}
|
||||||
|
body.update(overrides)
|
||||||
|
return body
|
||||||
|
|
||||||
|
|
||||||
|
def test_ragcore_provider_grounded_answer_maps_citations_to_sources(monkeypatch):
|
||||||
|
provider = RAGcoreKnowledgeProvider()
|
||||||
|
monkeypatch.setattr(provider._settings, "ragcore_space_id", "space-1")
|
||||||
|
monkeypatch.setattr(
|
||||||
|
provider,
|
||||||
|
"_client",
|
||||||
|
lambda: _FakeClient(post_response=_FakeResponse(200, _answers_body())),
|
||||||
|
)
|
||||||
|
answer = provider.ask("What must I do about damage?", "test-correlation-grounded")
|
||||||
|
assert answer.evidence_state == "grounded"
|
||||||
|
assert answer.answer
|
||||||
|
assert len(answer.sources) == 1
|
||||||
|
source = answer.sources[0]
|
||||||
|
assert source.document_id == "doc-1"
|
||||||
|
assert source.title == "Damage handling procedure"
|
||||||
|
assert source.version == "version-1"
|
||||||
|
assert source.section == "Detection"
|
||||||
|
assert source.excerpt
|
||||||
|
|
||||||
|
|
||||||
|
def test_ragcore_provider_not_answerable_is_insufficient_and_never_fabricates(monkeypatch):
|
||||||
|
provider = RAGcoreKnowledgeProvider()
|
||||||
|
monkeypatch.setattr(provider._settings, "ragcore_space_id", "space-1")
|
||||||
|
monkeypatch.setattr(
|
||||||
|
provider,
|
||||||
|
"_client",
|
||||||
|
lambda: _FakeClient(
|
||||||
|
post_response=_FakeResponse(
|
||||||
|
200,
|
||||||
|
_answers_body(
|
||||||
|
answer="This should never be shown.",
|
||||||
|
answerability="not_answerable",
|
||||||
|
citations=[],
|
||||||
|
),
|
||||||
|
)
|
||||||
|
),
|
||||||
|
)
|
||||||
|
answer = provider.ask("Unrelated question?", "test-correlation-insufficient")
|
||||||
|
assert answer.evidence_state == "insufficient"
|
||||||
|
assert answer.answer == ""
|
||||||
|
assert answer.sources == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_ragcore_provider_answerable_without_citations_is_insufficient(monkeypatch):
|
||||||
|
provider = RAGcoreKnowledgeProvider()
|
||||||
|
monkeypatch.setattr(provider._settings, "ragcore_space_id", "space-1")
|
||||||
|
monkeypatch.setattr(
|
||||||
|
provider,
|
||||||
|
"_client",
|
||||||
|
lambda: _FakeClient(
|
||||||
|
post_response=_FakeResponse(
|
||||||
|
200, _answers_body(answerability="answerable", citations=[])
|
||||||
|
)
|
||||||
|
),
|
||||||
|
)
|
||||||
|
answer = provider.ask("What must I do about damage?", "test-correlation-no-citations")
|
||||||
|
assert answer.evidence_state == "insufficient"
|
||||||
|
assert answer.sources == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_ragcore_provider_non_200_response_is_unavailable(monkeypatch):
|
||||||
|
provider = RAGcoreKnowledgeProvider()
|
||||||
|
monkeypatch.setattr(provider._settings, "ragcore_space_id", "space-1")
|
||||||
|
monkeypatch.setattr(
|
||||||
|
provider,
|
||||||
|
"_client",
|
||||||
|
lambda: _FakeClient(post_response=_FakeResponse(401, {"code": "AUTHENTICATION_REQUIRED"})),
|
||||||
|
)
|
||||||
|
answer = provider.ask("Anything?", "test-correlation-401")
|
||||||
|
assert answer.evidence_state == "unavailable"
|
||||||
|
|
||||||
|
|
||||||
|
def test_ragcore_provider_malformed_response_is_unavailable(monkeypatch):
|
||||||
|
provider = RAGcoreKnowledgeProvider()
|
||||||
|
monkeypatch.setattr(provider._settings, "ragcore_space_id", "space-1")
|
||||||
|
monkeypatch.setattr(
|
||||||
|
provider,
|
||||||
|
"_client",
|
||||||
|
lambda: _FakeClient(post_response=_FakeResponse(200, {"citations": "not-a-list"})),
|
||||||
|
)
|
||||||
|
answer = provider.ask("Anything?", "test-correlation-malformed")
|
||||||
|
assert answer.evidence_state == "unavailable"
|
||||||
|
|||||||
@@ -31,6 +31,7 @@ services:
|
|||||||
RAGCORE_WORKSPACE: ${RAGCORE_WORKSPACE:-mobilityops}
|
RAGCORE_WORKSPACE: ${RAGCORE_WORKSPACE:-mobilityops}
|
||||||
RAGCORE_COLLECTION: ${RAGCORE_COLLECTION:-internal-procedures}
|
RAGCORE_COLLECTION: ${RAGCORE_COLLECTION:-internal-procedures}
|
||||||
RAGCORE_API_TOKEN: ${RAGCORE_API_TOKEN:-}
|
RAGCORE_API_TOKEN: ${RAGCORE_API_TOKEN:-}
|
||||||
|
RAGCORE_SPACE_ID: ${RAGCORE_SPACE_ID:-}
|
||||||
N8N_WEBHOOK_URL: ${N8N_WEBHOOK_URL:-http://n8n:5678/webhook/mobilityops-return}
|
N8N_WEBHOOK_URL: ${N8N_WEBHOOK_URL:-http://n8n:5678/webhook/mobilityops-return}
|
||||||
N8N_WEBHOOK_TRIGGER_TOKEN: ${MOBILITYOPS_WEBHOOK_TRIGGER_TOKEN:-replace-me-n8n-webhook-trigger-token}
|
N8N_WEBHOOK_TRIGGER_TOKEN: ${MOBILITYOPS_WEBHOOK_TRIGGER_TOKEN:-replace-me-n8n-webhook-trigger-token}
|
||||||
N8N_CALLBACK_TOKEN: ${MOBILITYOPS_CALLBACK_TOKEN:-replace-me-n8n-callback-token}
|
N8N_CALLBACK_TOKEN: ${MOBILITYOPS_CALLBACK_TOKEN:-replace-me-n8n-callback-token}
|
||||||
|
|||||||
Reference in New Issue
Block a user