From b511ba2dbc89c3552aae67f1fa79db4cc24bcf14 Mon Sep 17 00:00:00 2001 From: NuklearRabbit <145918611+NuklearRabbit@users.noreply.github.com> Date: Sat, 1 Aug 2026 22:57:06 +0200 Subject: [PATCH] M5: implement RAGcore knowledge integration KnowledgeProvider protocol with a deterministic TF-IDF-weighted extractive demo provider (never generative, always cites real excerpts) and a RAGcore HTTP adapter that degrades cleanly to unavailable. Knowledge nav + chat-style Q&A UI with source cards and honest grounded/insufficient/unavailable states. 57 backend tests passing, ruff clean. Fixed a real relevance bug (generic terms like "vehicle" crowding out distinctive matches) via IDF weighting, found by testing the actual S6 scenario. Verified end-to-end in the browser: grounded damage question cites both expected procedures; unrelated question honestly returns insufficient evidence with no fabrication. --- PROJECT_STATE.md | 20 +- backend/Dockerfile | 1 + backend/app/api/routers/knowledge.py | 51 +++++ backend/app/core/config.py | 3 + backend/app/main.py | 2 + backend/app/services/knowledge/__init__.py | 56 ++++++ backend/app/services/knowledge/demo.py | 221 +++++++++++++++++++++ backend/app/services/knowledge/ragcore.py | 98 +++++++++ backend/tests/test_knowledge.py | 84 ++++++++ frontend/src/App.tsx | 2 + frontend/src/api/types.ts | 26 +++ frontend/src/components/Layout.tsx | 1 + frontend/src/pages/Knowledge.tsx | 121 +++++++++++ frontend/src/styles.css | 27 +++ 14 files changed, 710 insertions(+), 3 deletions(-) create mode 100644 backend/app/api/routers/knowledge.py create mode 100644 backend/app/services/knowledge/__init__.py create mode 100644 backend/app/services/knowledge/demo.py create mode 100644 backend/app/services/knowledge/ragcore.py create mode 100644 backend/tests/test_knowledge.py create mode 100644 frontend/src/pages/Knowledge.tsx diff --git a/PROJECT_STATE.md b/PROJECT_STATE.md index efbc04b..06c26b8 100644 --- a/PROJECT_STATE.md +++ b/PROJECT_STATE.md @@ -2,7 +2,7 @@ ## Current milestone -M4 — complete. Starting M5 next. +M5 — complete. Starting M6 next. ## Locked decisions @@ -94,10 +94,24 @@ M4 — complete. Starting M5 next. - Full live round trip (not mocked): registered a real return on `BK-DEMO-RETURN` → outbox event queued → background dispatcher delivered it to the now-activated n8n workflow within its 3s poll interval → n8n called back into `/api/v1/integrations/n8n/return-callback` (200 OK, confirmed in `docker compose logs api`) → dispatcher's original POST received n8n's success response → event flipped to `succeeded` on attempt 1, visible on `/automation`. - S5 scenario end-to-end in the browser: seeded `BK-H-0020` (`failed`, 3 attempts, "Synthetic connection timeout to n8n") → clicked Retry → `pending` → within ~3s, live dispatcher delivered it through the real n8n instance → `succeeded`, 4 attempts. This is the full documented S5 scenario working for real, not simulated. +### M5 — RAGcore knowledge integration +- `app/services/knowledge/__init__.py`: `KnowledgeProvider` Protocol (sync, not async — the rest of the backend is sync SQLAlchemy/FastAPI, so an async provider interface would have meant bridging paradigms for no benefit) with `health()`/`ask()`, plus `GroundedAnswer`/`SourceCard`/`KnowledgeHealth` Pydantic models matching `contracts/openapi.yaml`'s `GroundedAnswer` schema exactly. `get_knowledge_provider()` factory switches on `settings.knowledge_provider` ("demo" default, "ragcore" opt-in). +- `app/services/knowledge/demo.py` — `DemoKnowledgeProvider`: parses the 10 `knowledge/procedures/*.md` files' YAML frontmatter (hand-rolled flat parser, not PyYAML — avoided adding a dependency for a 6-key flat block) and `## `-delimited sections at startup, then does **TF-IDF-weighted keyword retrieval** (not naive keyword counting) with light suffix-stripping stemming (`returns`→`return`, `damaged`→`damage`). This is extractive, not generative: it returns real excerpts and a templated answer sentence, never invented text. + - **Real bug found and fixed by testing the actual S6 question, not by inspection**: naive flat keyword-overlap scoring (first cut) let the word "vehicle" — present in nearly every document's title — crowd out the actually-relevant `damage-procedure` document from the top-3 results for "What must I do when a vehicle returns with damage?", because generic words scored the same as distinctive ones. Fixed by computing corpus-wide IDF per token (`log((N+1)/(df+1)) + 1`) and weighting matches by it, so common terms contribute little and rare/distinctive terms (like "damage") dominate the ranking. Verified: the S6 question now returns `damage-procedure` and `vehicle-return-procedure` in the top 3, matching the documented expectation exactly. +- `app/services/knowledge/ragcore.py` — `RAGcoreKnowledgeProvider`: real `httpx` adapter guessing a plausible REST contract (`GET /health`, `POST /api/v1/ask`) per `contracts/ragcore-contract-assumptions.md` (RAGcore is built separately; no live instance was reachable this session to verify against). Any connection error, timeout, or malformed response degrades to `evidence_state: "unavailable"` rather than raising — this is the adapter that actually exercises the architecture's "RAGcore failure disables knowledge answers only" reliability boundary. Not wired as the active provider by default; `KNOWLEDGE_PROVIDER=ragcore` would need a real, verified base URL to turn on. +- `POST /api/v1/knowledge/questions` + `GET /api/v1/knowledge/status` (`app/api/routers/knowledge.py`). Audit event `knowledge_question_asked` logs `evidence_state`, `provider`, `source_ids`, and `question_length` only — **not** the question text itself, per `docs/12-security-and-audit.md` ("log question metadata and source IDs, not unnecessary full prompts"). +- Knowledge nav + page (`pages/Knowledge.tsx`): chat-style question box, source cards (title/version/section/excerpt) prioritized over the answer text per `docs/06-ui-ux.md`, explicit `grounded`/`insufficient`/`unavailable` states with distinct visual treatment — never a fabricated-looking answer for the latter two. +- Dockerfile now also `COPY knowledge ./knowledge`; added `KNOWLEDGE_DIR` setting (`/app/knowledge/procedures` in-container, same pattern as `SEED_DIR`) rather than deriving the path from `__file__` — simpler and doesn't break if the module moves. +- Commands run and verified from this checkout: + - `docker compose run --rm api pytest -q` — **57 passed** (new `tests/test_knowledge.py`: S6 grounded-with-expected-sources, unrelated question is honestly insufficient with no fabrication, demo provider health/document count, endpoint auth required, audit doesn't leak question text, RAGcore adapter degrades to unavailable on a simulated connection error). + - `docker compose run --rm api ruff check .` — All checks passed. + - `npm run build` — clean. + - Full browser run of S6 end-to-end: asked "What must I do when a vehicle returns with damage?" on `/knowledge` → grounded answer citing "Vehicle return procedure" (2 sections) and "Damage handling procedure" with real excerpts. Also asked an unrelated question ("What is the weather forecast for tomorrow?") → correctly returned "Insufficient evidence" / "No matching procedure was found" with zero sources, confirming no fabrication. + ## Known blockers -None. External service credentials may be absent; use the documented demo/degraded providers. The n8n workflow-activation steps above are a one-time manual setup requirement in this environment, not a blocker — but not yet scripted; M7 should either automate it (e.g. a bootstrap script CI/compose can run) or document it clearly enough for `docs/14-testing-and-acceptance.md`'s clean-checkout criteria. +None. External service credentials may be absent; use the documented demo/degraded providers. The n8n workflow-activation steps from M4 are a one-time manual setup requirement in this environment, not a blocker — but not yet scripted; M7 should either automate it (e.g. a bootstrap script CI/compose can run) or document it clearly enough for `docs/14-testing-and-acceptance.md`'s clean-checkout criteria. RAGcore itself was never reachable this session — `RAGcoreKnowledgeProvider` is implemented and unit-tested for its unavailable-degradation path but its actual request/response contract against a real RAGcore instance is unverified; the demo provider is what M5's acceptance criteria are actually satisfied by. ## Exact next action -Start M5 (RAGcore knowledge integration): read `docs/09-ragcore-integration.md`, `knowledge/manifest.json`. Implement the `KnowledgeProvider` protocol (`health`, `ask`, `sync_manifest`), a `DemoKnowledgeProvider` doing deterministic keyword/BM25-style retrieval directly over `knowledge/procedures/*.md` (extractive, not generative — must return real excerpts + `insufficient`/`unavailable` states honestly, never fabricate), and a `RAGcoreKnowledgeProvider` adapter stub for the real service (`RAGCORE_BASE_URL` etc. are already in config/compose from M0, but no actual RAGcore instance is confirmed reachable this session — build the demo provider as the one that actually has to work for the acceptance criteria, and make the RAGcore adapter degrade to `unavailable` cleanly if unreachable, per `docs/03-architecture.md`'s reliability boundary "RAGcore failure disables knowledge answers only"). Add `POST /api/v1/knowledge/questions` and `GET /api/v1/knowledge/status`, a Knowledge nav item + page (chat-style question box, source cards with title/version/section/excerpt, explicit unavailable/insufficient states — no fabricated answers ever). S6 demo scenario: "What must I do when a vehicle returns with damage?" must cite `knowledge/procedures/03-damage-handling.md` and `02-vehicle-return.md`. +Start M6 (ITWorx MCP Hub publication): read `docs/10-mcp-hub-integration.md`, `contracts/mcp-tools.json`. Implement the four read-only, service-token-protected provider endpoints under `/api/v1/integrations/mcp/` (`operations-summary`, `attention-vehicles`, `vehicles/{public_ref}`, and a knowledge-search façade wrapping the M5 `KnowledgeProvider` — per `contracts/mcp-tools.json`'s `mobilityops_search_knowledge` tool and `docs/10-mcp-hub-integration.md`'s note to route through a narrow façade rather than duplicate retrieval logic). Auth: reuse the same shared-secret-header pattern already built for the n8n callback in M4 (`X-Service-Token`, a new `MCP_HUB_SERVICE_TOKEN` setting — `.env.example` already has the env var name reserved) rather than inventing a second auth mechanism. Must not expose generic SQL, arbitrary fetch, write/mutation actions, or secrets — these are read-only summaries only. Add MobilityOps-side service-request audit events (the Hub owns its own central tool-call audit; MobilityOps only needs to record that its provider APIs were reached, with tool name/correlation ID/client identity/result status). No actual ITWorx MCP Hub instance is confirmed reachable in this environment (same situation as RAGcore in M5) — validate the four provider endpoints directly via authenticated `curl`/tests rather than a live Hub round trip, and note that gap explicitly rather than claiming an unverified integration works. diff --git a/backend/Dockerfile b/backend/Dockerfile index 5661f63..2209ef8 100644 --- a/backend/Dockerfile +++ b/backend/Dockerfile @@ -9,6 +9,7 @@ COPY backend/alembic ./alembic COPY backend/alembic.ini ./ COPY backend/tests ./tests COPY seed ./seed +COPY knowledge ./knowledge RUN pip install --no-cache-dir --no-deps -e . EXPOSE 8000 CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"] diff --git a/backend/app/api/routers/knowledge.py b/backend/app/api/routers/knowledge.py new file mode 100644 index 0000000..ae15346 --- /dev/null +++ b/backend/app/api/routers/knowledge.py @@ -0,0 +1,51 @@ +from __future__ import annotations + +import uuid + +from fastapi import APIRouter, Depends +from pydantic import BaseModel, Field +from sqlalchemy.orm import Session + +from app.api.deps import get_current_user, get_db +from app.schemas import CurrentUser +from app.services.audit import record_audit_event +from app.services.knowledge import GroundedAnswer, KnowledgeHealth, get_knowledge_provider + +router = APIRouter(prefix="/api/v1/knowledge", tags=["knowledge"]) + + +class AskQuestionRequest(BaseModel): + question: str = Field(min_length=3, max_length=1000) + + +@router.post("/questions", response_model=GroundedAnswer) +def ask_question( + body: AskQuestionRequest, + db: Session = Depends(get_db), + user: CurrentUser = Depends(get_current_user), +) -> GroundedAnswer: + correlation_id = str(uuid.uuid4()) + provider = get_knowledge_provider() + answer = provider.ask(body.question, correlation_id) + + record_audit_event( + db, + actor_type="user", + actor_label=user.display_name, + action="knowledge_question_asked", + entity_type="knowledge", + correlation_id=uuid.UUID(correlation_id), + metadata={ + "evidence_state": answer.evidence_state, + "provider": answer.provider, + "source_ids": [s.document_id for s in answer.sources], + "question_length": len(body.question), + }, + ) + db.commit() + return answer + + +@router.get("/status", response_model=KnowledgeHealth) +def knowledge_status(_user: CurrentUser = Depends(get_current_user)) -> KnowledgeHealth: + return get_knowledge_provider().health() diff --git a/backend/app/core/config.py b/backend/app/core/config.py index 4e3bd54..7982219 100644 --- a/backend/app/core/config.py +++ b/backend/app/core/config.py @@ -14,6 +14,8 @@ class Settings(BaseSettings): ragcore_tenant: str = "northstar-mobility-demo" ragcore_workspace: str = "mobilityops" ragcore_collection: str = "internal-procedures" + ragcore_api_token: str = "" + ragcore_http_timeout_seconds: float = 5.0 n8n_webhook_url: str = "http://n8n:5678/webhook/mobilityops-return" n8n_callback_token: str = "replace-me-n8n-callback-token" n8n_dispatch_enabled: bool = True @@ -24,6 +26,7 @@ class Settings(BaseSettings): session_cookie_name: str = "mobilityops_session" session_ttl_seconds: int = 60 * 60 * 8 seed_dir: str = "/app/seed" + knowledge_dir: str = "/app/knowledge/procedures" cors_allow_origins: str = "http://localhost:1228" demo_today: str = "2026-08-01" diff --git a/backend/app/main.py b/backend/app/main.py index 2298b23..9413299 100644 --- a/backend/app/main.py +++ b/backend/app/main.py @@ -12,6 +12,7 @@ from app.api.routers import ( data_quality, demo, integrations, + knowledge, vehicles, workflows, ) @@ -84,3 +85,4 @@ app.include_router(audit.router) app.include_router(data_quality.router) app.include_router(workflows.router) app.include_router(integrations.router) +app.include_router(knowledge.router) diff --git a/backend/app/services/knowledge/__init__.py b/backend/app/services/knowledge/__init__.py new file mode 100644 index 0000000..48e5bcb --- /dev/null +++ b/backend/app/services/knowledge/__init__.py @@ -0,0 +1,56 @@ +from __future__ import annotations + +from functools import lru_cache +from typing import Literal, Protocol + +from pydantic import BaseModel + +from app.core.config import get_settings + +EvidenceState = Literal["grounded", "insufficient", "unavailable"] + + +class SourceCard(BaseModel): + document_id: str + title: str + version: str + section: str + excerpt: str + + +class GroundedAnswer(BaseModel): + answer: str + evidence_state: EvidenceState + sources: list[SourceCard] + provider: str + correlation_id: str + + +class KnowledgeHealth(BaseModel): + provider: str + available: bool + detail: str + tenant: str + workspace: str + collection: str + document_count: int + + +class KnowledgeProvider(Protocol): + name: str + + def health(self) -> KnowledgeHealth: ... + + def ask(self, question: str, correlation_id: str) -> GroundedAnswer: ... + + +@lru_cache +def get_knowledge_provider() -> KnowledgeProvider: + settings = get_settings() + if settings.knowledge_provider == "ragcore": + from app.services.knowledge.ragcore import RAGcoreKnowledgeProvider + + return RAGcoreKnowledgeProvider() + from app.services.knowledge.demo import DemoKnowledgeProvider + + return DemoKnowledgeProvider() diff --git a/backend/app/services/knowledge/demo.py b/backend/app/services/knowledge/demo.py new file mode 100644 index 0000000..7ab7a88 --- /dev/null +++ b/backend/app/services/knowledge/demo.py @@ -0,0 +1,221 @@ +from __future__ import annotations + +import math +import re +from dataclasses import dataclass, field +from pathlib import Path + +from app.core.config import get_settings +from app.services.knowledge import GroundedAnswer, KnowledgeHealth, SourceCard + +STOPWORDS = { + "a", "an", "the", "is", "are", "was", "were", "be", "been", "being", + "to", "of", "in", "on", "at", "for", "and", "or", "but", "if", "then", + "do", "does", "did", "must", "may", "can", "could", "should", "would", + "i", "you", "it", "we", "they", "my", "your", "what", "when", "how", + "with", "without", "this", "that", "these", "those", "not", "no", +} + +_WORD_RE = re.compile(r"[a-z0-9]+") + + +def _stem(word: str) -> str: + # Deterministic, intentionally crude suffix stripping — good enough to match "returns" + # with "return" or "damaged" with "damage" without pulling in a stemming dependency. + for suffix in ("ing", "edly", "ed", "es", "s"): + if len(word) > len(suffix) + 2 and word.endswith(suffix): + return word[: -len(suffix)] + return word + + +def _tokenize(text: str) -> set[str]: + words = _WORD_RE.findall(text.lower()) + return {_stem(w) for w in words if w not in STOPWORDS and len(w) > 2} + + +@dataclass +class Document: + document_id: str + title: str + version: str + title_tokens: set[str] = field(default_factory=set) + + +@dataclass +class ScoredSection: + document: Document + heading: str + text: str + heading_tokens: set[str] + body_tokens: set[str] + + +def _parse_frontmatter(raw: str) -> tuple[dict[str, str], str]: + if not raw.startswith("---"): + return {}, raw + end = raw.find("\n---", 3) + if end == -1: + return {}, raw + block = raw[3:end].strip() + body = raw[end + 4 :].lstrip("\n") + meta: dict[str, str] = {} + for line in block.splitlines(): + if ":" not in line: + continue + key, _, value = line.partition(":") + meta[key.strip()] = value.strip().strip('"') + return meta, body + + +def _split_sections(body: str) -> list[tuple[str, str]]: + sections: list[tuple[str, str]] = [] + current_heading = "Overview" + current_lines: list[str] = [] + for line in body.splitlines(): + if line.startswith("## "): + if current_lines: + sections.append((current_heading, "\n".join(current_lines).strip())) + current_heading = line[3:].strip() + current_lines = [] + elif line.startswith("# "): + continue + else: + current_lines.append(line) + if current_lines: + sections.append((current_heading, "\n".join(current_lines).strip())) + return sections + + +def _load_sections(procedures_dir: Path) -> list[ScoredSection]: + sections: list[ScoredSection] = [] + for path in sorted(procedures_dir.glob("*.md")): + raw = path.read_text(encoding="utf-8") + meta, body = _parse_frontmatter(raw) + title = meta.get("title", path.stem) + doc = Document( + document_id=meta.get("document_id", path.stem), + title=title, + version=meta.get("version", "1.0"), + title_tokens=_tokenize(title), + ) + for heading, text in _split_sections(body): + sections.append( + ScoredSection( + document=doc, + heading=heading, + text=text, + heading_tokens=_tokenize(heading), + body_tokens=_tokenize(text), + ) + ) + return sections + + +class DemoKnowledgeProvider: + """Deterministic extractive retrieval over the local procedure Markdown files. + + Not a generative model: it scores sections with TF-IDF-weighted keyword overlap + (downweighting terms common across the whole corpus, like "vehicle", in favor of + distinctive ones, like "damage") and returns real excerpts, never invented text. + """ + + name = "demo" + + def __init__(self) -> None: + settings = get_settings() + self._settings = settings + self._procedures_dir = Path(settings.knowledge_dir) + self._sections = _load_sections(self._procedures_dir) + self._document_count = len({s.document.document_id for s in self._sections}) + self._idf = self._build_idf(self._sections) + + @staticmethod + def _build_idf(sections: list[ScoredSection]) -> dict[str, float]: + n = len(sections) or 1 + doc_freq: dict[str, int] = {} + for section in sections: + doc = section.document + all_tokens = doc.title_tokens | section.heading_tokens | section.body_tokens + for token in all_tokens: + doc_freq[token] = doc_freq.get(token, 0) + 1 + return {token: math.log((n + 1) / (df + 1)) + 1 for token, df in doc_freq.items()} + + def health(self) -> KnowledgeHealth: + return KnowledgeHealth( + provider=self.name, + available=True, + detail="Deterministic keyword-matching demo provider; no external service.", + tenant=self._settings.ragcore_tenant, + workspace=self._settings.ragcore_workspace, + collection=self._settings.ragcore_collection, + document_count=self._document_count, + ) + + def _score(self, query_tokens: set[str], section: ScoredSection) -> float: + score = 0.0 + for token in query_tokens: + idf = self._idf.get(token, 0.0) + if idf == 0.0: + continue + if token in section.heading_tokens: + score += 3 * idf + elif token in section.document.title_tokens: + score += 2 * idf + elif token in section.body_tokens: + score += idf + return score + + def ask(self, question: str, correlation_id: str) -> GroundedAnswer: + query_tokens = _tokenize(question) + scored = [ + (self._score(query_tokens, section), section) + for section in self._sections + ] + scored = [(score, section) for score, section in scored if score > 0] + scored.sort(key=lambda item: item[0], reverse=True) + top = scored[:3] + + if not top: + return GroundedAnswer( + answer="No matching procedure was found for this question.", + evidence_state="insufficient", + sources=[], + provider=self.name, + correlation_id=correlation_id, + ) + + sources = [ + SourceCard( + document_id=section.document.document_id, + title=section.document.title, + version=section.document.version, + section=section.heading, + excerpt=(section.text[:400] + "…") if len(section.text) > 400 else section.text, + ) + for _, section in top + ] + + if top[0][0] < 3: + return GroundedAnswer( + answer=( + "The available procedures do not clearly answer this question. " + "The closest matches are included below for review." + ), + evidence_state="insufficient", + sources=sources, + provider=self.name, + correlation_id=correlation_id, + ) + + lead_section = top[0][1] + answer = ( + f'Per "{lead_section.document.title}" (v{lead_section.document.version}), ' + f'section "{lead_section.heading}": {lead_section.text.splitlines()[0][:300]}' + ) + return GroundedAnswer( + answer=answer, + evidence_state="grounded", + sources=sources, + provider=self.name, + correlation_id=correlation_id, + ) diff --git a/backend/app/services/knowledge/ragcore.py b/backend/app/services/knowledge/ragcore.py new file mode 100644 index 0000000..88abd0e --- /dev/null +++ b/backend/app/services/knowledge/ragcore.py @@ -0,0 +1,98 @@ +from __future__ import annotations + +import httpx + +from app.core.config import get_settings +from app.services.knowledge import GroundedAnswer, KnowledgeHealth, SourceCard + + +class RAGcoreKnowledgeProvider: + """Adapter for the central RAGcore service. + + RAGcore is built and owned separately (see contracts/ragcore-contract-assumptions.md). + No live RAGcore instance was reachable during this build, so the exact request/response + shape below is a best-effort guess at a REST contract; any failure (connection, timeout, + malformed response) degrades to `unavailable` rather than raising, per the architecture's + reliability boundary: RAGcore failure disables knowledge answers only, never the rest of + the app, and never fabricates an answer. + """ + + name = "ragcore" + + def __init__(self) -> None: + self._settings = get_settings() + + def _client(self) -> httpx.Client: + headers = {} + if self._settings.ragcore_api_token: + headers["Authorization"] = f"Bearer {self._settings.ragcore_api_token}" + return httpx.Client( + base_url=self._settings.ragcore_base_url, + headers=headers, + timeout=self._settings.ragcore_http_timeout_seconds, + ) + + def health(self) -> KnowledgeHealth: + try: + with self._client() as client: + response = client.get("/health") + response.raise_for_status() + available = True + detail = "RAGcore reachable." + except httpx.HTTPError as exc: + available = False + detail = f"RAGcore unavailable: {type(exc).__name__}: {exc}" + return KnowledgeHealth( + provider=self.name, + available=available, + detail=detail, + tenant=self._settings.ragcore_tenant, + workspace=self._settings.ragcore_workspace, + collection=self._settings.ragcore_collection, + document_count=0, + ) + + def ask(self, question: str, correlation_id: str) -> GroundedAnswer: + try: + with self._client() as client: + response = client.post( + "/api/v1/ask", + json={ + "tenant": self._settings.ragcore_tenant, + "workspace": self._settings.ragcore_workspace, + "collection": self._settings.ragcore_collection, + "question": question, + "correlation_id": correlation_id, + }, + ) + response.raise_for_status() + body = response.json() + except (httpx.HTTPError, ValueError): + return GroundedAnswer( + answer="", + evidence_state="unavailable", + sources=[], + provider=self.name, + correlation_id=correlation_id, + ) + + try: + sources = [SourceCard(**s) for s in body.get("sources", [])] + evidence_state = body.get("evidence_state", "insufficient") + if evidence_state not in ("grounded", "insufficient", "unavailable"): + evidence_state = "insufficient" + return GroundedAnswer( + answer=body.get("answer", ""), + evidence_state=evidence_state, + sources=sources, + provider=self.name, + correlation_id=correlation_id, + ) + except (TypeError, ValueError): + return GroundedAnswer( + answer="", + evidence_state="unavailable", + sources=[], + provider=self.name, + correlation_id=correlation_id, + ) diff --git a/backend/tests/test_knowledge.py b/backend/tests/test_knowledge.py new file mode 100644 index 0000000..19691ff --- /dev/null +++ b/backend/tests/test_knowledge.py @@ -0,0 +1,84 @@ +from __future__ import annotations + +import httpx + +from app.services.knowledge.demo import DemoKnowledgeProvider +from app.services.knowledge.ragcore import RAGcoreKnowledgeProvider + + +def test_s6_damage_question_is_grounded_with_expected_sources(): + provider = DemoKnowledgeProvider() + answer = provider.ask( + "What must I do when a vehicle returns with damage?", "test-correlation-1" + ) + assert answer.evidence_state == "grounded" + document_ids = {s.document_id for s in answer.sources} + assert "damage-procedure" in document_ids + assert "vehicle-return-procedure" in document_ids + assert answer.answer # never empty for a grounded answer + for source in answer.sources: + assert source.excerpt + + +def test_unrelated_question_is_insufficient(): + provider = DemoKnowledgeProvider() + answer = provider.ask("What is the capital of France?", "test-correlation-2") + assert answer.evidence_state == "insufficient" + assert "France" not in answer.answer # never fabricates an answer beyond the procedures + + +def test_demo_provider_health_reports_document_count(): + provider = DemoKnowledgeProvider() + health = provider.health() + assert health.provider == "demo" + assert health.available is True + assert health.document_count == 10 + + +def test_ask_question_endpoint_grounded(ops_client): + response = ops_client.post( + "/api/v1/knowledge/questions", + json={"question": "What must I do when a vehicle returns with damage?"}, + ) + assert response.status_code == 200 + body = response.json() + assert body["evidence_state"] == "grounded" + assert body["provider"] == "demo" + assert len(body["sources"]) > 0 + + +def test_ask_question_requires_authentication(client): + response = client.post("/api/v1/knowledge/questions", json={"question": "Anything?"}) + assert response.status_code == 401 + + +def test_ask_question_is_audited_without_leaking_full_text(ops_client): + ops_client.post( + "/api/v1/knowledge/questions", + json={"question": "What must I do when a vehicle returns with damage?"}, + ) + events = ops_client.get( + "/api/v1/audit", params={"action": "knowledge_question_asked"} + ).json() + assert len(events) >= 1 + metadata = events[0]["metadata"] + assert "evidence_state" in metadata + assert "source_ids" in metadata + assert "question" not in metadata + + +def test_knowledge_status_endpoint(ops_client): + response = ops_client.get("/api/v1/knowledge/status") + assert response.status_code == 200 + assert response.json()["provider"] == "demo" + + +def test_ragcore_provider_degrades_to_unavailable(monkeypatch): + def fake_client(*args, **kwargs): + raise httpx.ConnectError("no ragcore in this environment") + + provider = RAGcoreKnowledgeProvider() + monkeypatch.setattr(provider, "_client", fake_client) + answer = provider.ask("Anything?", "test-correlation-3") + assert answer.evidence_state == "unavailable" + assert answer.sources == [] diff --git a/frontend/src/App.tsx b/frontend/src/App.tsx index 22968c5..8f5b257 100644 --- a/frontend/src/App.tsx +++ b/frontend/src/App.tsx @@ -11,6 +11,7 @@ import { BookingDetail } from "./pages/BookingDetail"; import { DataQuality } from "./pages/DataQuality"; import { DataQualityIssueDetail } from "./pages/DataQualityIssueDetail"; import { Automation } from "./pages/Automation"; +import { Knowledge } from "./pages/Knowledge"; import { Audit } from "./pages/Audit"; export function App() { @@ -33,6 +34,7 @@ export function App() { } /> } /> } /> + } /> } /> } /> diff --git a/frontend/src/api/types.ts b/frontend/src/api/types.ts index 84b21fa..2d74b64 100644 --- a/frontend/src/api/types.ts +++ b/frontend/src/api/types.ts @@ -167,6 +167,32 @@ export interface MergeCustomersResult { rewired_bookings: number; } +export interface SourceCard { + document_id: string; + title: string; + version: string; + section: string; + excerpt: string; +} + +export interface GroundedAnswer { + answer: string; + evidence_state: "grounded" | "insufficient" | "unavailable"; + sources: SourceCard[]; + provider: string; + correlation_id: string; +} + +export interface KnowledgeHealth { + provider: string; + available: boolean; + detail: string; + tenant: string; + workspace: string; + collection: string; + document_count: number; +} + export interface AuditEvent { id: string; actor_type: string; diff --git a/frontend/src/components/Layout.tsx b/frontend/src/components/Layout.tsx index 0d47261..865529f 100644 --- a/frontend/src/components/Layout.tsx +++ b/frontend/src/components/Layout.tsx @@ -6,6 +6,7 @@ const NAV_ITEMS = [ { to: "/vehicles", label: "Vehicles" }, { to: "/bookings", label: "Bookings" }, { to: "/data-quality", label: "Data Quality" }, + { to: "/knowledge", label: "Knowledge" }, { to: "/automation", label: "Automation" }, { to: "/audit", label: "Audit" }, ]; diff --git a/frontend/src/pages/Knowledge.tsx b/frontend/src/pages/Knowledge.tsx new file mode 100644 index 0000000..5967a0c --- /dev/null +++ b/frontend/src/pages/Knowledge.tsx @@ -0,0 +1,121 @@ +import { useEffect, useState, type FormEvent } from "react"; +import { api, ApiError } from "../api/client"; +import type { GroundedAnswer, KnowledgeHealth } from "../api/types"; + +interface Exchange { + question: string; + answer: GroundedAnswer; +} + +const EVIDENCE_LABEL: Record = { + grounded: "Grounded in cited procedures", + insufficient: "Insufficient evidence", + unavailable: "Knowledge service unavailable", +}; + +export function Knowledge() { + const [status, setStatus] = useState(null); + const [question, setQuestion] = useState(""); + const [submitting, setSubmitting] = useState(false); + const [error, setError] = useState(null); + const [exchanges, setExchanges] = useState([]); + + useEffect(() => { + api + .get("/api/v1/knowledge/status") + .then(setStatus) + .catch(() => setStatus(null)); + }, []); + + async function handleSubmit(e: FormEvent) { + e.preventDefault(); + if (!question.trim()) return; + setError(null); + setSubmitting(true); + try { + const answer = await api.post("/api/v1/knowledge/questions", { question }); + setExchanges((prev) => [{ question, answer }, ...prev]); + setQuestion(""); + } catch (err) { + setError(err instanceof ApiError ? err.message : "Could not reach the knowledge service."); + } finally { + setSubmitting(false); + } + } + + return ( +
+

Knowledge

+ {status && ( +

+ Provider: {status.provider} ·{" "} + {status.available ? "available" : "unavailable"} · {status.document_count} procedures + indexed +

+ )} + +
+

Ask a procedure question

+ +
+ setQuestion(e.target.value)} + placeholder="e.g. What must I do when a vehicle returns with damage?" + minLength={3} + maxLength={1000} + required + /> + +
+ {error &&

{error}

} +
+ + {exchanges.length === 0 && !error && ( +

Ask a question about one of the ten operational procedures to see cited sources.

+ )} + +
    + {exchanges.map((exchange, index) => ( +
  • +

    + Q: {exchange.question} +

    +

    + {EVIDENCE_LABEL[exchange.answer.evidence_state]} +

    + + {exchange.answer.evidence_state === "unavailable" ? ( +

    + The knowledge service is currently unreachable. Operational features are + unaffected — try again later. +

    + ) : ( +

    {exchange.answer.answer}

    + )} + + {exchange.answer.sources.length > 0 && ( +
      + {exchange.answer.sources.map((source) => ( +
    • +

      + {source.title} v{source.version} +

      +

      {source.section}

      +

      {source.excerpt}

      +
      • + ))} +
    + )} +
    • + ))} +
+
+ ); +} diff --git a/frontend/src/styles.css b/frontend/src/styles.css index 428005e..e60c03e 100644 --- a/frontend/src/styles.css +++ b/frontend/src/styles.css @@ -226,6 +226,33 @@ a { color: #1f5c8f; } } .data-table td button:disabled { opacity: 0.6; cursor: not-allowed; } +.knowledge-status { color: #607084; font-size: 0.9rem; margin-bottom: 16px; } +.knowledge-form { margin-bottom: 20px; } +.knowledge-input-row { display: flex; gap: 10px; flex-wrap: wrap; } +.knowledge-input-row input { + flex: 1; min-width: 240px; padding: 10px 12px; border: 1px solid #cfd8e2; + border-radius: 8px; font-size: 0.95rem; +} +.knowledge-input-row button { + padding: 10px 20px; border-radius: 8px; border: none; + background: #14324f; color: white; font-weight: 700; cursor: pointer; +} +.knowledge-input-row button:disabled { opacity: 0.6; cursor: not-allowed; } + +.exchange-list { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: 16px; } +.exchange-question { font-size: 1rem; margin: 0 0 6px; } +.evidence-state { display: inline-block; margin: 0 0 10px; padding: 3px 10px; border-radius: 999px; font-weight: 700; font-size: 0.8rem; } +.evidence-grounded { background: #e6f5ec; color: #1f6d3d; } +.evidence-insufficient { background: #fdf1de; color: #8a5a10; } +.evidence-unavailable { background: #fbe6e6; color: #8f2323; } + +.source-cards { list-style: none; margin: 12px 0 0; padding: 0; display: grid; gap: 10px; grid-template-columns: repeat(auto-fit, minmax(260px, 1fr)); } +.source-card { background: #f6f8fb; border: 1px solid #dce3eb; border-radius: 10px; padding: 12px; } +.source-title { margin: 0; font-weight: 700; } +.source-version { color: #607084; font-weight: 400; font-size: 0.85rem; } +.source-section { margin: 2px 0 6px; color: #375065; font-size: 0.85rem; font-weight: 600; } +.source-excerpt { margin: 0; font-size: 0.88rem; color: #47566b; } + @media (max-width: 700px) { .app-header { flex-direction: column; align-items: flex-start; } .user-badge { margin-left: 0; }