AI Knowledge Answer Reviewer
A trustworthy question-and-answer workflow that makes source material, uncertainty, feedback, and escalation visible.
The problem
Operations teams need faster access to procedure knowledge, but unexplained AI answers create trust and compliance risks. A plain chatbot response gives no way to verify the source, report a gap, or escalate an edge case.
Target user
Operations staff and customer-service staff who need a reliable answer quickly. Secondary user: the knowledge owner who maintains the source material and resolves gaps.
Product goal
Give users a trustworthy question-and-answer workflow that makes source material, uncertainty, feedback, and escalation visible.
User flow
- 1Select an operational area (Claims, Policy Servicing, Customer Onboarding).
- 2Ask a question, or choose a suggested question.
- 3Read a structured answer: answer, steps, exceptions, and sources.
- 4Open the source citation cards to verify the guidance.
- 5Rate the answer as Useful, Incomplete, or Inaccurate.
- 6If incomplete, create an escalation request with the question and context prefilled.
- 7A knowledge owner reviews feedback trends and resolves unresolved questions.
Design decisions
- Structure beats prose — splitting the answer into answer/steps/exceptions makes it scannable and reduces the risk of an overconfident wall of text.
- Sources are first-class — every answer shows its supporting sources with review dates, so trust is earned rather than asserted.
- Uncertainty is explicit — the "Review Recommended" label appears when the question is not clearly covered, instead of inventing an answer.
- Feedback is a product feature, not a footnote — Useful/Incomplete/Inaccurate feeds the owner dashboard and unlocks escalation.
AI interaction design
Input is KnowledgeAnswerService.answerQuestion(question, category). Output is a structured AiAnswer with steps, exceptions, sources, and a confidence label. The human review loop is: verify sources → rate → escalate if incomplete. Escalations are routed to a knowledge owner, closing the feedback loop.
Technical approach
Next.js App Router with a React client component for the ask flow and a separate owner dashboard route. TypeScript types for KnowledgeSource, AiAnswer, Feedback, and EscalationRequest. Atomic component structure with reusable atoms and feature-scoped components. Mock adapter with simulated latency and an explicit unavailable state. Feedback and escalations are held in lightweight in-memory stores so the owner dashboard reflects real interactions during a session.
Tradeoffs
- No real retrieval layer: the mock service matches on keywords against fictional documents.
- Feedback and escalations are session-scoped rather than persisted.
- The owner dashboard uses a small seed dataset to make trends legible in a demo.
Success measures
- Answer usefulness rating
- Escalation rate and time to resolution
- Time to find guidance
- Unresolved-question volume
- Source coverage
Next steps
Connect a real retrieval system with citations and evaluation, add authentication and role-based access, persist feedback and escalations, and measure answer quality against a labelled question set.