Customer Support Bot
Answer support questions from permitted evidence and safely hand off or execute authorized account actions with traceable outcomes.
Answer support questions from permitted evidence and safely hand off or execute authorized account actions with traceable outcomes. This worked design targets SDE-3; the scale figures below are exercise assumptions, not claims about a named company's implementation.
Clarify the prompt
| Ask the interviewer | Decision to record |
|---|---|
| Answers only or actions such as refunds and cancellation? | Actions require separate authorization and confirmation rules. |
| Which knowledge sources and tenants, and what evidence freshness is required? | |
| When must the bot hand off to a person? | Define low-confidence and high-impact triggers. |
Requirements and scope
The core deliverable is: Answer support questions from permitted evidence and safely hand off or execute authorized account actions with traceable outcomes. The first design implements the APIs below and the state constraints in the data model. Agree the latency, freshness, retention and failure policy before adding infrastructure. Every successful acknowledgment must have an explicit meaning: accepted intent, committed state or completed external work. These are separate milestones.
Capacity and bottleneck reasoning
Assume 100,000 chats/day, ten turns/chat: about 11.6 requests/s average and 116/s peak. At four retrieved 1 KB snippets/turn, context retrieval adds about 4 GB/day before model input/output. Latency and cost follow token lengths and tool calls, not chat count alone.
Keep average and peak demand separate. Add replication, indexes, retries and safety headroom to raw arithmetic before selecting capacity. Load-test the busiest ownership key and recovery traffic as well as uniform happy-path throughput.
APIs and data model
- POST /support/chats/{id}/turns {text,requestKey} -> cited answer or handoff
- POST /support/actions {proposedActionId,confirmationToken} -> action status
- GET /support/actions/{id}; POST /support/handoffs -> caseId
document(id,tenant,acl,version,valid_from,deleted_at)
chunk(id,document_id,version,embedding); conversation(id,tenant,owner)
action(action_id PK,request_key UNIQUE,user,tool,payload_hash,state,provider_result)Authorize resource ownership at the API boundary; do not trust IDs or state supplied by a client. Bind retry identity to the caller and payload for mutating operations. Reject an incompatible replay instead of returning an unrelated stored result.
Component relationships
Read the diagram
Follow the labeled relationship panels; they are separate directed paths, not one global timeline.
Step-by-step design
1. Establish the authority
Authenticate the conversation and retrieve only permitted, current evidence. Hybrid lexical/vector retrieval plus reranking improves recall, but verify document ACLs and deletion status before injecting snippets into context. Store evidence versions with each answer.
2. Make retries and concurrent work safe
Treat retrieved text and user input as untrusted data, not authority to invoke tools. The model proposes a typed action; a deterministic policy layer checks the caller, permitted operation, limits and any required confirmation before execution.
3. Scale and recover the critical path
Deduplicate action identity at the tool boundary and reconcile uncertain results. Evaluate answer grounding, refusal/handoff behavior and action correctness separately. When evidence is missing or the policy cannot establish authority, hand off with a redacted trace and existing action status.
Worked request and concurrency example
A customer asks for a refund and a retrieved article contains an injected instruction to ignore limits. The article supplies evidence only. The action policy loads the authenticated order, verifies refund eligibility and requires the approved confirmation. Action R uses one provider key. If the provider times out after accepting, the bot reports pending reconciliation and does not promise a second refund. A human receives R and its status rather than repeating the operation.
Failure and recovery walkthrough
A stale chunk contains another tenant's private policy. Pre-retrieval tenant filtering plus authoritative post-retrieval ACL checking must exclude it. Deleting an embedding later is insufficient if a cached answer still contains the text; cache scope and invalidation must follow document permissions.
Read the diagram
This numbered trace follows one scenario; explain the durable evidence at each transition.
Decisions and alternatives
| Decision | Trade-off |
|---|---|
| Alternative 1 | RAG makes changing evidence available without retraining but introduces retrieval and grounding errors. Fine-tuning style does not replace permission checks or current business state. |
| Alternative 2 | Automatic actions reduce handling time but increase impact of mistakes. Begin with answer-only and low-risk actions, then expand under measured policy and outcome controls. |
Operational contract
Monitor citation validity, tenant leakage tests, handoff rate, action failure/uncertainty age, token cost and resolution quality. Canary with human review and test prompt injection, deleted sources and repeated action requests.
Choose alert thresholds from the agreed user-facing objective. Assign an owner for stuck work and a replay/repair procedure that preserves the original business identity. Bound queue depth, concurrency and recovery throughput so an outage does not produce an uncontrolled catch-up burst.
Foundation, senior and staff depth
- Foundation: explain the user action, model, first complete request and the correctness constraint without relying on a list of technologies.
- Senior: defend this lesson's partition/coordination boundary, capacity estimate and failure recovery; explain the cost of the rejected alternative.
- Staff: identify failure domains and operational ownership, stage a rollout with measured guardrails, and state when rollback stops being safe.
Practice before revealing the answer
Spend 45 minutes: 5 clarifying scope, 7 on capacity/APIs/state, 12 on the first architecture, 15 on the concurrency and failure examples, and 6 on trade-offs and operations. Draw your own design before comparing it with this chapter.
Question: Can model confidence authorize an account action?
Show answer and explanation
Answer: No. Authorization comes from authenticated identity, current resource state and deterministic policy. Confidence may influence whether to answer or hand off, but cannot grant permissions or override transaction limits.
Verification and rollout
Reproduce the worked example using two concurrent actors or a repeated request where applicable. Inject the stated failure between the durable transition and its acknowledgment; verify the resulting state and user-visible outcome. For read/ML paths, test missing or stale dependencies and verify the documented fallback still enforces permissions. Use a small canary, compare correctness and user-facing latency with the baseline, and roll back when those guardrails regress. Do not use an average throughput number as evidence that a tail-latency target is met.
Primary technical reference
Official documentation for the mechanism used in this exercise. This lesson is an original interview design, not a description of undisclosed company internals.
Related study
Before this design, review these mechanisms and identify which owns the critical state transition:
Use the Interview Playbook to adapt the depth and timing, then return to the question breakdown overview.
Shared foundation: Security and Tenant Isolation
Security and Tenant Isolation: Enforce trusted tenant, object and field permissions across APIs, caches, projections and asynchronous work. Work its failure history before extending this design.