MLSD-023 · DESIGN PRACTICE
Practice: Permission-aware enterprise RAG
Draft the complete decision system before revealing the guidance. Use a 45-minute rehearsal budget and adapt it to the interviewer’s direction.
Preserve eligibility, available-at-decision inputs, mature labels, bounded work and compatible releases. Explain alternatives and test a normal, boundary and failure case.
Your practice draft
Design the enterprise RAG system under the synthetic 20-tenant, 250,000-document, 250-query/s workload. Cover initial backfill and continuous updates, source/version/job/chunk/index/manifest schemas, authorization before hydration, hybrid retrieval, token budgeting, citation validation, revocation, capacity, rollout and recovery. Then change one constraint: the embedding model changes, a source connector loses its change feed, or an ACL is revoked while the model provider is timing out.
Saved locally on this device. The worked answer does not score your draft. Existing lesson notes and completion are separate.
Progressive hints
Hint 1
Start with the decision, tenant/source/policy authorities, freshness and revocation targets, and a deterministic keyword-search baseline.
Hint 2
Draw separate offline and online planes. Define immutable source versions, durable idempotent jobs, staged indexes and the active manifest before selecting a vector database.
Hint 3
Trace a normal update, duplicate/out-of-order event, missed webhook, ACL revoke and provider timeout. State exactly what remains queryable and what must fail closed.
Self-review evidence
For each criterion, record Missing, Prompted, Independent or Adaptive reasoning. No automated scoring or offer prediction is provided.
- Framing: defines decision, users, tenants, sources, freshness, revocation, answer contract and assumptions.
- Data correctness: immutable source/document versions, provenance, ACL references, idempotent jobs, checkpoints and active manifest.
- Production migration: existing corpus backfill, incremental update, reconciliation, safe publication, rollback and online continuity.
- Security: current policy before text hydration, tenant-safe caches, prompt-injection boundary, immediate revocation and deletion verification.
- Retrieval/generation: dense+sparse fusion, bounded reranking, token budget, citation resolution, grounding and abstention.
- Operations: queue isolation, quotas, failure recovery, observability, independent metrics, governance and ownership.
- Evidence discipline: labels synthetic assumptions and distinguishes arithmetic/design from tested production performance.
Reveal worked design guidance
A strong design separates offline source ingestion from online query serving. It snapshots and inventories the existing corpus, runs bounded resumable backfill into a staged index, validates tenant/ACL/provenance/model compatibility and atomically publishes a manifest. New files and updates use idempotent source-version jobs; the old active version remains searchable until the replacement is validated. Reconciliation repairs missed/out-of-order events. Query requests derive tenant identity from verified authentication, resolve current permissions, retrieve bounded candidate IDs, filter and recheck authorization before hydration, then run hybrid retrieval/reranking, context budgeting and guarded generation. Citations are resolved to the exact authorized evidence versions; insufficient or invalid evidence leads to abstention. Revocation writes a durable tombstone and denies immediately while physical cleanup proceeds across indexes and caches. A model migration builds a parallel compatible index and cuts over only after shadow evaluation. Capacity estimates are explicitly synthetic; operational guardrails cover freshness, revocation, latency, cost, retrieval quality and provider failures.
Open the complete worked design in a new tab →The new tab keeps this draft visible. Save it on this device before closing the page.