Learning pathsPricingChecking sign-in…
← ML curriculum

AISD-005 · DESIGN PRACTICE

Practice: Real-time voice assistant

Draft the complete decision system before revealing the guidance. Use a 45-minute rehearsal budget and adapt it to the interviewer’s direction.

Preserve eligibility, available-at-decision inputs, mature labels, bounded work and compatible releases. Explain alternatives and test a normal, boundary and failure case.

Your practice draft

Design a real-time voice assistant for mobile and browser clients. Support duplex audio, streaming ASR, natural turn-taking, barge-in, safe tool calls and streaming TTS. Explain latency budgets, session state, provider failure, privacy and capacity.

Saved locally on this device. The worked answer does not score your draft. Existing lesson notes and completion are separate.

Progressive hints

Hint 1

Start with media session and separate control/media channels.

Hint 2

Define stage-level time-to-first-audio and end-of-turn SLOs.

Hint 3

Keep ASR partial hypotheses distinct from committed transcript.

Hint 4

Use turn IDs/generations to fence stale LLM and TTS output.

Hint 5

Require deterministic policy and confirmation for side effects.

Hint 6

Bound audio queues and make cancellation/reconnect explicit.

Self-review evidence

For each criterion, record Missing, Prompted, Independent or Adaptive reasoning. No automated scoring or offer prediction is provided.

  • Media and control channels are separated and session identity is server-derived.
  • Latency is budgeted end-to-end, including endpointing and playback.
  • ASR provisional and committed transcript states are distinct.
  • Turn ownership and generation fencing prevent stale audio.
  • Barge-in cancels downstream work and stops playback promptly.
  • Tool calls are typed, authorized, idempotent and confirmed when high impact.
  • TTS streaming and playback buffers are bounded.
  • Provider fallback is stage-specific and does not duplicate side effects.
  • Audio, transcript, consent and retention policies are explicit.
  • Evaluation includes ASR slices, endpointing, interruption, intelligibility and task success.
Reveal worked design guidance

Authenticate a tenant-scoped session and negotiate WebRTC media plus a reliable control channel. Normalize audio and run VAD/streaming ASR, emitting provisional hypotheses and immutable committed segments. The turn manager performs endpointing and assigns turn generations. For a committed turn, assemble bounded context and invoke the LLM under deadline and policy. Validate typed tool proposals through deterministic authorization, idempotency and confirmation. Stream safe text chunks to TTS and bounded playback buffers. On barge-in, advance generation, stop playback, cancel LLM/TTS and reject stale chunks. Persist turn metadata separately from optional audio recordings. Apply stage-specific provider fallback, reconnect cursors, privacy retention, fair admission and per-stage observability.

Open the complete worked design in a new tab →

The new tab keeps this draft visible. Save it on this device before closing the page.