YouTube

Interview scope and guarantees

Upload video, process playable renditions, retrieve metadata, and stream with adaptive quality. Search and recommendations are separate extensions. A video becomes playable only when a valid manifest references ready segments; processing failure should be visible to the uploader.

Capacity worksheet

Assume 100,000 uploads/day at 200 MB each: 20 TB/day input. Three output renditions totaling 2× input add 40 TB/day. One million concurrent viewers at 3 Mbps require 3 Tbps of delivery, so CDN distribution matters far more than scaling metadata API instances.

Concrete API contract

POST /videos/uploads {requestKey,size,checksum} -> 201 {videoId,uploadId,partUrls,expiresAt}
POST /uploads/{uploadId}/complete {parts,checksum} -> 202 {videoId,state:"processing"}
POST /videos/{id}/publish {expectedVersion} -> 202 {processingState}
GET /videos/{id} -> {metadata,manifestUrl}
GET /videos/{id}/processing -> {renditions,state}

Data model and access paths

videos(video_id PK,owner_id,state,source_key,visibility,version)
transcode_tasks(video_id,profile,generation,state,output_prefix)
manifests(video_id,version,segment_keys,checksum)
rights(video_id,viewer_scope,expires_at)
uploads(upload_id PK,video_id,owner_id,request_key,request_hash,state,expires_at); UNIQUE(owner_id,request_key)

Evolve a solution and explain each change

Figure — Three architecture decisions for YouTube, including the pressure each introduces.

Step 1: Store and serve video

Upload an original object and create metadata without marking it playable. One raw codec cannot serve every device or network.

Step 2: Transcode asynchronously

Generate a pinned set of renditions and publish their manifests after verification. Partial outputs and duplicate workers can create invalid playback states.

Step 3: Deliver adaptively

Use a private origin and CDN; clients select segment renditions based on buffer and bandwidth. A CDN improves delivery, not transcoding capacity or access control.

Responsibility overview

Figure — Connected responsibilities for YouTube. Trace the authoritative and derived paths separately.

Worked end-to-end scenario

A creator completes upload v12. Metadata becomes processing, and an outbox schedules transcoding for source version 1. Workers write segments to immutable output keys. Only a verified manifest referencing available segments becomes playable. If a worker crashes after writing the 720p rendition, retry resumes or replaces that attempt without publishing an incomplete manifest. A viewer begins at a modest bitrate; if bandwidth falls, the next segment can use a lower rendition. Removing a video updates visibility immediately at the authorization boundary while physical deletion and cached-byte reclamation follow a stated policy.

Why these access paths matter

Video metadata uses video ID and owner indexes. Renditions are keyed by video version, codec/profile, and segment identity. Processing attempts carry a fencing token; manifest publication checks the current source and attempt. Popular segments benefit from edge caching, while rare originals need not occupy every edge. Track segment errors separately from successful metadata reads.

Build the baseline first

1. Uploader → object storage → verified source.
2. Job queue → transcoder → immutable segments.
3. Manifest publication → CDN → player.

Evolve the design under load

1. Profile queues → bounded GPU/CPU workers.
2. Popular segments → edge cache → origin shield.
3. Playback telemetry → quality aggregates → capacity decisions.

Defend the hardest decision

Adaptive bitrate playback requests short segments at a suitable quality based on measured throughput and buffer health. Aligned keyframes make switching renditions practical. Transcoding is asynchronous and independently retryable per profile or chunk. Publish a new manifest atomically only after every referenced required output is available. Keeping immutable segment URLs lets caches serve them safely without invalidation on every metadata change.

Failure and recovery analysis

A transcoder crashes after writing some outputs. Retry into an attempt-specific prefix, verify the finished set, then commit the winning manifest generation. Collect orphan attempts later. During an origin outage, cached segments may continue playing, but cache misses fail; this is not full availability. A rights revocation requires an authorization policy consistent with URL and token lifetimes.

Security and privacy boundary

Scan uploads, enforce content ownership and viewing permissions, and limit signed playback token lifetime.

Interview follow-ups with reasoning

Question: How do you reduce startup latency?

Question: What changes for live video?

Question: How do you manage processing cost?

Operate and verify the design

Time to first playable rendition, transcode retry age, startup delay and rebuffer ratio.

Kill a transcoder after partial output and verify no published manifest references missing segments.

A second scenario to test transfer

A video has 360p and 720p outputs ready, while 1080p repeatedly fails. If the product permits partial readiness, publish the two working renditions and retry the failing profile independently. The player should not advertise unavailable segments. Track time-to-first-playable separately from time-to-all-renditions so operators see the user impact of processing delays.

What should happen when one rendition fails but lower qualities are valid?

Compare alternatives

A design-changing exercise

All but one required segment exists. Should the manifest be advertised as ready?

Design workshop: bytes, processing, and playable publication

Upload, process, publish and watch are core. Recommendations and live broadcasting are excluded. Choose ten-minute first-playable processing for an illustrative small upload target, playback startup p95 under two seconds and immutable published segments. Processing deadlines depend on admitted media size and queue backlog; do not promise one bound for arbitrarily large files.

POST /videos/uploads returns both videoId and uploadId, scoped signed part URLs and expiry. Completion verifies part list, content identity and supported checksums. The transaction then marks uploaded and emits a transcode intent. A publish request records visibility intent; the video becomes playable only when the required baseline rendition and manifest are verified. A retried complete returns the original processing identity, not a new transcode generation.

The DAG is probe→validated source→audio/video rendition jobs→segment verification→manifest publication. Pin codec settings, source version and generation. Rendition tasks have unique(video,source_version,profile,generation). Publish via compare-and-swap against the current generation after checking every advertised segment. A stale worker can leave orphan bytes for GC, but cannot replace a newer manifest.

Figure — First playable before every rendition is ready.

A minimal VOD media playlist for a six-second segment is:

#EXTM3U
#EXT-X-TARGETDURATION:6
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:6.0,
segments/g3/360p/0000.ts
#EXTINF:6.0,
segments/g3/360p/0001.ts
#EXT-X-ENDLIST

The variant playlist advertises only verified renditions with bandwidth and resolution. Align switching boundaries and keyframes across renditions; sequence numbers alone do not establish identical media time. The player estimates bandwidth and buffer and requests a lower rendition before underrun. The API handles authorization and metadata; it should not proxy every segment byte.

Figure — Ten-minute video sizing at illustrative rates.

One million complete plays at average 2 Mb/s for ten minutes transfer 150 TB, before transport overhead. Delivery bandwidth can dominate storage cost. A private CDN validates signed viewer/video/version grants, caches immutable segments under safe keys, and uses an origin shield to coalesce cold fetches. A viewer without a grant cannot fetch an origin object directly. Public content can use longer TTL; revoking a private grant has an explicit lifetime bound unless the edge checks current policy.

A task failure retries just that profile with bounded exponential backoff, then enters failed_profile rather than retrying the entire video forever. Delete cleanup waits for retention and live manifest references; GC does not delete current rendition bytes because one old task failed. Monitor queue age, source→first-playable time, profile failures, manifest errors, startup latency and rebuffer ratio separately.

Exercise: 1080p segments exist but one segment checksum fails. May the manifest list 1080p because the directory exists?

HTTP Live Streaming, RFC 8216 defines playlist/segment semantics; the DAG and readiness policy here are original application choices.

Technical references

Object-store multipart upload.

PostgreSQL transaction isolation and concurrent updates.