LeetCode

Interview scope and guarantees

Browse problems, submit source code, execute hidden tests, return verdicts, and rank contest participants. Untrusted code must not access host resources, credentials, other submissions, or the network. Define CPU, memory, output, and wall-time limits separately.

Capacity worksheet

Assume 100,000 contestants submit once every 2 minutes: about 833 submissions/s. At 2 CPU-seconds/submission the workload consumes about 1,667 fully utilized CPU cores; headroom, startup and heavy languages increase the requirement. Polling 100,000 users every 5 seconds creates 20,000 status reads/s, motivating cached verdicts and backoff.

Concrete API contract

POST /submissions {problemId,language,source,contestId?} -> 202 {submissionId}
GET /submissions/{id} -> {state,verdict,runtime}
GET /contests/{id}/leaderboard?cursor=...
POST /submissions/{id}/cancel

Data model and access paths

submissions(id PK,user_id,request_key,request_hash,problem_version,source_key,state,generation); UNIQUE(user_id,request_key)
results(submission_id UNIQUE,judge_version,verdict,cpu_ms,memory_bytes)
first_solves(contest_id,user_id,problem_id,accepted_at); PK(contest_id,user_id,problem_id)
contest_scores(contest_id,user_id,score,penalty,version); PK(contest_id,user_id)
execution_leases(submission_id PK,generation,expires_at)

Evolve a solution and explain each change

Figure — Three architecture decisions for LeetCode, including the pressure each introduces.

Step 1: Run one submission

Persist code and return a submission ID before execution. Executing untrusted code inside the API process is unsafe.

Step 2: Use isolated workers

Compile and run in restricted sandboxes with CPU, memory, time, and output limits. Workers can crash and queues can deliver the same submission twice.

Step 3: Fence attempts

Lease jobs with attempt tokens and publish a result only for the current attempt. A stale worker must not overwrite a newer accepted result.

Responsibility overview

Figure — Connected responsibilities for LeetCode. Trace the authoritative and derived paths separately.

Worked end-to-end scenario

A student submits code and receives s42. The queue delivers s42 to worker W1, which leases attempt 1. W1 stalls after compiling; its lease expires and W2 starts attempt 2. W1 eventually reports success. The result update includes attempt 1, so it is rejected as stale. W2 runs the fixed test-set version under resource limits and commits the authoritative result for attempt 2. The interface polls or receives a notification after commit. Hidden-test contents remain inside the execution boundary; compiler output is capped and escaped before display.

Why these access paths matter

Submission reads use the student and submission ID; execution uses a pinned problem/test-set version and language image. Index queued or expired attempts by availability time. The queue message names durable state, not the only copy of code. Record compile errors separately from resource-limit failures and incorrect answers so the same verdict does not conceal different mechanisms.

Build the baseline first

1. Submit API → submission and outbox → work queue.
2. Runner → isolated sandbox → versioned tests.
3. Result commit → status read → user.

Evolve the design under load

1. Fair queues → language pools → bounded sandbox slots.
2. Result events → score projector → sorted ranking.
3. Immutable source and tests → object storage.

Defend the hardest decision

Containers share a kernel; a high-assurance judge may use stronger isolation such as microVMs. Apply namespaces, unprivileged identity, syscall restrictions, read-only images, no ambient credentials, disabled egress, and cgroup resource limits. CPU time and wall time catch different abuses. Limit child processes, output size, filesystem usage and compilation time. Test fixtures and expected answers must remain outside user-readable paths.

Failure and recovery analysis

A worker exceeds its lease but continues executing. A new attempt receives a higher generation; the result store accepts only the currently valid generation. Killing the old sandbox limits cost, while fencing prevents stale results from winning. A contest score update must be idempotent by accepted result identity, not by blindly incrementing a score after each queue delivery.

Security and privacy boundary

Never execute learner source in the web server process. Separate judge workers from production secrets and test sandbox escape scenarios before enabling real execution.

Interview follow-ups with reasoning

Question: Why does a Redis sorted set not solve all ranking?

Question: How do you absorb contest bursts?

Question: What if the judge version changes?

Operate and verify the design

Queue wait by language, sandbox startup, resource-limit kills and stale-result rejections.

Pause a runner past lease expiry, start a new attempt and verify that the old result cannot win.

Compare alternatives

Design workshop: judging and ranking are different pipelines

Core flows are problem lookup, source submission, judged result and contest leaderboard. A production editor and interactive debugger are extensions. Assume result status is observable within five seconds for small tests under admitted load, leaderboard freshness within five seconds and no stale worker can publish a final verdict. A timeout is a verdict only under the pinned execution policy, not a queue-delay measurement.

Store source and signed immutable test manifests in object storage. The admission transaction creates submission(user_id,request_key,request_hash,problem_version,source_key), unique by user/request, and an outbox event. A retry returns the original submission; a changed source under the same key returns 409. Workers fetch the pinned language image/test version. Compile, execute and compare are separate phases with separate time/memory/output budgets. Compilation failure returns a sanitized diagnostic; expected answers never enter user-readable paths. A trusted comparator reads capped output after the sandbox stops. It cannot run an arbitrary contestant-supplied comparator.

Figure — Judge result to durable score projection.

For a 90-minute contest choose: more distinct solved problems wins; equal solve counts use smaller sum of first accepted solve times since contest start; remaining ties use user ID. This is a concrete scenario policy, not a claim about LeetCode's actual scoring. With ten problems and durations at most 5,400 seconds, total penalty is at most 54,000. Define score = solved×54,001 + (54,000−penalty). Redis ZRANGE board 0 99 REV returns higher scores first. Equal numeric scores need an explicit user-ID tie policy; Redis reverse lexicographic member order can be selected or an application tie layer can impose ascending IDs.

# Durable score transaction: result delivery can repeat.
if insert_or_lower_first_solve_time(contest, user, problem, accepted_at):
    solved, penalty = recompute_from_first_solves(contest, user)
    version = increment_score_version(contest, user)
    emit_score_event(user, solved * 54001 + 54000 - penalty, version)
# Projection uses Lua: ignore version <= applied version,
# then ZADD the absolute score and store the version atomically.

Unique(contest,user,problem) prevents repeated accepted submissions increasing solved count. If an earlier accepted result arrives late, conditionally lower first_solve_time and recompute even when the row exists. Score updates carry per-user versions; an old event cannot overwrite a newer absolute score. Contest/problem ownership makes first-solve timing unambiguous. Store accepted submission time rather than result completion time if the product chooses that rule, and state the cutoff behavior for queued work after contest end.

Figure — Leaderboard worked example.

Rebuild a missing leaderboard from first-solves at watermark W, write a new Redis key, replay changes after W and atomically switch the read alias. Database writes and Redis writes are not one transaction; outbox and versioned projection bridge them. At 100,000 viewers polling every five seconds, serve 20,000 reads/s from a cache/replicated read tier rather than aggregate submissions for every read. Partition contests, bound leaderboard pages and report snapshot version. Do not broadcast every score update to every participant unless the product needs it.

Use per-language queues with tenant/user admission quotas and weighted scheduling. Set example budgets such as 2 CPU seconds, 5 wall seconds, 256 MiB, 32 processes and 1 MiB stdout, then tune per problem; these are illustrative limits, not universal safe defaults. Cancellation revokes current attempt ownership and kills the sandbox, but consumed CPU still counts. A stale result fails its conditional commit. Track queue wait separately from compile/run latency.

Exercise: A has an accepted solve at second 500, delivered twice, then an earlier valid solve at second 450 arrives. What changes?

Redis sorted sets documents ranking primitives; first-solve semantics, version checks and rebuilds belong to this application.

Technical references

PostgreSQL transaction isolation and concurrent updates.