Prerequisites and scoped previews
LLMs, transformers and token generation
Build a useful mental model of tokenization, transformer blocks, self-attention and autoregressive generation.
What, when, why and how
Context is a versioned request artifact
- what
- Context includes instructions, user input, selected conversation state, retrieved evidence, tool results and output constraints.
- when
- Use it in RAG, assistants, coding agents and long-running conversations.
- why
- Implicit assembly causes stale memory, injection, missing provenance and irreproducible answers.
- how
- Record ordered segments, source/version IDs, trust labels, token counts, policy scope and assembly version.
The context window is a shared budget
- what
- Instructions, history, evidence, tool results and output reservation compete for model-specific capacity.
- when
- Use this before retrieval, summarization or multi-turn history growth.
- why
- Naive truncation can remove the question or safety instructions and may cause rejection.
- how
- Reserve output and overhead first, allocate segment quotas, then trim by explicit priority and record omissions.
Relevance and trust outrank volume
- what
- Selection should maximize useful authorized evidence while preserving provenance and distinguishing source text from instructions.
- when
- Use it for enterprise RAG, memory and repository context.
- why
- More passages can add noise, conflicting versions or malicious instructions.
- how
- Apply current ACLs before hydration, deduplicate, rerank, preserve source spans and recheck permissions before delivery.
1. Calculate the usable input budget
Synthetic exercise: input limit 8,192 tokens; reserve 1,000 for output, 192 for provider/serialization margin and 500 for fixed instructions. Remaining working budget = 8,192 − 1,000 − 192 − 500 = 6,500 tokens. These are exercise assumptions; actual limits and accounting vary by model/provider.
Allocate quotas: current user turn 500, recent history 1,000, retrieved evidence 4,000 and tool results 1,000. The sum is 6,500. A policy may reallocate unused capacity, but it must not silently consume protected instruction or output reserves.
| Segment | Target tokens | Overflow policy |
|---|---|---|
| Current user turn | 500 | Do not silently truncate; ask to narrow |
| Recent history | 1,000 | Summarize older turns with provenance |
| Retrieved evidence | 4,000 | Rerank, deduplicate, drop low utility |
| Tool results | 1,000 | Project allowlisted fields and paginate |
2. Assemble context in a stable order
Authenticate principal, resolve tenant scope, load current policy, select conversation state, retrieve candidate IDs, apply authorization before hydrating content, rerank authorized evidence and pack segments. Recheck authorization near delivery to catch revocation races.
Persist a context manifest: request ID, model/tokenizer and assembly-policy versions, ordered segment/source IDs, revisions, trust labels, token counts and dropped-item reasons. Do not retain raw sensitive prompts by default; retention is a deliberate governance decision.
VISUAL WALKTHROUGH
See the system in motion
Watch authorized evidence and conversation state become bounded, provenance-preserving model context.
Authenticate principal and tenant
Resolve current permissions
Select relevant history and evidence
Deduplicate and rank authorized passages
Allocate token budget by segment
Delimit untrusted evidence
Persist the context manifest
Loading interactive walkthrough…
3. Handle overflow without corrupting meaning
Never truncate the serialized prompt blindly. Preserve required instructions and the current question; summarize older history, rerank evidence, remove duplicates and reduce tool output to allowlisted fields. Validate the final serialized request after packing.
Summaries are lossy derived data. Store source turn range, summary version and known omissions; do not present a summary as a verbatim transcript. Retrieve authoritative records for high-impact decisions.
4. Trust boundaries and degraded behavior
Documents, uploads, web pages, code and tool output remain untrusted data. Delimit and label them, but enforce authorization and tool policy outside the model. Prompt wording is defense-in-depth, not a security boundary.
If retrieval or memory is unavailable, answer only from available authorized input, ask for clarification or abstain. Never widen tenant scope or silently reuse stale context. Deny revoked content immediately through authoritative checks while asynchronous index/cache cleanup completes.
Diagrams and relationships
Authorization and trust labels are enforced before the bounded context reaches the model.
Scroll horizontally on smaller screens, or enlarge the diagram.
Engineer, Senior and Staff follow-ups
These are teaching lenses. Each answer changes the scope of reasoning; they are not official employer leveling criteria.
Engineer: Why reserve output tokens before packing evidence?
Input capacity is shared with generated output and provider overhead. Without reservation, generation may be rejected or leave insufficient room.
Engineer: What belongs in a context manifest?
Ordered segment/source references, revisions, trust labels, tokenizer/model and policy versions, token counts and omission reasons.
Senior: Why filter permissions before content hydration?
It prevents unauthorized text from reaching rerankers, logs or model context; recheck current permissions to handle revocation races.
Senior: How do you avoid truncating safety instructions?
Treat required instructions as protected segments, validate the final serialization and fail closed or reduce lower-priority segments.
Staff: How can context be reproducible without retaining raw prompts?
Keep immutable source IDs, versions, ordering, counts, policy versions and privacy-safe hashes; raw content retention requires explicit policy.
Staff: How do you support multiple tenants and models?
Use tenant-scoped authorization/storage, per-tenant quotas, model-specific tokenizers and budgets, and isolated cache keys.
Practice and self-review
Your practice draft
Assemble a support-assistant context with an 8,192-token limit, 1,000-token output reservation and four competing sources. Define budget, truncation, trust boundary and recovery.
Saved locally on this device. The worked answer does not score your draft. Existing lesson notes and completion are separate.
Progressive hints
Hint 1
Write the assumptions and contract before selecting a framework.
Hint 2
Trace a normal request and one failure or adversarial case.
Hint 3
Separate measured facts from synthetic exercise values and assign policy ownership.
Reveal worked answer guidance
A strong solution to this exercise makes the relevant boundary explicit, pins versions, validates inputs and outputs, handles failure without silently widening permissions, and defines evidence that would prove the behavior. Assemble a support-assistant context with an 8,192-token limit, 1,000-token output reservation and four competing sources. Define budget, truncation, trust boundary and recovery.
Evidence to check in your answer
- Contracts and assumptions are explicit.
- A concrete worked example is correct.
- Failure and degraded behavior are addressed.
- Security, evaluation and ownership are explicit.
For each item, record Missing, Prompted, Independent or Adaptive evidence. Adaptive means you can revise the reasoning under a changed constraint. Reading completion and readiness are separate.
Sources and verification scope
Original synthetic examples and reference calculations. Sources support narrowly scoped background; they do not establish production performance or independent chapter review.
Understanding and counting tokens ↗ · Tokenization, model-specific counts, full request accounting and context/output limits; exact API fields depend on endpoint.
Context windows ↗ · Context-window capacity, input/output accounting and context management concepts; sizes and behavior are model-specific.
Primary background links checked during authoring on 2026-10-19. Arithmetic, temporal eligibility, maturity and split reference functions are checked by the bounded local fixture suite. SQL is explanatory, not a migrated production query. No GPU, external dataset, model API or training job is required.