Learning pathsPricingChecking sign-in…
← ML curriculum

AIF-002 · FOUNDATION

Context windows and context engineering

Context engineering is the application layer that decides what the model can see. It combines versioned instructions, selected history, authorized retrieval and tool results under a token budget while preserving trust labels and provenance.

Author self-review; independent technical review pending. Synthetic deterministic examples; no model API, benchmark or production deployment.

Prerequisites and scoped previews

LLMs, transformers and token generation

Build a useful mental model of tokenization, transformer blocks, self-attention and autoregressive generation.

What, when, why and how

Context is a versioned request artifact

what
Context includes instructions, user input, selected conversation state, retrieved evidence, tool results and output constraints.
when
Use it in RAG, assistants, coding agents and long-running conversations.
why
Implicit assembly causes stale memory, injection, missing provenance and irreproducible answers.
how
Record ordered segments, source/version IDs, trust labels, token counts, policy scope and assembly version.

The context window is a shared budget

what
Instructions, history, evidence, tool results and output reservation compete for model-specific capacity.
when
Use this before retrieval, summarization or multi-turn history growth.
why
Naive truncation can remove the question or safety instructions and may cause rejection.
how
Reserve output and overhead first, allocate segment quotas, then trim by explicit priority and record omissions.

Relevance and trust outrank volume

what
Selection should maximize useful authorized evidence while preserving provenance and distinguishing source text from instructions.
when
Use it for enterprise RAG, memory and repository context.
why
More passages can add noise, conflicting versions or malicious instructions.
how
Apply current ACLs before hydration, deduplicate, rerank, preserve source spans and recheck permissions before delivery.

1. Calculate the usable input budget

Synthetic exercise: input limit 8,192 tokens; reserve 1,000 for output, 192 for provider/serialization margin and 500 for fixed instructions. Remaining working budget = 8,192 − 1,000 − 192 − 500 = 6,500 tokens. These are exercise assumptions; actual limits and accounting vary by model/provider.

Allocate quotas: current user turn 500, recent history 1,000, retrieved evidence 4,000 and tool results 1,000. The sum is 6,500. A policy may reallocate unused capacity, but it must not silently consume protected instruction or output reserves.

SegmentTarget tokensOverflow policy
Current user turn500Do not silently truncate; ask to narrow
Recent history1,000Summarize older turns with provenance
Retrieved evidence4,000Rerank, deduplicate, drop low utility
Tool results1,000Project allowlisted fields and paginate

2. Assemble context in a stable order

Authenticate principal, resolve tenant scope, load current policy, select conversation state, retrieve candidate IDs, apply authorization before hydrating content, rerank authorized evidence and pack segments. Recheck authorization near delivery to catch revocation races.

Persist a context manifest: request ID, model/tokenizer and assembly-policy versions, ordered segment/source IDs, revisions, trust labels, token counts and dropped-item reasons. Do not retain raw sensitive prompts by default; retention is a deliberate governance decision.

VISUAL WALKTHROUGH

See the system in motion

Watch authorized evidence and conversation state become bounded, provenance-preserving model context.

Process overviewAvailable even when interactive playback cannot load.
  1. Authenticate principal and tenant

  2. Resolve current permissions

  3. Select relevant history and evidence

  4. Deduplicate and rank authorized passages

  5. Allocate token budget by segment

  6. Delimit untrusted evidence

  7. Persist the context manifest

Loading interactive walkthrough…

3. Handle overflow without corrupting meaning

Never truncate the serialized prompt blindly. Preserve required instructions and the current question; summarize older history, rerank evidence, remove duplicates and reduce tool output to allowlisted fields. Validate the final serialized request after packing.

Summaries are lossy derived data. Store source turn range, summary version and known omissions; do not present a summary as a verbatim transcript. Retrieve authoritative records for high-impact decisions.

4. Trust boundaries and degraded behavior

Documents, uploads, web pages, code and tool output remain untrusted data. Delimit and label them, but enforce authorization and tool policy outside the model. Prompt wording is defense-in-depth, not a security boundary.

If retrieval or memory is unavailable, answer only from available authorized input, ask for clarification or abstain. Never widen tenant scope or silently reuse stale context. Deny revoked content immediately through authoritative checks while asynchronous index/cache cleanup completes.

Diagrams and relationships

Budgeted context assembly

Authorization and trust labels are enforced before the bounded context reaches the model.

Scroll horizontally on smaller screens, or enlarge the diagram.

Budgeted context assemblyAuthorization and trust labels are enforced before the bounded context reaches the model. Instructions + user to Rank + deduplicate: trusted policy. History + memory to Rank + deduplicate: selected state. Authorized evidence to Rank + deduplicate: ACL-filtered. Rank + deduplicate to Token budget + manifest: priority + tokens. Token budget + manifest to Model request: bounded context.trusted policyselected stateACL-filteredpriority + tokensbounded contextInstructions + userHistory + memoryAuthorized evidenceRank + deduplicateToken budget + manifestModel request
Budgeted context assemblyAuthorization and trust labels are enforced before the bounded context reaches the model. Instructions + user to Rank + deduplicate: trusted policy. History + memory to Rank + deduplicate: selected state. Authorized evidence to Rank + deduplicate: ACL-filtered. Rank + deduplicate to Token budget + manifest: priority + tokens. Token budget + manifest to Model request: bounded context.trusted policyselected stateACL-filteredpriority + tokensbounded contextInstructions + userHistory + memoryAuthorized evidenceRank + deduplicateToken budget + manifestModel request

Engineer, Senior and Staff follow-ups

These are teaching lenses. Each answer changes the scope of reasoning; they are not official employer leveling criteria.

Engineer: Why reserve output tokens before packing evidence?

Input capacity is shared with generated output and provider overhead. Without reservation, generation may be rejected or leave insufficient room.

Engineer: What belongs in a context manifest?

Ordered segment/source references, revisions, trust labels, tokenizer/model and policy versions, token counts and omission reasons.

Senior: Why filter permissions before content hydration?

It prevents unauthorized text from reaching rerankers, logs or model context; recheck current permissions to handle revocation races.

Senior: How do you avoid truncating safety instructions?

Treat required instructions as protected segments, validate the final serialization and fail closed or reduce lower-priority segments.

Staff: How can context be reproducible without retaining raw prompts?

Keep immutable source IDs, versions, ordering, counts, policy versions and privacy-safe hashes; raw content retention requires explicit policy.

Staff: How do you support multiple tenants and models?

Use tenant-scoped authorization/storage, per-tenant quotas, model-specific tokenizers and budgets, and isolated cache keys.

Practice and self-review

Your practice draft

Assemble a support-assistant context with an 8,192-token limit, 1,000-token output reservation and four competing sources. Define budget, truncation, trust boundary and recovery.

Saved locally on this device. The worked answer does not score your draft. Existing lesson notes and completion are separate.

Progressive hints

Hint 1

Write the assumptions and contract before selecting a framework.

Hint 2

Trace a normal request and one failure or adversarial case.

Hint 3

Separate measured facts from synthetic exercise values and assign policy ownership.

Reveal worked answer guidance

A strong solution to this exercise makes the relevant boundary explicit, pins versions, validates inputs and outputs, handles failure without silently widening permissions, and defines evidence that would prove the behavior. Assemble a support-assistant context with an 8,192-token limit, 1,000-token output reservation and four competing sources. Define budget, truncation, trust boundary and recovery.

Evidence to check in your answer

  • Contracts and assumptions are explicit.
  • A concrete worked example is correct.
  • Failure and degraded behavior are addressed.
  • Security, evaluation and ownership are explicit.

For each item, record Missing, Prompted, Independent or Adaptive evidence. Adaptive means you can revise the reasoning under a changed constraint. Reading completion and readiness are separate.

Sources and verification scope

Original synthetic examples and reference calculations. Sources support narrowly scoped background; they do not establish production performance or independent chapter review.

Understanding and counting tokens ↗ · Tokenization, model-specific counts, full request accounting and context/output limits; exact API fields depend on endpoint.

Context windows ↗ · Context-window capacity, input/output accounting and context management concepts; sizes and behavior are model-specific.

Primary background links checked during authoring on 2026-10-19. Arithmetic, temporal eligibility, maturity and split reference functions are checked by the bounded local fixture suite. SQL is explanatory, not a migrated production query. No GPU, external dataset, model API or training job is required.