Interview Q/A
Build crisp spoken answers across 55 original questions covering production trade-offs, debugging scenarios, system design, and interview-ready explanations.
Depth
Showing 55 of 55 questions
Short answer
ReAct alternates between reasoning about the current state, selecting an action, executing that action, and feeding the observation back into the next step. The loop ends when the model returns a final answer or the runtime reaches a hard stopping condition.
Interview-ready answer
A production ReAct loop is explicit: build the prompt from the goal and scratchpad, ask the model for either a tool action or final answer, parse and validate the output, execute only allow-listed tools, append a sanitized observation, and repeat within a step or token budget. I would also trace every transition and handle malformed output or tool failures as observations rather than letting the entire run crash.
Example
For a support question, the agent may search documentation, observe that the result is stale, query a status tool, then produce a grounded answer with both observations.
Common mistakes
Short answer
Reactive agents choose the next action from the current observation, plan-and-execute agents create a broader plan before carrying out steps, and reflective agents critique outcomes or trajectories before retrying. Each adds capability but also latency, state, and failure modes.
Interview-ready answer
I choose the simplest pattern that fits the task. Reactive control works well when the next step depends heavily on fresh tool results. Plan-and-execute helps when dependencies and budgets benefit from an explicit sequence, but plans must be revised when observations invalidate assumptions. Reflection can improve quality after a failed attempt, although unrestricted self-critique can add cost without reliable gains, so I bound retries and evaluate whether reflection actually improves outcomes.
Common mistakes
Short answer
An agent stops on a validated terminal result or a deterministic runtime limit such as maximum steps, token budget, deadline, repeated action detection, cancellation, or policy block. The model may propose completion, but application code owns termination.
Interview-ready answer
I use layered stop conditions. A semantic condition accepts a well-formed final result that satisfies the task contract; safety and resource conditions stop on budgets, deadlines, user cancellation, denied actions, or repeated no-progress states. The runtime should return an explicit terminal status and preserve the trace so a caller can distinguish success, partial completion, timeout, and policy refusal.
Common mistakes
Short answer
Prevent loops with hard step, time, token, and cost budgets plus repeated-state detection. Recover by stopping safely, returning the best verified partial result, preserving the trace, and classifying why progress stalled before any bounded retry.
Interview-ready answer
I would combine hard limits with no-progress detection based on repeated tool calls, identical observations, unchanged plans, or oscillation between states. Near the budget, the runtime can request a concise finalization attempt, but it must still enforce the hard stop. Recovery should not blindly restart: classify parsing, tool, context, or planning failure; compact or repair state; retry at most a small number of times; and surface an actionable terminal result with trace and budget usage.
Common mistakes
Short answer
A good schema has one clear purpose, an unambiguous name and description, typed fields, meaningful constraints, explicit required values, and examples only where they reduce ambiguity. It should be easy for both the model and the executor to validate.
Interview-ready answer
I design tool schemas like stable public APIs. Inputs should avoid overlapping semantics, use enums and bounds where possible, distinguish optional fields from nullable values, and describe units and side effects. Smaller task-focused tools are usually easier to select than one broad tool with many modes, but too many near-duplicates can confuse routing, so the catalog also needs coherent naming and descriptions.
Common mistakes
Short answer
Validate arguments against a strict schema, normalize types, enforce semantic constraints and authorization, then validate the returned result against a stable output contract. Treat validation failures as typed observations, not executable instructions.
Interview-ready answer
I use multiple layers: structural schema validation, domain checks such as ranges and resource existence, permission checks tied to the authenticated actor, and policy checks for high-impact actions. Outputs receive the same discipline because external tools can be buggy or compromised. The agent gets a sanitized typed error it can reason about, while logs retain diagnostic detail without leaking secrets.
Common mistakes
Short answer
Classify the failure before acting: retry transient errors with bounded exponential backoff and jitter, repair invalid arguments, choose a fallback for unavailable capabilities, and stop on permanent or unauthorized failures. Every retry must respect idempotency and the overall budget.
Interview-ready answer
I first distinguish timeout, rate limit, dependency outage, invalid input, authentication, and policy failure. Only transient and idempotent operations receive automatic retries, using exponential backoff, jitter, per-call timeouts, and a run-level deadline. For side-effecting tools I use idempotency keys or require confirmation. The final trace should show attempts and the terminal classification so operators can separate tool instability from agent-planning errors.
Common mistakes
Short answer
Reduce overlap, improve names and descriptions, retrieve a small relevant tool subset, use routing examples, and evaluate confusion pairs. Hierarchical routing can select a domain first and a specific tool second without exposing the entire catalog.
Interview-ready answer
I start by fixing the catalog because routing cannot compensate for indistinguishable APIs. Then I use metadata retrieval or a deterministic capability router to present only likely tools, with examples for common confusion pairs. For high-risk actions, selection and execution are separate decisions: the model can propose a tool, while policy and authorization layers approve it. I measure top-k tool recall, wrong-tool rate, argument validity, latency, and behavior as the catalog changes.
Common mistakes
Short answer
Short-term memory supports the active task, long-term memory stores durable facts or preferences across sessions, and episodic memory records specific past interactions or outcomes. They differ in retention, retrieval, structure, and privacy expectations.
Interview-ready answer
I treat short-term memory as a bounded working set, often the recent turns plus summarized state. Long-term memory contains carefully selected durable information such as preferences or stable facts. Episodic memory preserves events with time, context, action, and outcome so similar past situations can be retrieved. In production these stores need separate ranking, freshness, consent, deletion, and tenant-isolation policies.
Common mistakes
Short answer
Store information only when it has future utility, sufficient confidence, user permission, and a defined retention policy. Prefer explicit user preferences and verified outcomes over speculative model summaries or transient conversation details.
Interview-ready answer
I score candidates on usefulness, durability, confidence, sensitivity, and duplication. Important facts carry provenance, timestamps, ownership, and expiration, and users should be able to inspect or delete durable personal memory. I also separate raw events from derived summaries so a future correction can invalidate dependent memory rather than leaving stale conclusions embedded in the system.
Common mistakes
Short answer
Use relevance thresholds, freshness weighting, provenance, expiration, deduplication, and contradiction handling. Retrieved memories should be presented as evidence with confidence and time context, not inserted as unquestioned instructions.
Interview-ready answer
I combine semantic relevance with metadata filters for user, task, recency, source, and status. A reranker can balance relevance against freshness, while TTLs and explicit invalidation remove known stale entries. Contradictory memories should be surfaced or reconciled rather than silently selecting one. Evaluation should include stale and adversarial memory cases because clean benchmark retrieval can hide serious production failures.
Common mistakes
Short answer
Use separate stores for verified profile facts, user preferences, task state, and timestamped episodes. Gate writes through consent and confidence rules, retrieve with strict user isolation, and provide inspection, correction, deletion, and expiration controls.
Interview-ready answer
I would capture candidate memories after each session, classify sensitivity and type, deduplicate them, and require explicit confirmation for uncertain or sensitive facts. Durable records include provenance, version, timestamps, confidence, and deletion state. Retrieval is scoped to the authenticated user and task, then reranked for relevance and freshness before a compact memory view enters the prompt. Auditing and user controls are first-class because memory errors accumulate across sessions.
Common mistakes
Short answer
A basic pipeline ingests and chunks documents, creates searchable representations, retrieves candidates for a query, optionally reranks them, builds a bounded context, generates an answer, and returns citations or evidence.
Interview-ready answer
I divide the system into offline indexing and online serving. Indexing handles parsing, chunking, metadata, embeddings or lexical indexes, versioning, and deletion. Serving rewrites or classifies the query, applies authorization filters, retrieves and reranks candidates, selects context within a token budget, generates with grounding instructions, and records retrieval and answer metrics. Each stage needs observable inputs and outputs for debugging.
Common mistakes
Short answer
Chunking determines the unit that can be found and placed in context, while reranking improves the order of retrieved candidates using a more precise relevance signal. Small chunks improve precision but can lose context; large chunks add context but increase noise and token cost.
Interview-ready answer
I choose chunk boundaries based on document structure and expected questions, then measure retrieval recall at a generous candidate depth. A reranker can recover precision by scoring query-document pairs more carefully than the first-stage retriever, but it adds latency and cost. I tune chunk size, overlap, candidate count, reranker depth, and final context assembly together because optimizing one stage in isolation can reduce end-to-end grounded answer quality.
Common mistakes
Short answer
Faithfulness asks whether claims are supported by supplied evidence, answer relevance asks whether the response addresses the query, context precision measures how much retrieved context is useful, and context recall measures whether required evidence was retrieved.
Interview-ready answer
I evaluate retrieval before generation using labeled relevant documents, recall at k, precision, and ranking metrics. Then I evaluate the answer for supported claims, completeness, citation correctness, and usefulness. Model judges can scale semantic evaluation, but I calibrate them against human labels and keep deterministic checks for citations, access control, and known facts. Segmenting failures prevents a generation fix from masking a retrieval problem.
Common mistakes
Short answer
Correct context does not guarantee that the model attends to it, resolves conflicts, follows grounding instructions, or avoids adding unsupported prior knowledge. Context may also be poorly ordered, overloaded, ambiguous, or vulnerable to injected instructions.
Interview-ready answer
I would inspect whether the needed evidence survived context assembly, where it appears, whether other chunks conflict, and whether the prompt clearly requires evidence-backed claims. Long or noisy context can dilute attention, and retrieved documents can contain prompt injection. Mitigations include better context selection and ordering, claim-level citations, constrained answer formats, explicit abstention, injection isolation, and faithfulness evaluation. The trace must preserve retrieved and final context, not just document IDs.
Common mistakes
Short answer
An agent can reach a correct answer through unsafe, wasteful, unauthorized, or brittle behavior. Final accuracy misses tool choice, argument validity, evidence use, policy compliance, retries, latency, cost, and whether success was reproducible.
Interview-ready answer
I evaluate both outcome and trajectory. Outcome metrics tell me whether the task completed, while trajectory metrics show whether the agent selected appropriate tools, respected permissions and budgets, used observations, and recovered correctly. This matters because a lucky final answer can hide a dangerous action, and an acceptable partial result may follow a safe response to an unavailable dependency. Product decisions need both layers.
Common mistakes
Short answer
Deterministic evaluators apply explicit repeatable rules such as exact match, schema checks, or invariant tests. Model-based evaluators judge semantic or qualitative properties, offering flexibility at the cost of variance, bias, latency, and expense.
Interview-ready answer
I use deterministic checks wherever the property can be stated precisely: valid JSON, required citations, forbidden tools, numeric bounds, or known test cases. Model judges are useful for relevance, coherence, and nuanced rubric scoring. They need calibrated prompts, blind comparisons where possible, repeated sampling for unstable cases, and human review on critical slices. A layered evaluator is easier to trust than asking one judge to decide everything.
Common mistakes
Short answer
Trajectory evaluation should measure action selection, argument validity, observation use, progress, policy compliance, recovery behavior, and resource efficiency. It should allow multiple valid paths while detecting prohibited or ineffective transitions.
Interview-ready answer
I define invariants and milestones rather than one exact trace. For example, a support agent must query an authorized source before claiming account status, must not repeat the same failed action beyond a limit, and must stop after a policy denial. I score tool appropriateness, state transitions, evidence use, recovery, and budgets, then connect failures to the earliest incorrect step. This creates actionable feedback instead of only labeling the final run.
Common mistakes
Short answer
Start from real tasks and failure logs, define important user and risk segments, include normal, boundary, adversarial, and dependency-failure cases, then maintain versioned labels and holdout sets. Coverage should reflect impact, not only frequency.
Interview-ready answer
I build a task taxonomy across intents, tools, languages, user types, difficulty, and safety risk. Production traces reveal common and surprising failures, while synthetic generation expands sparse edge cases but requires review. I prevent leakage between prompt development and holdout sets, version datasets with the product and tool catalog, and report performance by slice. High-impact rare failures deserve explicit weight even if they barely affect the average.
Common mistakes
Short answer
Separate deterministic invariants from probabilistic quality, control test inputs and tool responses, run multiple seeds or samples, and compare distributions with confidence bounds. Reserve exact assertions for behavior the runtime truly guarantees.
Interview-ready answer
Unit tests should stub model and tool boundaries to verify orchestration invariants deterministically. End-to-end evaluations can sample multiple runs and report task success, policy violations, latency, cost, and trajectory scores as distributions. I use thresholds with minimum sample sizes and investigate regressions by slice rather than failing on one random output. Recording model version, prompt version, seed where supported, and full trace makes failures reproducible enough to debug.
Common mistakes
Short answer
Agent observability is the ability to understand a run through structured traces, logs, metrics, inputs, outputs, tool calls, model usage, and state transitions. It supports debugging, evaluation, operations, and cost control.
Interview-ready answer
Unlike a single API request, an agent run can fail across planning, retrieval, tools, state, or policy. I model the run as a trace with spans for model calls, tool calls, retrieval, validation, and approvals, all tied to versions and a correlation ID. Metrics aggregate latency, cost, success, and failure classes, while secure logs retain enough context to reproduce issues without exposing secrets or sensitive user data.
Common mistakes
Short answer
Log a run ID, actor and tenant context, versions, timestamps, model and tool spans, sanitized inputs and outputs, token and cost usage, state transitions, retries, approvals, errors, and final status. Apply redaction and retention policies.
Interview-ready answer
I want enough data to reconstruct control flow without turning logs into a sensitive-data dump. Each event includes correlation IDs, parent span, component version, duration, status, and typed attributes. Prompts, retrieved context, and tool payloads are redacted or referenced through controlled storage. I also log policy decisions and user approvals because they explain why an action was allowed or denied.
Common mistakes
Short answer
Traces preserve ordered spans, inputs, outputs, state changes, timing, and errors, so you can find the earliest incorrect transition rather than blaming the final response. Parent-child relationships show downstream effects.
Interview-ready answer
I compare expected invariants with the trace and locate the first divergence: wrong retrieval, malformed tool arguments, stale state, ignored observation, or policy rejection. Span timing separates slow model calls from tool timeouts, while version metadata shows whether a deployment changed behavior. A good trace viewer also links evaluation feedback to exact events, turning a vague bad answer into a concrete repair target.
Common mistakes
Short answer
Track end-to-end and per-step latency, token and tool cost, task success, groundedness or rubric scores, policy violations, tool error rates, retry counts, timeout rates, abandonment, and availability. Segment metrics by task and risk.
Interview-ready answer
I use percentiles rather than averages for latency and cost, and decompose them by model, retrieval, and tool spans. Quality metrics depend on the task but should include both outcome and trajectory. Reliability includes dependency failures, retries, loops, fallbacks, and terminal statuses. Business or user metrics such as completion and escalation matter too, because a technically valid response may still be unhelpful.
Common mistakes
Short answer
Capture comparable successful and failed traces, control model and tool variability, segment by versions and dependencies, and locate the earliest divergent state. Reproduce with recorded inputs and deterministic tool fixtures before changing prompts.
Interview-ready answer
I first define the failure precisely and query traces by task, model version, tool version, latency, and error class. I diff a failed run against a nearby success to find whether retrieval, planning, tool output, context size, or timing diverged. Then I replay with frozen tool responses and a controlled model stub to isolate orchestration, followed by sampled live-model runs. The fix gets a regression case and a production monitor for recurrence.
Common mistakes
Short answer
Prompt injection is untrusted content attempting to alter model behavior or override instructions. Tool-using agents are more exposed because a successful injection can influence real actions, data access, messages, or transactions.
Interview-ready answer
I treat retrieved documents, web pages, emails, and tool output as data rather than trusted instructions. The runtime separates instruction channels, minimizes available tools and credentials, validates every action against policy and user intent, and requires approval for sensitive effects. Detection can add signals, but prevention depends on capability isolation because no prompt can reliably neutralize every adversarial string.
Common mistakes
Short answer
Separate planning from execution, use least-privilege credentials, validate intent and parameters, preview effects, require approval, add idempotency and transaction controls, and maintain auditable logs. Prefer reversible staged operations.
Interview-ready answer
The model may draft an action, but a policy layer evaluates actor, resource, scope, risk, and current user confirmation. For destructive or financial operations I use a two-phase flow: generate a human-readable preview, obtain explicit approval bound to exact parameters, then execute with a short-lived token or idempotency key. Where possible, actions enter a reversible pending state and a deterministic service enforces limits.
Common mistakes
Short answer
Require approval when actions are high-impact, irreversible, costly, externally visible, legally sensitive, ambiguous, or outside previously granted scope. Approval should show exact effects and expire if the action changes.
Interview-ready answer
I derive approval policy from risk rather than adding humans to every step. Read-only and reversible low-risk actions can proceed within delegated scope, while payments, deletion, external communication, permission changes, and sensitive data disclosure need explicit confirmation. The approval record includes actor, action, parameters, preview, timestamp, and policy version, and any material parameter change invalidates it.
Common mistakes
Short answer
Enforce tenant and user scope in every server-side data and tool operation, use least-privilege credentials, avoid model-controlled identifiers as authorization, and propagate trusted identity through traces, retrieval filters, caches, and memory stores.
Interview-ready answer
Authentication establishes the actor, but each resource access still needs authorization based on tenant, role, ownership, and action. The runtime injects trusted scope into tool calls rather than accepting it from model output. Retrieval indexes, memory, caches, background jobs, and logs must preserve the same boundary. I add row-level security where available, per-tenant encryption or keys for sensitive domains, audit trails, and adversarial tests that attempt cross-tenant references.
Common mistakes
Short answer
A production architecture needs an API and identity boundary, orchestrator, model gateway, tool registry, state and memory stores, retrieval, policy and approval controls, sandboxing where needed, observability, evaluation, and reliable job execution.
Interview-ready answer
I draw clear boundaries: the orchestrator owns state transitions and budgets; the model gateway handles providers, versions, retries, and usage; tools expose typed capabilities behind authorization; storage separates durable records from working state; and policy gates sensitive actions. Tracing and evaluation wrap every component. Queues, idempotency, cancellation, and dead-letter handling matter for long-running work, while fallbacks define how the system degrades when a model or dependency fails.
Common mistakes
Short answer
Use an agent when the path cannot be enumerated reliably and decisions benefit from language understanding or adaptive tool choice. Use a deterministic workflow when steps, rules, and failure handling are known and repeatability is more valuable.
Interview-ready answer
I prefer deterministic software for payments, authorization, compliance, and stable business processes. An agent is useful at uncertain boundaries such as interpreting a request, selecting among many information sources, or adapting a research plan. Hybrid systems are common: the agent proposes or routes, while a workflow executes validated states. The decision should consider error cost, observability, latency, evaluation coverage, and how often the process changes.
Common mistakes
Short answer
Use authenticated session state for active turns, a consent-aware durable memory service for verified facts and preferences, retrieval scoped to the user, summarization for context budgets, and user controls for inspection and deletion.
Interview-ready answer
The request loads recent session state and retrieves a small set of relevant durable memories using user, task, freshness, and sensitivity filters. The orchestrator builds context, runs the agent, and records trace events. After the response, a memory pipeline proposes candidate facts, deduplicates and classifies them, and writes only those allowed by policy or confirmed by the user. Every record has provenance, timestamps, version, and deletion semantics, and evaluation includes stale and cross-user memory attacks.
Common mistakes
Short answer
Run generated code in ephemeral isolated sandboxes with strict CPU, memory, time, filesystem, process, and network limits. Use allow-listed dependencies, no production credentials, sanitized inputs and outputs, and complete execution auditing.
Interview-ready answer
The orchestrator submits code and test fixtures to a sandbox service through a typed job API. Each job receives a fresh image, non-root user, read-only base filesystem, temporary workspace, seccomp or equivalent process restrictions, and no network unless an explicit allow list is required. Secrets never enter the sandbox. A supervisor enforces wall-clock and resource limits, captures bounded output, destroys the environment, and returns a signed result. Abuse detection and queue isolation protect the platform itself.
Common mistakes
Short answer
Use a deterministic orchestrator that assigns bounded roles, validates messages, tracks shared state, enforces per-agent and global budgets, and owns retries and termination. Agents should not recursively create uncontrolled work.
Interview-ready answer
I would model the workflow as a state machine or durable job graph rather than free-form agent conversation. Each worker receives a typed task and scoped tools, returns a structured result, and emits trace events. The orchestrator applies idempotency, deadlines, retry policies by failure class, concurrency limits, and a global cost budget. Shared artifacts live in versioned storage rather than prompts. A verifier can assess outputs before progression, and terminal states include success, partial completion, blocked, timeout, and policy failure.
Common mistakes
Short answer
Prompt engineering is the practice of designing and refining inputs to a language model to produce reliable, structured, and high-quality outputs. It is critical because a prompt determines whether the model follows instructions, formats output correctly, stays grounded in context, and resists adversarial content.
Interview-ready answer
I treat prompts as a runtime configuration layer, not a substitute for application logic. The system prompt defines immutable rules and safety constraints, while user-facing prompts handle the specific task. I iterate on prompts through structured evaluation rather than intuition: define success criteria, collect representative inputs, measure against a rubric, and version-control the prompt alongside code. A good prompt is concise, unambiguous, and testable — and it degrades predictably when inputs drift.
Common mistakes
Short answer
Zero-shot prompting asks the model to perform a task without examples. One-shot provides a single example, and few-shot provides multiple examples in the prompt to demonstrate the pattern, format, and reasoning style expected.
Interview-ready answer
I choose the shot count based on task ambiguity and output structure. Zero-shot works well for common tasks with unambiguous outputs. Few-shot helps with niche formats or complex reasoning — but each example consumes context tokens and can bias the model toward patterns in those examples. I place examples after the instruction and before the real input, ensure they cover typical and edge-case variations, and keep the total under 5–10 examples unless retrieval-augmented selection is used.
Common mistakes
Short answer
Chain-of-thought prompting asks the model to show intermediate reasoning before giving the final answer. It improves accuracy on arithmetic, logic, and multi-step tasks by exposing the model's reasoning to later steps and making partial credit and error localization possible.
Interview-ready answer
I use CoT for tasks that benefit from explicit reasoning steps such as math, multi-hop retrieval, and complex classification. For simple or well-structured tasks, CoT adds latency and token cost without measurable gain. Variants like self-consistency run multiple CoT chains and vote on the final answer, which improves reliability at higher cost. I evaluate whether CoT actually helps the specific task rather than applying it by default.
Common mistakes
Short answer
Provide an explicit schema or format specification in the prompt, include one or two valid examples, use constrained decoding or JSON-mode features where available, and validate outputs against the schema after generation with clear error handling.
Interview-ready answer
I specify the required structure upfront using the target format itself — JSON schema, XML DTD, or markdown template — rather than describing it in prose. I include a compact example, use the model's structured-output mode when available, and always validate the parsed result against a schema. For production systems, a retry-with-feedback loop lets the model fix minor formatting errors without manual intervention. I also test with malformed inputs to confirm the format constraint holds under adversarial conditions.
Common mistakes
Short answer
Define task-specific metrics, build a representative evaluation set, score outputs against a rubric, version prompts with code, and use A/B comparisons before deployment. Treat prompt evaluation as a continuous process, not a one-time review.
Interview-ready answer
I build a labeled evaluation set that covers typical inputs, edge cases, adversarial inputs, and known failure modes. Each test case has a rubric or expected output. For each prompt version, I run the eval set, compute pass rates by slice, and compare against the baseline. Automated scoring can use exact match, schema validation, or a model judge calibrated against human labels. Prompts are versioned alongside application code, and a regression in any slice blocks deployment. This turns prompt engineering from craft into measurable engineering.
Common mistakes
Short answer
Fine-tuning updates model weights on a domain-specific dataset to improve performance on a targeted task. Use it when prompt engineering and RAG cannot achieve the needed accuracy, latency, or cost profile and you have sufficient high-quality labeled data.
Interview-ready answer
I consider fine-tuning only after prompt engineering and retrieval-based approaches have been pushed to their limits. Fine-tuning can reduce prompt length, lower latency, and improve adherence to domain conventions. But it requires labeled data, risks catastrophic forgetting, introduces model-serving complexity, and may not fix knowledge-gap issues that RAG handles more naturally. I prefer PEFT methods to start, and I evaluate the fine-tuned model against the pre-trained baseline on both target and general capabilities.
Common mistakes
Short answer
LoRA (Low-Rank Adaptation) freezes the original weights and injects trainable low-rank matrices into attention layers. This reduces memory and storage requirements dramatically while maintaining most of the quality of full fine-tuning.
Interview-ready answer
LoRA decomposes the weight update into two low-rank matrices whose product approximates the delta. Training updates only these small adapters, which typically reduces trainable parameters by 10,000x and GPU memory by 2–3x. At inference, the adapters can be merged into the base weights or loaded separately, making it practical to serve many fine-tuned variants from one base model. I choose rank based on task complexity — rank 8 to 16 for most tasks, 32 to 64 for more complex domains.
Common mistakes
Short answer
Catastrophic forgetting occurs when the model loses previously learned capabilities after fine-tuning on a narrow domain. Prevent it with mixed training that includes general data, elastic weight consolidation, replay buffers, LoRA adapters that preserve base weights, and thorough evaluation on general benchmarks.
Interview-ready answer
I mitigate forgetting by reserving 10–20% of each training batch for general-domain data, using LoRA or other PEFT methods that preserve full base weights, and tracking evaluation metrics on both target and general benchmarks throughout training. If general performance drops unacceptably, I reduce learning rate, increase general-data proportion, or switch to a larger base model with more capacity. The acceptance threshold depends on whether the fine-tuned model needs strong general capabilities.
Common mistakes
Short answer
Use prompt engineering first for rapid iteration. Use RAG when the answer depends on external, updateable, or access-controlled knowledge. Use fine-tuning when the task requires a specific behavior, style, or format that cannot be achieved through prompting alone and you have sufficient labeled data.
Interview-ready answer
Each technique addresses a different bottleneck. Prompt engineering changes how the model uses its existing knowledge. RAG supplies new or private information without weight changes. Fine-tuning modifies the model's behavior, tone, or domain fluency. In production these often complement each other: RAG supplies current facts while fine-tuning adapts the model's domain conventions. I start with the cheapest lever — prompt engineering — and add complexity only when metrics show a clear gap.
Common mistakes
Related
Short answer
RLHF (Reinforcement Learning from Human Feedback) aligns model outputs with human preferences through a three-stage process: supervised fine-tuning on demonstrations, training a reward model from human comparisons, and optimizing the policy with PPO or a similar algorithm against the reward model.
Interview-ready answer
RLHF addresses the gap between language-modeling objectives and helpful, harmless behavior. The reward model learns what humans prefer from pairwise comparisons, then the policy is tuned to maximize that reward while staying near the SFT initialization via a KL penalty. This reduces harmful outputs but can also reduce diversity and cause the model to exploit reward-model blind spots. Direct preference optimization (DPO) simplifies the pipeline by treating preference data as the direct training signal. I evaluate alignment interventions on safety, helpfulness, and capability retention.
Common mistakes
Short answer
LLMOps is the practice of managing the lifecycle of LLM-powered applications — prompt management, inference serving, cost tracking, guardrails, evaluation, and monitoring. It differs from MLOps by emphasizing prompt-driven behavior, API-based model access, token costs, and qualitatively different failure modes such as hallucination and prompt injection.
Interview-ready answer
Traditional MLOps focuses on training, deploying, and monitoring custom models. LLMOps shifts focus to managing pre-trained model APIs or self-hosted inference — prompt versioning, structured output handling, latency and token budgets, content safety, and model fallbacks. Evaluation moves from accuracy on a held-out set to semantic quality, grounding, safety, and user satisfaction. The operational surface is narrower in some ways — you do not retrain models — but broader in others because model behavior is less predictable and harder to pin down.
Common mistakes
Short answer
Monitor latency (TTFT, inter-token latency, end-to-end), token and cost usage, error rates (timeouts, rate limits, invalid responses), safety guardrail activations, output quality via sampling and model judges, and business metrics such as completion rate and user satisfaction.
Interview-ready answer
I set up three monitoring layers. Operational metrics — latency percentiles, throughput, error codes, and cost — catch infrastructure issues. Safety metrics — guardrail hit rates, content-filter triggers, and injection-detection alerts — catch abuse and policy violations. Quality metrics — sampled output review, model-judge scores, and user feedback — catch behavioral regressions. Traces link these layers so I can investigate a slow, costly, or low-quality response back to the specific prompt, retrieval, or tool call that caused it.
Common mistakes
Short answer
Reduce cost through prompt compression, shorter system prompts, model routing (cheap model for simple tasks, expensive one for complex), caching (exact and semantic), batching, quantization, and smaller models for subtasks where quality allows.
Interview-ready answer
I start with observability to understand where tokens are spent: long system prompts, verbose model responses, unnecessary retrieval, or expensive model calls for simple tasks. Then I apply targeted optimizations: prompt compression and shorter instructions, semantic caching for repeated queries, model routing based on task complexity, and quantization for self-hosted models. Each optimization must be evaluated against the quality baseline — I never reduce cost blindly. For high-volume applications, a 10% prompt-length reduction at constant quality is often the easiest win.
Common mistakes
Short answer
Pin model versions in configuration, deploy new versions to a shadow or canary audience first, evaluate on quality and safety metrics, and maintain the ability to roll back to the previous version instantly. Version prompts and system configurations alongside model versions.
Interview-ready answer
I treat model deployments like any software deployment: version-pinned, gated by evaluation, and reversibly rolled out. A new model version is evaluated against the current baseline on a held-out evaluation set covering quality, safety, latency, and cost. If it passes, it goes to a canary traffic slice while the production version serves the rest. Metrics are compared, and the rollback flips a configuration toggle rather than rebuilding infrastructure. Prompt and tool catalog versions are locked to model versions because a prompt tuned for GPT-4 may behave differently on Claude or a fine-tuned variant.
Common mistakes
Short answer
Design a layered fallback: retry with backoff, switch to a cheaper or smaller model, degrade to a retrieval-only answer, serve a cached response, or show a graceful error with clear user communication. Each layer has a cost and quality profile.
Interview-ready answer
I build a fallback chain with clear degradation semantics. On the primary model failure, retry with exponential backoff and jitter up to a budget. Then fall back to a secondary model (different provider or smaller variant) that can produce an acceptable response. If all model calls fail, serve the best cached response for the query or a retrieval-only summary. If nothing works, return a well-designed fallback UI — no raw error messages. Each stage emits a trace event so I can monitor fallback rates and adjust budgets or add capacity where fallbacks trigger too often.
Common mistakes
Short answer
The Transformer is a neural architecture that processes sequences using self-attention rather than recurrence. Its key components are multi-head attention, feed-forward networks, positional encoding, layer normalization, and residual connections. It excels at parallelising over sequence positions, making it efficient to train on large data.
Interview-ready answer
I would describe the Transformer as an encoder-decoder or decoder-only stack where every token can attend to every other token through self-attention. The attention mechanism computes queries, keys, and values from the input, then uses scaled dot-product attention to aggregate information across positions. Multi-head attention runs several attention copies in parallel so the model can learn different relationship types. Feed-forward layers add per-token nonlinear transformations, while residual connections and layer norm keep training stable. Positional encoding — either learned or sinusoidal — gives the model information about token order since attention itself is permutation invariant.
Common mistakes
Short answer
Each input token produces a query, key, and value vector. Queries and keys determine pairwise attention scores through a dot product, which are then normalised with softmax. The resulting weights are used to compute a weighted sum of the value vectors, producing the attention output for each token.
Interview-ready answer
I think of Q, K, V as a content-based lookup. The query represents what the current token is looking for, the key represents what each token offers, and the value is the information that will be passed along if a match is found. The attention score is the dot product of query and key, scaled by the inverse square root of the dimension to prevent vanishing gradients in the softmax. The weighted sum of values lets each token incorporate context from positions that have high attention scores, making the representation context-aware.
Common mistakes
Short answer
Tokenization converts raw text into integer tokens that the model can process. BPE (Byte Pair Encoding) iteratively merges the most frequent adjacent byte pairs in the training corpus to build a fixed-size vocabulary of subword units, balancing vocabulary size against coverage of rare words.
Interview-ready answer
Tokenisation bridges raw strings and model embeddings. BPE starts with individual bytes or characters as the base vocabulary, then counts adjacent token pairs in the corpus. The most frequent pair is merged into a new token, and the process repeats until the desired vocabulary size is reached. This lets the model handle any input via subword composition while keeping common words as single tokens. In practice, I verify that important domain terms are not split into meaningless pieces and consider adding domain-specific tokens to reduce sequence length and inference cost.
Common mistakes
Short answer
KV cache stores the key and value tensors from earlier attention computations during autoregressive generation. Since each new token only needs to attend to all previous tokens, recomputing all keys and values from scratch is wasteful. Caching them reduces the per-step computation from O(n²) to O(n) and significantly lowers latency.
Interview-ready answer
In autoregressive generation, each new token's attention step still needs keys and values from every prior position. Without KV cache, the model would recompute those tensors for every prefix position on every step. By caching them in memory after the first computation, each subsequent step only computes Q, K, V for the new token, then uses the full KV from the cache for attention. The trade-off is increased memory usage — the cache grows linearly with sequence length — which is why techniques like Paged Attention, sliding-window cache, or quantised caching are important for long sequences.
Common mistakes
Short answer
Temperature scales the logit distribution before softmax — lower values sharpen the distribution toward the most likely token, higher values flatten it for more randomness. Top-k limits sampling to the k highest-probability tokens, while top-p (nucleus) sampling selects the smallest set of tokens whose cumulative probability exceeds p. They are often combined to control creativity and coherence.
Interview-ready answer
Temperature divides logits by the temperature value before applying softmax. At low temperatures the probability mass concentrates on the top token, making output deterministic and repetitive. At high temperatures the distribution becomes more uniform, increasing diversity but risking incoherence. Top-k sets a fixed cutoff so long-tail tokens never get sampled, while top-p adapts the cutoff based on the distribution shape — narrow distributions keep fewer candidates, broad distributions keep more. In production I use top-p with a moderate temperature and tune the combination against the specific task, measuring both quality and diversity metrics.
Common mistakes
Learn
Use Curriculum for structured concept guides and deeper explanations.
Revise
Use Interview Q/A for concise spoken answers and production trade-offs.
Practice
Use Problems to implement the patterns and inspect execution feedback.