LLM interviews test whether you can ship a model you don't own

An LLM demo can survive a meeting while the system behind it is falling apart. The interview begins when someone asks how you know an answer is grounded, what you log after a prompt injection and why latency doubled after retrieval was added. A folder full of clever prompts gives you very little cover at that point.

The LLM / GenAI track follows the material these conversations tend to reach. It covers prompting and context design, transformers and tokenization, pre-training, alignment, SFT and PEFT, RAG, tools, evaluation, hallucination, security, quantization and inference. There is also a staff-level session on MoE, multimodal systems, reasoning models and model merging. Choose it only when the job calls for that depth.

Prompting is part of system design

Prompt questions should lead to versioning and measurement. Explain when you would use examples, how you would structure instructions and what happens when the context grows crowded. Then say how a prompt change moves through review and an eval set before it reaches users. Tuesday’s wording tweak should not quietly damage a workflow that passed on Friday.

Context engineering is about selecting and ordering information under a finite budget. An 80-page PDF will not fit usefully just because the model accepts a large window. Discuss what gets retrieved, what gets summarized, how instructions are separated from untrusted content and which material is dropped first when the request becomes too large.

The fundamentals still matter. Tokenization can explain odd costs, truncated inputs and uneven multilingual behavior. Attention questions may ask you to place Q, K and V, then move quickly to memory and long-context limits. A candidate who only knows product APIs often discovers the missing layer when the interviewer asks why the same text behaves differently after tokenization.

Retrieval fails in ordinary ways

Retrieval usually fails in an ordinary place. The relevant paragraph ranked 14th, a chunk split the needed sentence, metadata filters excluded the document or the index was stale. Begin with the retrieval trace and inspect queries, candidate documents, scores and the context finally sent to the model. That diagnostic path runs through the RAG Pipeline Design interview, where every missing passage needs a cause.

Advanced retrieval adds hybrid search, query rewriting and re-rankers. More chunks can make an answer worse by crowding the context with near matches. Explain the experiment you would run before changing the whole stack. One carefully labeled set of difficult queries is often more useful than another diagram with arrows moving between boxes.

Tool calling has similar operational edges. Schemas drift, arguments fail validation, tools time out and the returned data can be wrong. Describe retries, idempotency and permission boundaries. Structured output puts a JSON shape around the response, but consumers still need to handle refusals, missing fields and output that passes syntax while carrying bad meaning.

Evaluation ends arguments with evidence

Evaluation starts with the job the application must complete. For a support assistant, that may include groundedness, resolution, safe refusal and cost per successful interaction. A coding assistant or extraction system needs a different set, and the LLM Evaluation interview keeps those measures attached to the product choice. Generic benchmark scores cannot answer whether this application helped this user with this policy document.

Judge models are useful and imperfect. They can prefer longer answers, inherit model biases or reward wording that resembles their rubric. Explain how you would calibrate a judge against human labels and inspect disagreements. If a candidate says “we used an LLM judge” and stops, the interviewer has several productive follow-ups waiting.

Fabrication is a product incident. Retrieval may miss, context can conflict and the model will still produce fluent text. The Hallucination and Reliability interview carries that failure into the choice to cite sources, refuse, ask a clarifying question or route work to a human. An instruction telling the model to avoid fabrication is a wish unless evaluation and runtime behavior support it.

Security and cost belong in the design

Prompt injection matters whenever untrusted content enters the context. A webpage, PDF or email can contain instructions aimed at the model. If the system can call tools, the damage can extend beyond a poor answer. Separate data from instructions, limit tool permissions, confirm sensitive actions and log enough context to investigate abuse.

Inference questions usually arrive as a trade between quality, latency and cost. Discuss model size, batching, KV cache, quantization and speculative decoding only as far as your experience supports. If you have never tuned a custom kernel, say that. A sensible measurement plan beats an invented CUDA adventure that falls apart under 2 follow-ups.

Roles that train models will go further into data curation, distributed training and scaling behavior. Alignment sessions cover RLHF, DPO and GRPO. SFT and PEFT questions ask when adapters are enough, when full fine-tuning is justified and how training data stays separate from evaluation. A posting centered on prompts, RAG and eval does not need a volunteer lecture on GRPO.

Draw the system you have shipped

Prepare one system on a whiteboard. Put the prompt, retrieval, model call, tools, logs and eval path in specific places. Mark the trust boundaries. Then ask what you would remove if latency had to fall, what you would inspect after quality dropped and how you would contain a user attempting injection.

I like this exercise because it exposes invented familiarity quickly, including my own. Every vague box creates a question. “Vector database” becomes chunking, freshness and access control. “Monitoring” becomes the exact event you record and the person who reads it after a bad answer.

By the end of your preparation, you should be able to close the demo tab and keep talking. The LLM / GenAI track can pressure-test the sections your role calls for. Bring a diagram detailed enough to explain where an answer came from, what it cost and who can stop the system when it behaves badly.